Develop Reference

Android HttpUrlConnection: Post Multipart

I'm trying to post a test File to a spring rest servlet deployed on tomcat using Android. I'm developing on Android 4.1.2, but I have verified same problem on 4.0.3.
The problem is that the file upload requires a very long time (about 70 seconds for a 4MB file), also in local network. The time is equiparable using a 3g connection. I've excluded that it could be a server problem: executing the same call with curl it takes 1 / 2 seconds, and using apache as backend results are the same.
Using HttpClient works fine.
I'm using Spring Android RestClient 1.0.1.RELEASE and, given Android version and the fact that I'm not overriding default behaviour, it uses HttpUrlConnection instead of HttpClient to make http requests.
I have also implemented my custom ClientHttpRequestFactory in order to manipulate some details of SSL connection and I have defined my own implementation of ClientHttpRequestInterceptor in order to modify authentication header.
I have also set setBufferRequestBody(false) in order to avoid OutOfMemoryException on big files. But this property have no effects on time required.
MyClientHttpRequestFactory:
public class MyClientHttpRequestFactory extends SimpleClientHttpRequestFactory{
#Override
protected void prepareConnection(HttpURLConnection connection, String httpMethod) throws IOException {
super.prepareConnection(connection, httpMethod);
connection.setConnectTimeout(240 * 1000);
connection.setReadTimeout(240 * 1000);
if ("post".equals(httpMethod.toLowerCase())) {
setBufferRequestBody(false);
}else {
setBufferRequestBody(true);
}
}
#Override
protected HttpURLConnection openConnection(URL url, Proxy proxy) throws IOException {
final HttpURLConnection httpUrlConnection = super.openConnection(url, proxy);
if (url.getProtocol().toLowerCase().equals("https")
&&
settings.selfSignedCert().get())
{
try {
((HttpsURLConnection)httpUrlConnection).setSSLSocketFactory(getSSLSocketFactory());
((HttpsURLConnection)httpUrlConnection).setHostnameVerifier(new NullHostnameVerifier());
} catch (Exception e) {
MyLog.e(LOG_TAG, "OpenConnection", e);
}
}
return httpUrlConnection;
}
MyClientHttpRequestInterceptor:
public class MyClientHttpRequestInterceptor implements ClientHttpRequestInterceptor {
#Override
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
final HttpHeaders headers = request.getHeaders();
headers.setAuthorization(new HttpBasicAuthentication( settings.username().get(), settings.password().get()));
if (settings.enable_gzip().get()) {
headers.setAcceptEncoding(ContentCodingType.GZIP);
}
return execution.execute(request, body);
}
}
And here my Rest call:
List<ClientHttpRequestInterceptor> interceptors = Arrays.asList((ClientHttpRequestInterceptor)myClientHttpRequestInterceptor);
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(new FormHttpMessageConverter());
restTemplate.getMessageConverters().add(new StringHttpMessageConverter());
restTemplate.getMessageConverters().add(new MappingJackson2HttpMessageConverter());
restTemplate.setInterceptors(interceptors);
MultiValueMap<String, Object> parts = new LinkedMultiValueMap<String, Object>();
parts.add("file", new FileSystemResource("/sdcard/test/4MB_file"));
HttpEntity<MultiValueMap> requestEntity = new HttpEntity<MultiValueMap>(parts);
restTemplate.exchange(myUrl, HttpMethod.POST, requestEntity, Integer.class).getBody();
}
Looking at Spring Android source code, the next lines of code my request is passing through are:
public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException {
HttpURLConnection connection = openConnection(uri.toURL(), this.proxy);
prepareConnection(connection, httpMethod.name());
if (this.bufferRequestBody) {
return new SimpleBufferingClientHttpRequest(connection);
} else {
return new SimpleStreamingClientHttpRequest(connection, this.chunkSize);
}
}
Because of this.bufferRequestBody is false, return new SimpleStreamingClientHttpRequest(connection, this.chunkSize); is executed (with chunkSize = 0)
SimpleStreamingClientHttpRequest(HttpURLConnection connection, int chunkSize) {
this.connection = connection;
this.chunkSize = chunkSize;
// Bugs with reusing connections in Android versions older than Froyo (2.2)
if (olderThanFroyo) {
System.setProperty("http.keepAlive", "false");
}
}
and then:
ClientHttpRequest delegate = requestFactory.createRequest(request.getURI(), request.getMethod());
delegate.getHeaders().putAll(request.getHeaders());
if (body.length > 0) {
FileCopyUtils.copy(body, delegate.getBody());
}
return delegate.execute();
From here is all android subsystem I think..
I have dumped tcp traffic and analyzed it:
POST /urlWherePost HTTP/1.1
Content-Type: multipart/form-data;boundary=nKwsP85ZyyzSDuAqozCTuZOSxwF1jLAtd0FECUPF
Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxxxx=
Accept-Encoding: gzip
User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.1.2; sdk Build/MASTER)
Host: 192.168.168.225:8080
Connection: Keep-Alive
Content-Length: 4096225
--nKwsP85ZyyzSDuAqozCTuZOSxwF1jLAtd0FECUPF
Content-Disposition: form-data; name="file"; filename="4MB_file"
Content-Type: application/octet-stream
Content-Length: 4096000
I've tryed to re-create similar request with curl:
curl --verbose
-H "Connection: Keep-Alive"
-H "Content-Type: multipart/form-data"
-H "Accept-Encoding: gzip"
-H "Content-Disposition: form-data; name=\"file\"; filename=\"4MB_file\""
-H "Content-Type: application/octet-stream"
--user xxx:xxx
-X POST
--form file=#4MB_file
http://192.168.168.225:8080/urlWherePost
but with curl the post is ok.
Posting json data is not a problem (maybe small body size). But when I try to send "big" files the time increase.
Looking in DDMS shell, on Network Statistics I've also found that the network throughput is never over 250kb in TX. There seems to be a bootleneck, but how to investigate it? Where I can look, which parameter can I change?
Thank you for any suggestion!

Have you tried using the MultipartEntity method? I had the same problem when downloading big amounts of JSON data from the server, but I switched to this method and caught all the data that the server provided me.
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost("http://myurl.com");
try {
MultipartEntity entity = new MultipartEntity();
entity.addPart("type", new StringBody("json"));
entity.addPart("data", new JSONObject(data));
httppost.setEntity(entity);
HttpResponse response = httpclient.execute(httppost);
} catch (ClientProtocolException e) {
} catch (IOException e) {
} catch (JSONException e){
}

To upload big files, You can use this library android-async-http

For simple and easy to use, I recommend this lib https://github.com/koush/ion.
I use it on my project and it works perfectly.

Multipart/form-data construction with android

I am trying to make an HttpPost with multiPart/form-data via my android app. I have a postman test that is working with my api and the preview of that request in postman, looks like this:
POST /api/0.1/content/upload HTTP/1.1
Host: 54.221.194.167
X-AUTHORIZATION: 166e649911ff424eb14446cf398bd7d6
Cache-Control: no-cache
Postman-Token: 2412eba9-f72d-6f3b-b124-6070b5b26644
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="file01"
{"mime_type":"image/jpeg","title":"IMG_20140131_111622"}
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="file01"; filename="addedaslib.jpg"
Content-Type: image/jpeg
----WebKitFormBoundaryE19zNvXGzXaLvS5C
I am trying to replicate that using the multipart/form-data with my android HttpPost but it doesn't seem to be working. Is there a way to "preview" my request and see how it is actually posting to the api? what am i doing wrong? my code:
public HttpResponse invokeXAUTHPOSTService(String url, String token, File file) {
client = new DefaultHttpClient();
HttpPost request = new HttpPost(url);
HttpResponse response = null;
MultipartEntity mpe = new MultipartEntity();
try {
Log.v("API", "URL:"+url);
request.setHeader("Content-Type", "multipart/form-data");
request.addHeader("X-AUTHORIZATION",token);
request.addHeader("Cache-Control", "no-cache");
DRPContentForUpload content = new DRPContentForUpload(file);
String jsonObject = DRPJSONConverter.toJson(content);
FormBodyPart part1= new FormBodyPart("file01", new StringBody(jsonObject));
FormBodyPart part2= new FormBodyPart("file01", new FileBody(file));
mpe.addPart(part1);
mpe.addPart(part2);
//
request.setEntity(mpe);
Log.v("RAW REQUEST", "request looks like:"+mpe.toString());
response = client.execute(request);
EDIT
I was able to talk to my API team, and they said my post actually looks like this:
--0ieMJK6PPwcrM_K3KQvl6eNDGqooZPzJcvHOm0
Content-Disposition: form-data; name="file01"
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
{"mime_type":"image/jpeg","title":"IMG_20140131_111622"}
--0ieMJK6PPwcrM_K3KQvl6eNDGqooZPzJcvHOm0
Content-Disposition: form-data; name="file01"; filename="IMG_20140131_111622.jpg"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
That said its still not working and spitting back an error that i am missing params
here is a screen shot of the libraries included in my project:

So after searching high and low for an answer, and almost giving up, this link finally helped
here is the final working code:
public HttpResponse invokeXAUTHPOSTService(String url, String token, File file) {
client = new DefaultHttpClient();
HttpPost request = new HttpPost(url);
HttpResponse response = null;
DRPContentForUpload content = new DRPContentForUpload(file);
String jsonObject = DRPJSONConverter.toJson(content);
String BOUNDARY= "--eriksboundry--";
request.setHeader("Content-Type", "multipart/form-data; boundary="+BOUNDARY);
request.addHeader("X-AUTHORIZATION",token);
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE,BOUNDARY,Charset.defaultCharset());
try {
entity.addPart("file01", new StringBody(jsonObject));
entity.addPart("file01", new FileBody(file));
request.addHeader("Accept-Encoding", "gzip, deflate");
} catch (UnsupportedEncodingException e) {
Log.v("encoding exception","E::: "+e);
e.printStackTrace();
}
request.setHeader("Accept", "application/json");
request.setHeader("Content-Type", "multipart/form-data; boundary="+BOUNDARY);
request.setEntity(entity);
try {
response = client.execute(request);
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return response;
}

Is the problem that the mime-type for your image isn't being set properly? Try to use the alternative constructor for FileBody:
FormBodyPart part2 = new FormBodyPart("file01", new FileBody(file, "image/jpeg"));
Giving both parts of the multi-part request the same name is a bit suspicious, but it doesn't seem to be causing you problems here.
If you want to see what your body looks like before sending, try using EntityUtils.toString(Entity) on the multi-part entity after adding all of the elements. The image is simply binary so won't print very well, but the headers should be readable.

How to send a JSON object over POST Request with Android

I am trying to build a small application in which the application will communicate with a php script with the help of JSON objects. I successfully implemented the GET Request test application but using JSON with post is creating problems. The code generates no error but my php script reply with no nothing except an empty Array() which implies that nothing was sent over the connection with code:
<?php print_r($_REQUEST); ?>
and trying with
<?php print($_REQUEST['json']); ?>
throws HTML back to the application with json variable not found error.
I have already tried a few solutions mentioned here including: How to send a JSON object over Request with Android? and How to send a json object over httpclient request with android so it would be great if you can point out my mistake and can briefly describe what exactly I was doing wrong. Thanks.
Here is the code snippet for from where the JSON Object is converted into string and then attached to a Post variable.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost httppostreq = new HttpPost(wurl);
StringEntity se = new StringEntity(jsonobj.toString());
se.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
httppostreq.setEntity(se);
//httppostreq.setHeader("Accept", "application/json");
//httppostreq.setHeader("Content-type", "application/json");
//httppostreq.setHeader("User-Agent", "android");
HttpResponse httpresponse = httpclient.execute(httppostreq);
HttpEntity resultentity = httpresponse.getEntity();
Here is TCP Stream Dump collected through wireshark if it can help:
POST /testmysql.php?test=true HTTP/1.1
Content-Length: 130
Content-Type: application/json;charset=UTF-8
Content-Type: application/json;charset=UTF-8
Host: 192.168.100.4
Connection: Keep-Alive
User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
{"weburl":"hashincludetechnology.com","header":{"devicemodel":"GT-I9100","deviceVersion":"2.3.6","language":"eng"},"key":"value"}HTTP/1.1 200 OK
Date: Mon, 30 Apr 2012 22:43:10 GMT
Server: Apache/2.2.17 (Win32)
Content-Length: 34
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
Array
(
[test] => true
)
Test // echo statement left intentionally.

you are using PHP on the server side, so your HTTP entity must be a multipart encoded one. See this link. You are using a string entity, but this is not correct. It must be a MultipartEntity, which emulates what the browser does when you submit a form in a web page. MultipartEntity should be in httpmime jar.
Once you have your multipart entity, simply add a Part named "json", and set its contents to the string representation of your json-encoded object.
Note that this answer is because you use PHP on the server side, so you must use its "protocol" to read variables via $_REQUEST. If you used your own request parser oh the server side, even a StringEntity could be ok. See HTTP_RAW_POST_DATA

The below should work. Make sure to set the appropriate keys for what your form post is expecting at the top. Also I included how to send an image as well as other various json data, just delete those lines if that is not necessary.
static private String postToServerHelper(
String action,
JSONObject jsonData,
byte[] imageData){
// keys for sending to server
/** The key for the data to post to server */
final String KEY_DATA = "data";
/** The key for the action to take on server */
final String KEY_ACTION = "action";
/** The return code for a successful sync with server */
final int GOOD_RETURN_CODE = 200;
/** The key for posting the image data */
final String KEY_IMAGE = "imageData";
/** The image type */
final String FILE_TYPE = "image/jpeg";
/** The encoding type of form data */
final Charset ENCODING_TYPE = Charset.forName("UTF-8");
// the file "name"
String fileName = "yourFileNameHere";
// initialize result string
String result = "";
// initialize http client and post to correct page
HttpClient client = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://www.yourdomain.com/yourpage.php");
// set to not open tcp connection
httpPost.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
// build the values to post, the action and the form data, and file data (if any)
MultipartEntity multipartEntity = new MultipartEntity(
HttpMultipartMode.BROWSER_COMPATIBLE);
try{
multipartEntity.addPart(KEY_ACTION, new StringBody(action, ENCODING_TYPE));
multipartEntity.addPart(KEY_DATA, new StringBody(jsonData.toString(), ENCODING_TYPE));
if (imageData != null){
multipartEntity.addPart(KEY_IMAGE, new ByteArrayBody(imageData, FILE_TYPE, fileName));
}
}catch (Exception e){
return e.getMessage();
}
// set the values to the post
httpPost.setEntity(multipartEntity);
int statusCode= -1;
// send post
try {
// actual send
HttpResponse response = client.execute(httpPost);
// check what kind of return
StatusLine statusLine = response.getStatusLine();
statusCode = statusLine.getStatusCode();
// good return
if (statusCode == GOOD_RETURN_CODE) {
// read return
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(content));
String line;
while ((line = reader.readLine()) != null) {
builder.append(line + "\n");
}
content.close();
result = builder.toString();
// bad return
} else {
return String.parse(statusCode);
}
// different failures
} catch (ClientProtocolException e) {
return e.getMessage();
} catch (IOException e) {
return e.getMessage();
}
// return the result
return result;
}

DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
JSONObject clientList = new JSONObject ();
clientList.put("name","");
clientList.put("email","");
clientList.put("status","");
clientList.put("page","");
JSONObject listclient = new JSONObject ();
listclient.put("mydetail", clientList);
//--List nameValuePairs = new ArrayList(1);
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("token", tokenid));
nameValuePairs.add(new BasicNameValuePair("json_data", listclient.toString()));
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
Log.d("JSON",nameValuePairs.toString());
//-- Storing Response
HttpResponse httpResponse = httpClient.execute(httpPost);

Android HttpPost failes - server receives null query string parameter

I have a simple WCF web service on my machine which I have developed to serve Android and IOS devices.
The service has a single method as following :
[OperationContract]
[WebInvoke(RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Wrapped, UriTemplate = "/?message={message}")]
string ServiceMessage(string message);
I have 3 clients , one .NET test client using HttpWebRequest which works fine , one IOS client which works fine and one Android client which I have developed with the HttpPost and HttpClient classes that fails.
Using Fiddler reverse proxy I have debugged the output of the .net client :
POST http://127.0.0.1:8888/Service1.svc/?message=_|JSON MESSAGE BODY|_HTTP/1.1
Content-Type: application/json; charset=utf-8
Host: 127.0.0.1:8888
Content-Length: 338
Expect: 100-continue
Connection: Keep-Alive
_|JSON MESSAGE BODY|_
On the other hand , this is the output of the Android HTTP Post :
POST http://10.0.2.2:8888/Service1.svc/ HTTP/1.1
Content-Type: application/json; charset=utf-8
Content-Length: 139
Host: 10.0.2.2:8888
Connection: Keep-Alive
User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
Expect: 100-Continue
message=_|JSON MESSAGE BODY|_
As you can see , .Net puts the message parameter on the Post at the top and does not
put the message variable name at the bottom while Android does not put the message body at the Post at the top and does put the message variable name at the bottom.
This is my Android post code ,
HttpPost httpPost = new HttpPost(url);
String messageBody = "message=" + jsonMessageParameter;
StringEntity entity = new StringEntity(messageBody);
httpPost.setEntity(entity);
httpPost.setHeader("Content-Type", "application/json; charset=utf-8");
HttpResponse response = hc.execute(httpPost);
InputStream content = response.getEntity().getContent();
String json = ConvertStreamToString(content);
When calling with this code , the server method is called but the message method parameter is null.
I tried playing with the Uri.Builder class to also make the android post put the message at the header , but doesnt quite work.
If can someone help me out here , I am stuck on this for hours over hours.
Thank you in advance ,
James
EDIT :
I changed the Android code to :
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(1);
nameValuePairs.add(new BasicNameValuePair("message", jsonMessageParameter));
HttpPost httpPost = new HttpPost(url);
UrlEncodedFormEntity urlEncodedFromEntity = new UrlEncodedFormEntity(
nameValuePairs);
urlEncodedFromEntity.setContentType(new BasicHeader("Content-Type",
"application/json; charset=utf-8"));
httpPost.setEntity(urlEncodedFromEntity);
InputStream postStream = httpPost.getEntity().getContent();
String postOutput = ConvertStreamToString(postStream);
HttpResponse response = hc.execute(httpPost);
InputStream content = response.getEntity().getContent();
String json = ConvertStreamToString(content);
But still the Fiddler monitoring is as following :
POST http://10.0.2.2:8888/Service1.svc/ HTTP/1.1
Content-Length: 189
Content-Type: application/json; charset=utf-8
Host: 10.0.2.2:8888
Connection: Keep-Alive
User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
Expect: 100-Continue
message=_|MESSAGE_JSON|_

There a number of points here.
First, you specifically added message= to your Android POST body:
String messageBody = "message=" + jsonMessageParameter;
This can be also problematic since you specified Content-Type: application/json but by adding message= you are not providing a valid JSON object.
Second, why does the .Net implementation replicate the JSON object both as parameter in the URL and in the body? This looks strange and very uncommon for a POST request, and can cause problem if your JSON object makes the URL exceed the maximum URL length.
So I would try removing message= in the body and removing the JSON object as a URL parameter, since servers processing POSTs should read the body and not the URL.

HttpPost is for posting, afaik.
String url;
input = url + URLEncoder.encode(messageBody, "utf-8");
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(input);
InputStream stream = httpclient.execute(httpget).getEntity().getContent();
String s = streamToString(stream);
JSONObject jObject = new JSONObject(s);
This very exact code (plus some try/catch) works for me for querying GoogleBooks, probably you can adapt it to your service with little effort?
Best regards.

How to send HTTP POST request and receive response?

I'm going to create mobile application that works with CommuniGate Pro server.
For example, I need to make the following Android Client C - CGP Server S conversation and get XIMSS.nonce node value:
C:GET /ximsslogin/ HTTP/1.1
Host: myserver.com
Content-Type: text/xml
Content-Length: 42
<XIMSS><listFeatures id="list" /><XIMSS>
S:HTTP/1.1 200 OK
Content-Length: 231
Connection: keep-alive
Content-Type: text/xml;charset=utf-8
Server: CommuniGatePro/5.3
<XIMSS><nonce>2C3E575E5498CE63574D40F18D00C873</nonce><language>german</language><response id="s"/></XIMSS>
Example, in ActionScript 3.0 it looks this way:
var loader:Loader = new Loader();
loader.addEventListener(Event.COMPLETE, completeHandler);
var urlRequest:URLRequest = new URLRequest(...);
urlRequest.method = ...;
urlRequest.data = ...;
loader.load(urlRequest);
private function completeHandler(...):void { ... };
How will it look in Java for Android 2.1?

As Schnapple says your question seems very broad and is confusing to read and understand.
Here is some general code to send a HTTP POST and get a response from a server though that may be helpful.
public String postPage(String url, File data, boolean returnAddr) {
ret = null;
httpClient.getParams().setParameter(ClientPNames.COOKIE_POLICY, CookiePolicy.RFC_2109);
httpPost = new HttpPost(url);
response = null;
FileEntity tmp = null;
tmp = new FileEntity(data,"UTF-8");
httpPost.setEntity(tmp);
try {
response = httpClient.execute(httpPost,localContext);
} catch (ClientProtocolException e) {
System.out.println("HTTPHelp : ClientProtocolException : "+e);
} catch (IOException e) {
System.out.println("HTTPHelp : IOException : "+e);
}
ret = response.getStatusLine().toString();
return ret;
}