Develop Reference

Android Http Response is incomplete. Returns an unterminated json object

I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.

Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?

I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.

I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.

android HttpGet incomplete response BufferedReader

Im doing a simple http get,
I see on my result an incomplete response,
what Im doing wrong?
here the code:
class GetDocuments extends AsyncTask<URL, Void, Void> {
#Override
protected Void doInBackground(URL... urls) {
Log.d("mensa", "bajando");
//place proper url
connect(urls);
return null;
}
public static void connect(URL[] urls)
{
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet("http://tiks.document.dev.chocolatecoded.com.au/documents/api/get?type=tree");
// Execute the request
HttpResponse response;
try {
response = httpclient.execute(httpget);
// Examine the response status
Log.d("mensa",response.getStatusLine().toString());
// Get hold of the response entity
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String result= convertStreamToString(instream);
// now you have the string representation of the HTML request
Log.d("mensa", "estratagema :: "+result);
JSONObject jObject = new JSONObject(result);
Log.d("mensa", "resposta jObject::"+jObject);
Log.d("mensa", "alive 1");
JSONArray contacts = null;
contacts = jObject.getJSONArray("success");
Log.d("mensa", "resposta jObject::"+contacts);
Log.d("mensa", "alive");
//instream.close();
}
} catch (Exception e) {}
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
Log.d("mensa", "linea ::"+line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
}
i call it with:
GetDocuments get = new GetDocuments();
URL url = null;
try {
url = new URL("ftp://mirror.csclub.uwaterloo.ca/index.html");
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//URL url = new URL("http://www.google.es");
get.execute(url);
edit 1
I refer to incomplete as the response that gets truncated?
please notice in below image of response how string gets truncated,
is this because of the log size?,
but the other problem is that it doesn't parse?
thanks!

I don't know if this is going to resolve your problem but you can get rid of your method and use simply:
String responseString = EntityUtils.toString(response.getEntity());

I've had exactly the same issue for the last couple of days. I found that my code worked over WiFi but not 3G. In other words I eliminated all the usual threading candidates. I also found that when I ran the code in the debugger and just waited for (say) 10 seconds after client.execute(...) it worked.
My guess is that
response = httpclient.execute(httpget);
is an asynchronous call in itself and when it's slow returns a partial result... hence JSON deserialization goes wrong.
Instead I tried this version of execute with a callback...
try {
BasicResponseHandler responseHandler = new BasicResponseHandler();
String json = httpclient.execute(httpget, responseHandler);
} finally {
httpclient.close();
}
And suddenly it all works. If you don't want a string, or want your own code then have a look at the ResponseHandler interface. Hope that helps.

I have confirmed that this is because size limit of java string. I have checked this by adding the string "abcd" with the ressponse and printed the response string in logcat. But the result is the truncated respose without added string "abcd".
That is
try {
BasicResponseHandler responseHandler = new BasicResponseHandler();
String json = httpclient.execute(httpget, responseHandler);
json= json+"abcd";
Log.d("Json ResponseString", json);
} finally {
httpclient.close();
}
So I put an arrayString to collect the response. To make array, I splitted My json format response by using "}"
The code is given below(This is a work around only)
BasicResponseHandler responseHandler = new BasicResponseHandler();
String[] array=client.execute(request, responseHandler).split("}");
Then you can parse each objects in to a json object and json array with your custom classes.
If you get any other good method to store response, pls share because i am creating custom method for every different json responses );.
Thank you
Arshad

Hi Now I am using Gson library to handle the responses.
http://www.javacodegeeks.com/2011/01/android-json-parsing-gson-tutorial.html
Thanks
Arshad

I cant' comment directly due to reputation, but in response to https://stackoverflow.com/a/23247290/4830567 I felt I should point out that the size limit of a Java String is about 2GB (Integer.MAX_VALUE) so this wasn't the cause of the truncation here.
According to https://groups.google.com/d/msg/android-developers/g4YkmrFST6A/z8K3vSdgwEkJ it is logcat that has a size limit, which is why appending "abcd" and printing in logcat didn't work. The String itself would have had the appended characters. The previously linked discussion also mentioned that size limits with the HTTP protocol itself can occasionally be a factor, but that most servers and clients handle this constraint internally so as to not expose it to the user.

Android service - data connection lost and can't restore

The background of my service: it implements LocationListener and in LocationManager instance (locMananager) registers for updates:
manager.requestLocationUpdates(LocationManager.GPS_PROVIDER, 0, 0, this);
In onLocationChanged method it calls a method named recordTrace with a current location, which then calls getHttpResponse to send the location coordinates to a server. The latter method is as follows:
public InputStream getHttpResponse(String url, ArrayList<NameValuePair> params, int timeout) throws Exception {
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, timeout);
httpClient.setParams(httpParams);
HttpPost post = new HttpPost(url);
if(params != null && !params.isEmpty()) {
try {
post.setEntity(new UrlEncodedFormEntity(params));
} catch (UnsupportedEncodingException e) {
Log.e(TAG, "HttpPost.setEntity Error: " + e.getMessage());
lastError = "błąd HTTP";
}
}
CookieStore store = new BasicCookieStore();
if(localCookies != null && localCookies.size() > 0) {
for(int i = 0; i < localCookies.size(); i++) {
store.addCookie(localCookies.get(i));
}
}
HttpContext context = new BasicHttpContext();
context.setAttribute(ClientContext.COOKIE_STORE, store);
HttpResponse response = null;
HttpEntity entity = null;
InputStream content = null;
try {
response = httpClient.execute(post, context);
store = (CookieStore) context.getAttribute(ClientContext.COOKIE_STORE);
List<Cookie> cookies = store.getCookies();
if(cookies != null && cookies.size() > 0) {
localCookies = new ArrayList<BasicClientCookie>();
for(Cookie cookie : cookies) {
localCookies.add((BasicClientCookie) cookie);
}
}
entity = response.getEntity();
content = entity.getContent();
return content;
} catch (ClientProtocolException e) {
throw e;
} catch (IOException e) {
throw e;
} catch (Exception e) {
throw e;
}
}
params is a NameValuePair with prepared data, timeout is set to 5000 (5 seconds). ArrayList localCookies holds cookies saved before, after successful logging in (to keep the session).
The problem is: when I loose a mobile signal (i.e. when I go to a subway) and restore it, I get IOException, which is unrecoverable unless I restart the phone.
Any help would be appreciated. I'm loosing my mind and going bald!
Thanks,
Peter.
EDIT
I've done some research. After the method getHttpResponse is invoked, I utilize InputStream returned by it, but don't close it after all. Do you thing that might be the issue? Is this possible that the operator breaks the connection and then establishes a new one, whereas my InputStream somehow "keeps" the former connection and produces the problems with data transfer?
I added a finally block where now the InputSTream is closed. However, since it's hard to cause the problem on demand (it doesn't happen regularly), I can't check if closing stream solves it.

After a few days of testing it seems I've found the solution. Calling 2 methods solves the issue:
close the InputStream returned by httpResponse.getEntity()
shutdown the connection by executing httpClient.getConnectionManager().shutdown()
Code snippet of a complete request and response handling:
String url = "www.someurl.com";
ArrayList<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("login", "mylogin"));
params.add(new BasicNameValuePair("password", "mypassword"));
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
try {
httpPost.setEntity(new UrlEncodedFormEntity(params));
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
InputStream content = httpEntity.getContent();
/*
* utilize content here...
*/
content.close(); // here's the point
httpClient.getConnectionManager().shutdown(); // the second important thing
}
catch (UnsupportedEncodingException e) {}
catch (ClientProtocolException e) {}
catch (IOException e) {}
I'm answering my own question since I've spent a lot of time on searching what's causing the problem and I think I can save somebody's time. After a few days the aplication is still working and doesn't break the connection.
Regards,
Peter.

How to send a JSON object over POST Request with Android

I am trying to build a small application in which the application will communicate with a php script with the help of JSON objects. I successfully implemented the GET Request test application but using JSON with post is creating problems. The code generates no error but my php script reply with no nothing except an empty Array() which implies that nothing was sent over the connection with code:
<?php print_r($_REQUEST); ?>
and trying with
<?php print($_REQUEST['json']); ?>
throws HTML back to the application with json variable not found error.
I have already tried a few solutions mentioned here including: How to send a JSON object over Request with Android? and How to send a json object over httpclient request with android so it would be great if you can point out my mistake and can briefly describe what exactly I was doing wrong. Thanks.
Here is the code snippet for from where the JSON Object is converted into string and then attached to a Post variable.
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpPost httppostreq = new HttpPost(wurl);
StringEntity se = new StringEntity(jsonobj.toString());
se.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
httppostreq.setEntity(se);
//httppostreq.setHeader("Accept", "application/json");
//httppostreq.setHeader("Content-type", "application/json");
//httppostreq.setHeader("User-Agent", "android");
HttpResponse httpresponse = httpclient.execute(httppostreq);
HttpEntity resultentity = httpresponse.getEntity();
Here is TCP Stream Dump collected through wireshark if it can help:
POST /testmysql.php?test=true HTTP/1.1
Content-Length: 130
Content-Type: application/json;charset=UTF-8
Content-Type: application/json;charset=UTF-8
Host: 192.168.100.4
Connection: Keep-Alive
User-Agent: Apache-HttpClient/UNAVAILABLE (java 1.4)
{"weburl":"hashincludetechnology.com","header":{"devicemodel":"GT-I9100","deviceVersion":"2.3.6","language":"eng"},"key":"value"}HTTP/1.1 200 OK
Date: Mon, 30 Apr 2012 22:43:10 GMT
Server: Apache/2.2.17 (Win32)
Content-Length: 34
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
Array
(
[test] => true
)
Test // echo statement left intentionally.

you are using PHP on the server side, so your HTTP entity must be a multipart encoded one. See this link. You are using a string entity, but this is not correct. It must be a MultipartEntity, which emulates what the browser does when you submit a form in a web page. MultipartEntity should be in httpmime jar.
Once you have your multipart entity, simply add a Part named "json", and set its contents to the string representation of your json-encoded object.
Note that this answer is because you use PHP on the server side, so you must use its "protocol" to read variables via $_REQUEST. If you used your own request parser oh the server side, even a StringEntity could be ok. See HTTP_RAW_POST_DATA

The below should work. Make sure to set the appropriate keys for what your form post is expecting at the top. Also I included how to send an image as well as other various json data, just delete those lines if that is not necessary.
static private String postToServerHelper(
String action,
JSONObject jsonData,
byte[] imageData){
// keys for sending to server
/** The key for the data to post to server */
final String KEY_DATA = "data";
/** The key for the action to take on server */
final String KEY_ACTION = "action";
/** The return code for a successful sync with server */
final int GOOD_RETURN_CODE = 200;
/** The key for posting the image data */
final String KEY_IMAGE = "imageData";
/** The image type */
final String FILE_TYPE = "image/jpeg";
/** The encoding type of form data */
final Charset ENCODING_TYPE = Charset.forName("UTF-8");
// the file "name"
String fileName = "yourFileNameHere";
// initialize result string
String result = "";
// initialize http client and post to correct page
HttpClient client = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://www.yourdomain.com/yourpage.php");
// set to not open tcp connection
httpPost.getParams().setBooleanParameter(CoreProtocolPNames.USE_EXPECT_CONTINUE, false);
// build the values to post, the action and the form data, and file data (if any)
MultipartEntity multipartEntity = new MultipartEntity(
HttpMultipartMode.BROWSER_COMPATIBLE);
try{
multipartEntity.addPart(KEY_ACTION, new StringBody(action, ENCODING_TYPE));
multipartEntity.addPart(KEY_DATA, new StringBody(jsonData.toString(), ENCODING_TYPE));
if (imageData != null){
multipartEntity.addPart(KEY_IMAGE, new ByteArrayBody(imageData, FILE_TYPE, fileName));
}
}catch (Exception e){
return e.getMessage();
}
// set the values to the post
httpPost.setEntity(multipartEntity);
int statusCode= -1;
// send post
try {
// actual send
HttpResponse response = client.execute(httpPost);
// check what kind of return
StatusLine statusLine = response.getStatusLine();
statusCode = statusLine.getStatusCode();
// good return
if (statusCode == GOOD_RETURN_CODE) {
// read return
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(content));
String line;
while ((line = reader.readLine()) != null) {
builder.append(line + "\n");
}
content.close();
result = builder.toString();
// bad return
} else {
return String.parse(statusCode);
}
// different failures
} catch (ClientProtocolException e) {
return e.getMessage();
} catch (IOException e) {
return e.getMessage();
}
// return the result
return result;
}

DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost(url);
JSONObject clientList = new JSONObject ();
clientList.put("name","");
clientList.put("email","");
clientList.put("status","");
clientList.put("page","");
JSONObject listclient = new JSONObject ();
listclient.put("mydetail", clientList);
//--List nameValuePairs = new ArrayList(1);
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(2);
nameValuePairs.add(new BasicNameValuePair("token", tokenid));
nameValuePairs.add(new BasicNameValuePair("json_data", listclient.toString()));
httpPost.setEntity(new UrlEncodedFormEntity(nameValuePairs));
Log.d("JSON",nameValuePairs.toString());
//-- Storing Response
HttpResponse httpResponse = httpClient.execute(httpPost);

IIS seems to think an Android HttpPost is a GET

UPDATE: These problems were caused by a reverse proxy performing a 301 redirect. Altering the url to the destination of the redirect fixed the issue.
I am struggling to make a POST request from android to a web service.
I have a web service running on IIS7 with the following:
<OperationContract()> _
<Web.WebInvoke(BodyStyle:=WebMessageBodyStyle.Bare, Method:="POST", RequestFormat:=WebMessageFormat.Xml, ResponseFormat:=WebMessageFormat.Xml, UriTemplate:="HelloWorld")> _
Function HelloWorld() As XmlElement
When I send a POST request to this url from Firefox it works as expected.
When I make the request from an Android device using the following code:
String sRequest = "http://www.myserviceurl.com/mysevice/HelloWorld";
ArrayList<NameValuePair> arrValues = new ArrayList<NameValuePair>();
arrValues.add(new BasicNameValuePair("hello", "world"));
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpRequest = new HttpPost(sRequest);
httpRequest.setHeader("Content-Type", "application/x-www-form-urlencoded");
httpRequest.setEntity(new UrlEncodedFormEntity(arrValues));
HttpResponse response = httpClient.execute(httpRequest);
I get a Method Not Allowed 405 response and when looking in the IIS logs the request to this url appears as a "GET".
If I change the target of the request to a PHP script that echoes $_SERVER['REQUEST_METHOD'] the output is POST.
The web.config of the web service has GET, HEAD and POST as verbs.
Is there something I have overlooked?

I had to implement a workaround by disabling the automatic redirect and then catching the response code and redirect URL and reexecuting the POST.
// return false so that no automatic redirect occurrs
httpClient.setRedirectHandler(new DefaultRedirectHandler()
{
#Override
public boolean isRedirectRequested(HttpResponse response, HttpContext context)
{
return false;
}
});
Then when I issued the request
response = httpClient.execute(httpPost, localContext);
int code = response.getStatusLine().getStatusCode();
// if the server responded to the POST with a redirect, get the URL and reexecute the POST
if (code == 302 || code == 301)
{
httpPost.setURI(new URI(response.getHeaders("Location")[0].getValue()));
response = httpClient.execute(httpPost, localContext);
}

try:
DefaultHttpClient http = new DefaultHttpClient();
HttpResponse res;
try {
HttpPost httpost = new HttpPost(s);
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.DEFAULT_CONTENT_CHARSET));
res = http.execute(httpost);
InputStream is = res.getEntity().getContent();
BufferedInputStream bis = new BufferedInputStream(is);
ByteArrayBuffer baf = new ByteArrayBuffer(50);
int current = 0;
while((current = bis.read()) != -1){
baf.append((byte)current);
}
res = null;
httpost = null;
String ret = new String(baf.toByteArray(),encoding);
return ret;
}
catch (ClientProtocolException e) {
// TODO Auto-generated catch block
return e.getMessage();
}
catch (IOException e) {
// TODO Auto-generated catch block
return e.getMessage();
}