Develop Reference

Android Http Response is incomplete. Returns an unterminated json object

I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.

Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?

I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.

I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.

Android HttpClient response

I'm trying to send json data to a php script from my Android application with HttpClient, and get the response.
Android Code
private void sendPurchase(String SKU) throws IOException{
Log.e("sendPurchase","Inside sendPurchase");
final SharedPreferences prefs = getGCMPreferences(getApplicationContext());
int pur_user = prefs.getInt("C_user", Integer.MIN_VALUE);
InputStream inputStream = null;
String result = "";
HttpClient httpclient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://www.*.com/includes/purchase.php");
JSONObject json = new JSONObject();
try {
json.put("PUR_sku", SKU);
json.put("PUR_user", pur_user);
} catch (JSONException e) { Log.e("SendPurchase","Problem with Json Object"); }
Log.i("JSONObject", json.toString());
StringEntity se = new StringEntity(json.toString(), HTTP.UTF_8);
httpPost.setEntity(se);
httpPost.setHeader("Accept", "application/json");
httpPost.setHeader("Content-type", "application/json");
HttpResponse httpResponse = httpclient.execute(httpPost);
inputStream = httpResponse.getEntity().getContent();
if(inputStream != null){ result = convertInputStreamToString(inputStream); }
else{result = "Did not work!"; }
Log.e("RESULT",result);
}
private static String convertInputStreamToString(InputStream inputStream) throws IOException{
BufferedReader bufferedReader = new BufferedReader( new InputStreamReader(inputStream));
String line = "";
String result = "";
while((line = bufferedReader.readLine()) != null)
result += line;
inputStream.close();
return result;
}
And the PHP script
<?
$auth=0;
require('./connexion.php');
$data = file_get_contents('php://input');
//$data = '{"PUR_sku":"singleone","PUR_user":"3"}';
$json = json_decode($data,true);
/* Some database stuff ... */
echo "Retour ".print_r($json)." et ".$json['PUR_sku']." et ".$json['PUR_user'];
?>
When i launch the app and execute sendPurchase function, it seems to be ok until the execution of the HttpPost. In the logcat i get all the logs with correct params, except the last log "RESULT" that does not appear.
That's why i guess something is going wrong with the HttpPost execution, but actually i don't know if the problem comes from the application side or the php script side...
When i execute the php script alone in a web browser, replacing first $data line by the second one, everything is ok. But when it comes from the application it's not ok...
The Json Object sent (i hope) to the script seems ok too : {"PUR_user":3,"PUR_sku":"singleone"}
(the sendPurchase function is executed in Background).
Any idea about what i'm doing wrong ? Thanks !
/EDIT/
Here is the logcat for #RyuZz solution.
My code is about purchasing an item, consume it and send new value to my database on a web server. The purchase & consume are ok, but i can't send the values to the web server.
And again, when i execute the php script alone in a web browser, replacing first $data line by the second one, everything is ok.
Note that i have another similar code to register user to GCM, using HttpClient, and that code works fine.
06-25 14:07:12.968: D/IabHelper(21833): Successfully consumed sku: singleconf
06-25 14:07:12.968: D/IabHelper(21833): Ending async operation: consume
06-25 14:07:12.979: D/CONSUME(21833): Consumption finished. Purchase: PurchaseInfo(type:inapp):{"orderId":"12999763169054705758.1353445524837889","packageName":"com.*.*","productId":"singleconf","purchaseTime":1435234296875,"purchaseState":0,"purchaseToken":"bohbcbiigcbidfficbikebnk.AO-J1OzuQ_SsNTG1h9MtUvbaPc3PeN9nBHG-qBOE82ao1rTDFNrgA7tYQcMdECxCVFrrZEn_QifQ28OcIupyesZI-5cjDILFODYpBEaeqMfE0wCAeMFkJLfNUK_TsKPMj7F2sBDdgOYx"}, result: IabResult: Successful consume of sku singleconf (response: 0:OK)
06-25 14:07:12.979: D/CONSUME(21833): You bought & consumed a single conf
06-25 14:07:12.979: D/CONSUME(21833): End consumption flow.
06-25 14:07:12.979: E/Purchase Background(21833): Inside doInBackground
06-25 14:07:12.979: E/sendPurchase(21833): Failed to send HTTP POST request due to: java.lang.NullPointerException

You can try the following instead of HttpClient which is anyway deprecated:
try{
int pur_user = prefs.getInt("C_user", Integer.MIN_VALUE);
URL url = new URL("http://www.*.com/includes/purchase.php");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("Accept", "application/json");
connection.setRequestMethod("POST");
JSONObject jsonObject = new JSONObject();
jsonObject.put("PUR_sku", SKU);
jsonObject.put("PUR_user", pur_user);
//convert JSONObject to JSON to String
json = jsonObject.toString();
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write(json);
writer.close();
responseCode = connection.getResponseCode();
if(responseCode == 200) {
InputStream content = connection.getInputStream();
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(content, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null)
{
sb.append(line).append("\n");
}
result = sb.toString();
//TODO get your stuff from result
content.close();
} catch (Exception ex) {
Log.e(TAG, "Failed to parse JSON due to: " + ex);
} finally {
connection.disconnect();
}
} else {
Log.e(TAG, "Server responded with status code: " + responseCode);
}
} catch(Exception ex) {
Log.e(TAG, "Failed to send HTTP POST request due to: " + ex);
}
if this isn't working, please post the logcat.
Don't forget to implement the required permissions in your manifest:
<uses-permission
android:name="android.permission.INTERNET" />

How to handle 500 Internal Server Error when using JSON

I am relatively new to Android and I am using JSON to get data from a server. On the first loop at line 22, the StringBuilder contains, 500 Internal Server Error and then the jArray ends up coming back null. How can I handle this error?
public static JSONObject getJSON() {
String jsonString = "";
InputStream inStream = null;
//http post
JSONObject jArray = null;
try {
HttpClient httpClient = new DefaultHttpClient(new BasicHttpParams());
HttpPost httpPost = new HttpPost(WS_URL);
httpPost.setHeader("Content-type", "application/json");
HttpResponse response = httpClient.execute(httpPost);
HttpEntity entity = response.getEntity();
inStream = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(inStream, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
inStream.close();
jsonString = sb.toString();
jArray = new JSONObject(jsonString);
//outputTransactions(jArray);
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return jArray;
}

Though its a late reply but it might help others. You need to check the response status from the server before parsing it as JSON.
For ex.
int status_code=response.getStatusLine().getStatusCode();
if(status_code!=200){
Log.d("MYLOG","ERROR! Response status is"+status_code);
}
else{
inStream = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(inStream, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
inStream.close();
// Rest of your code......
}
or Optionally you can check the status code and display the error to the user
Like:
else if(status_code==404){
Log.d("MYLOG","Sorry! Page not found! Check the URL ");
}else if(status_code==500){
Log.d("MYLOG","Server is not responding! Sorry try again later..");
}
Hope it helps for newbies like you :-)

A "500 Internal Server" error means the server had a problem responding to your request. You are not getting a JSON string response.
Then when you try to create your jArray, the string is not valid JSON and the JSONObject cannot parse it - it returns "null" as you say.
You can parse the server response to see if it contains this string, and then create whatever jArray object you want, but you can't get a JSON object from a non-JSON string.

Take a look at this answer: https://stackoverflow.com/a/8148785/1974614
You should check the statusCode against 500

You should consider use a library to handle the REST requests like: http://square.github.io/retrofit/
If you use a library like that you can get an object from json when a success response is available and other when an error occur.
MyApi mylogin = restAdapter.create(MyApi.class); //this is how retrofit create your api
mylogin.login(username,password,new Callback<String>() {
#Override
public void success(String s, Response response) {
//process your response if login successfull you can call Intent and launch your main activity
}
#Override
public void failure(RetrofitError retrofitError) {
retrofitError.printStackTrace(); //to see if you have errors
}
});
}

I got the same problem like you and I solved it because I missed a part while adding GSON .jar files to adding my serverside project. I think you should carrefully add external libraries to your project too like me.With these links I could aware of problem .
LINK 1
LINK 2

Encoding Issue with HttpUrlConnection in Android

I want to send an XML message to a server from my Android Mobile app via HTTP post.
I tried it with HttpUrlConnection, following these steps:
URL url = new URL(vURL);
HttpUrlConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
// Adding headers (code removed)
conn.setRequestProperty("Content-Type", "text/xml; charset=utf-16");
OutputStream out = new BufferedOutputStream(conn.getOutputStream());
// Adding XML message to the connection output stream
// I have removed exception handling to improve readability for posting it here
out.write(pReq.getBytes()); // here pReq is the XML message in String
out.close();
conn.connect();
Once I get the response, the stream reading part is in done this manner:
BufferedReader in = null;
StringBuffer sb;
String result = null;
try {
InputStreamReader isr = new InputStreamReader(is);
// Just in case, I've also tried:
// new InputStreamReader(is, "UTF-16");
// new InputStreamReader(is, "UTF-16LE");
// new InputStreamReader(is, "UTF-16BE");
// new InputStreamReader(is, "UTF-8");
in = new BufferedReader(isr);
sb = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null)
sb.append(line);
in.close();
result = sb.toString();
} catch (Exception e) {
e.printStackTrace();
}
Now the result string I get is in some unreadable format/encoding.
When I try the same thing with HttpClient it works correctly. Here is the streaming reading part once I get an HttpResponse after the HttpClient.execute call:
BufferedReader in = null;
InputStream is;
StringBuffer sb;
String decompbuff = null;
try {
is = pResponse.getEntity().getContent();
InputStreamReader isr = new InputStreamReader(is);
in = new BufferedReader(isr);
// Prepare the String buffer
sb = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null)
sb.append(line);
in.close();
// gZip decompression of response. Note: message was compressed before
// posting it via HttpClient (Posting code is not mentioned here)
decompbuff = Decompress(sb.toString());
} catch (Exception e) {
e.printStackTrace();
}
return decompbuff;
Some help is appreciated in understanding the problem.

One (severe) problem could be that you're ignoring the encoding of input and output.
Input
conn.setRequestProperty("Content-Type", "text/xml; charset=utf-16");
OutputStream out = new BufferedOutputStream(conn.getOutputStream());
// Adding XML message to the connection output stream
// I have removed exception handling to improve readability for posting it here
out.write(pReq.getBytes()); // <-- you use standard platform encoding
out.close();
better:
out.write(pReq.getBytes("UTF-16"));
Output
You probably ignored compression, which would better look like this (taken from DavidWebb):
static InputStream wrapStream(String contentEncoding, InputStream inputStream)
throws IOException {
if (contentEncoding == null || "identity".equalsIgnoreCase(contentEncoding)) {
return inputStream;
}
if ("gzip".equalsIgnoreCase(contentEncoding)) {
return new GZIPInputStream(inputStream);
}
if ("deflate".equalsIgnoreCase(contentEncoding)) {
return new InflaterInputStream(inputStream, new Inflater(false), 512);
}
throw new RuntimeException("unsupported content-encoding: " + contentEncoding);
}
// ...
InputStream is = wrapStream(conn.getContentEncoding(), is);
InputStreamReader isr = new InputStreamReader(is, "UTF-16");
in = new BufferedReader(isr);
sb = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null)
sb.append(line); // <-- you're swallowing linefeeds!
in.close();
result = sb.toString();
It would be better to let the XML-Parser consume your InputStream directly. Don't create a JAVA string, but let the parser scan the bytes. It will automatically detect the encoding of the XML.
Generally there might be still an issue, because we don't know what type of UTF-16 you use. Can be BigEndian or LittleEndian. That's why I asked, if you really need UTF-16. If you don't have to treat with some asian languages, UTF-8 should be more efficient and easier to use.
So the "solution" I gave you is not guaranteed to work - you have to fiddle with UTF-16 BE/LE a bit and I wish you good luck and patience.
Another remark: in your example above you first construct the String and then Decompress it. That is the wrong order. The stream comes compressed (gzip, deflate) and must be decompressed first. Then you get the String.

how to load a big webpage into a string

I'm a novice with Java and Android, but not to programming and HTTP. This HTTP GET method, mostly copied from other examples using the Apache HTTP classes, only retrieves the first few K of a large webpage. I checked that the webpage does not have lines longer than 8192 bytes (is that possible?), but out of webpages around 40K I get back maybe 6K, maybe 20K. The number of bytes read does not seem to have a simple realtionship with the total webpage size, or the webpage modulus 8192, or with the webpage content.
Any ideas folks?
Thanks!
public static String myHttpGet(String url) throws Exception {
BufferedReader in = null;
try {
HttpClient client = getHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(url));
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sbuffer = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null) {
sbuffer.append(line + "\n");
}
in.close();
String result = sbuffer.toString();
return result;
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}

No need to write you own HttpEntity-to-String code, try EntityUtils instead:
// this uses the charset the server encoded the entity in
String result = EntityUtils.toString(entity);

It looks as if the problem is with pages from a certain website starting Goo... I'm not having this problem with large pages from other sites. So the code is probably OK.