Develop Reference

Android Http Response is incomplete. Returns an unterminated json object

I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.

Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?

I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.

I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.

Converting String to Android JSONObject loses utf-8

I am trying to get a (JSON formatted) String from a URL and consume it as a Json object. I lose UTF-8 encoding when I convert the String to JSONObject.
This is The function I use to connect to the url and get the string:
private static String getUrlContents(String theUrl) {
StringBuilder content = new StringBuilder();
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = bufferedReader.readLine()) != null) {
content.append(line + "\n");
}
bufferedReader.close();
} catch(Exception e) {
e.printStackTrace();
}
return content.toString();
}
When I get data from server, the following code displays correct characters:
String output = getUrlContents(url);
Log.i("message1", output);
But when I convert the output string to JSONObject the Persian characters becomes question marks like this ??????. (messages is the name of array in JSON)
JSONObject reader = new JSONObject(output);
String messages = new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8");
Log.i("message2", messages);

You're telling Java to convert the string (with key message) to bytes using ISO-8859-1 and than to create a new String from these bytes, interpreted as UTF-8.
new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8");
You could simply use:
String messages = reader.getString("messages");

You can update your code as the following:
private static String getUrlContents(String theUrl) {
StringBuilder content = new StringBuilder();
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "utf-8"));
String line;
while ((line = bufferedReader.readLine()) != null) {
content.append(line).append("\n");
}
bufferedReader.close();
} catch(Exception e) {
e.printStackTrace();
}
return content.toString().trim();
}

You've got two encoding issues:
The server sends text encoded in a character set. When you setup your InputStreamReader, you need to pass the encoding the server used so it can be decoded properly. The character encoding is usually given in the Content-type HTTP response, in the charset field. JSON is typically UTF-8 encoded, but can also be legally UTF-16 and UTF-32, so you need to check. Without a specified encoding, your system environment will be used when marshalling bytes to Strings, and vice versa . Basically, you should always specify the charset.
String messages = new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8"); is obviously going to cause issues (if you have non-ascii characters) - it's encoding the string to ISO-8995-1 and then trying to decode it as UTF-8.
A simple regex pattern can be used to extract the charset value from the Content-type header before reading the inputstream. I've also included a neat InputStream -> String converter.
private static String getUrlContents(String theUrl) {
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
InputStream is = urlConnection.getInputStream();
// Get charset field from Content-Type header
String contentType = urlConnection.getContentType();
// matches value in key / value pair
Pattern encodingPattern = Pattern.compile(".*charset\\s*=\\s*([\\w-]+).*");
Matcher encodingMatcher = encodingPattern.matcher(contentType);
// set charsetString to match value if charset is given, else default to UTF-8
String charsetString = encodingMatcher.matches() ? encodingMatcher.group(1) : "UTF-8";
// Quick way to read from InputStream.
// \A is a boundary match for beginning of the input
return new Scanner(is, charsetString).useDelimiter("\\A").next();
} catch(Exception e) {
e.printStackTrace();
}
return null;
}

Not sure if this will help, but you might be able to do something like this:
JSONObject result = null;
String str = null;
try
{
str = new String(output, "UTF-8");
result = (JSONObject) new JSONTokener(str).nextValue();
}
catch (Exception e) {}
String messages = result.getString("messages");

HTTP GET in async task does not work

I get a response from a HTTPPOST, take a substring from it which is URL and then do a HTTPGET on it, I am unable to get any response. Any inputs on what the issue could be?
String pkmspogout1 = result.substring(result.indexOf(",")+1,result.length());
Log.i("Lpkmspogout1 ", pkmspogout1);
HttpClient httpclient4 = new DefaultHttpClient();
HttpResponse logoutresponse ;
try {
HttpGet httpget = new HttpGet(pkmspogout1);
logoutresponse = httpclient4.execute(httpget);
BufferedReader in2 = new BufferedReader(new InputStreamReader(logoutresponse.getEntity().getContent()));
StringBuffer sb2 = new StringBuffer("");
String line2 = "";
String NL2 = System.getProperty("line.separator");
while ((line2 = in2.readLine()) != null) {
sb2.append(line + NL2);
}
in2.close();
String pkmslogout5 = sb2.toString();
Log.i("logoutresponse from pkms", pkmslogout5+"");
} catch (Exception e) {}

Is pkmspogout1 a correct url?
Also you should make sure to do the following to see any errors:
catch (Exception e) {e.printStackTrace();}
Try encoding the url as follows:
String[] url = pkmspogout1.split("com"); //Or whatever is the last part of the main url.
//The letter "." means any character btw as it is a regular expression.
String finalUrl = url[0];
for(int i=0; i<url.length; i++){
finalUrl += URLEncoder.encode(url[i],"UTF-8");
}
Slightly crude to be honest.... Why when forming on server side would be better to encode it then.
You could just try:
pkmspogout1 = pkmspogout1.replace("http://","");
pkmspogout1 = URLEncoder.encode(pkmspogout1);
pkmspogout1 = "http://"+pkmspogout1;
See if it is only the "http://" that is ruining your url. I still assume that it is the url that is the issue more than anything else as you told me hardcoded it works fine.

It is working finally, Phew !I tried (pkmspogout1.trim()) and it worked ! (spaces before and after the URL were the issue).

HttpUrlConnection.getInputStream returns empty stream in Android

I make a GET request to a server using HttpUrlConnection.
After connecting:
I get response code: 200
I get response message: OK
I get input stream, no exception thrown but:
in a standalone program I get the body of the response, as expected:
{"name":"my name","birthday":"01/01/1970","id":"100002215110084"}
in a android activity, the stream is empty (available() == 0), and thus I can't get
any text out.
Any hint or trail to follow? Thanks.
EDIT: here it is the code
Please note: I use import java.net.HttpURLConnection; This is the standard
http Java library. I don't want to use any other external library. In fact
I did have problems in android using the library httpclient from apache (some of their anonymous .class can't be used by the apk compiler).
Well, the code:
URLConnection theConnection;
theConnection = new URL("www.example.com?query=value").openConnection();
theConnection.setRequestProperty("Accept-Charset", "UTF-8");
HttpURLConnection httpConn = (HttpURLConnection) theConnection;
int responseCode = httpConn.getResponseCode();
String responseMessage = httpConn.getResponseMessage();
InputStream is = null;
if (responseCode >= 400) {
is = httpConn.getErrorStream();
} else {
is = httpConn.getInputStream();
}
String resp = responseCode + "\n" + responseMessage + "\n>" + Util.streamToString(is) + "<\n";
return resp;
I see:
200
OK
the body of the response
but only
200
OK
in android

Trying the code of Tomislav I've got the answer.
My function streamToString() used .available() to sense if there is any data received,
and it returns 0 in Android. Surely, I called it too soon.
If I rather use readLine():
class Util {
public static String streamToString(InputStream is) throws IOException {
StringBuilder sb = new StringBuilder();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
sb.append(line);
}
return sb.toString();
}
}
then, it waits for the data to arrive.
Thanks.

You can try with this code that will return response in String:
public String ReadHttpResponse(String url){
StringBuilder sb= new StringBuilder();
HttpClient client= new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
try {
HttpResponse response = client.execute(httpget);
StatusLine sl = response.getStatusLine();
int sc = sl.getStatusCode();
if (sc==200)
{
HttpEntity ent = response.getEntity();
InputStream inpst = ent.getContent();
BufferedReader rd= new BufferedReader(new InputStreamReader(inpst));
String line;
while ((line=rd.readLine())!=null)
{
sb.append(line);
}
}
else
{
Log.e("log_tag","I didn't get the response!");
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return sb.toString();
}

The Stream data may not be ready, so you should check in a loop that the data in the stream is available before attempting to access it.
Once the data is ready, you should read it and store in another place like a byte array; a binary stream object is a nice choice to read data as a byte array. The reason that a byte array is a better choice is because the data may be binary data like an image file, etc.
InputStream is = httpConnection.getInputStream();
byte[] bytes = null;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] temp = new byte[is.available()];
while (is.read(temp, 0, temp.length) != -1) {
baos.write(temp);
temp = new byte[is.available()];
}
bytes = baos.toByteArray();
In the above code, bytes is the response as byte array. You can convert it to string if it is text data, for example data as utf-8 encoded text:
String text = new String(bytes, Charset.forName("utf-8"));

how to load a big webpage into a string

I'm a novice with Java and Android, but not to programming and HTTP. This HTTP GET method, mostly copied from other examples using the Apache HTTP classes, only retrieves the first few K of a large webpage. I checked that the webpage does not have lines longer than 8192 bytes (is that possible?), but out of webpages around 40K I get back maybe 6K, maybe 20K. The number of bytes read does not seem to have a simple realtionship with the total webpage size, or the webpage modulus 8192, or with the webpage content.
Any ideas folks?
Thanks!
public static String myHttpGet(String url) throws Exception {
BufferedReader in = null;
try {
HttpClient client = getHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(url));
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sbuffer = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null) {
sbuffer.append(line + "\n");
}
in.close();
String result = sbuffer.toString();
return result;
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}

No need to write you own HttpEntity-to-String code, try EntityUtils instead:
// this uses the charset the server encoded the entity in
String result = EntityUtils.toString(entity);

It looks as if the problem is with pages from a certain website starting Goo... I'm not having this problem with large pages from other sites. So the code is probably OK.