Develop Reference

Android Http Response is incomplete. Returns an unterminated json object

I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.

Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?

I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.

I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.

Android JSONException: JSONArray must start with [ - and it does

Okay, I know this question has been asked before, but all the answers did not help at all. First, let me recap the problem:
I have a (PHP-generated) JSON element, the PHP code looks like this:
if (!headers_sent()) header('Content-type: application/json');
echo "[{\"test\":\"success\"}]";
No other echo, print or anything else before that, so the output really is just
[{"test":"success"}]
and nothing else. The PHP file is saved as pure UTF-8 (checked and double-checked).
Now, my Android code looks as follows:
StringBuilder builder = new StringBuilder();
HttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("http://myurl");
try {
HttpResponse response = client.execute(httpGet);
StatusLine statusLine = response.getStatusLine();
int statusCode = statusLine.getStatusCode();
if (statusCode == 200) {
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(content, "UTF-8"), 8);
String line;
while ((line = reader.readLine()) != null) {
builder.append(line);
}
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
String menuData = builder.getString();
ArrayList<HashMap<String, String>> tempList = new ArrayList<HashMap<String, String>>();
boolean change = false;
try {
JSONArray jsonMenu = new JSONArray(menuData);
for (int i = 0; i < jsonMenu.length(); i++) {
JSONObject jsonObject;
try {
jsonObject = jsonMenu.getJSONObject(i);
HashMap<String, String> menuItem = new HashMap<String, String>();
menuItem.put("test", jsonObject.getString("test"));
tempList.add(menuItem);
change = true;
} catch (JSONException e) {
}
}
} catch (JSONException e) {
}
if(change) {
// here comes the listview update, but never triggered anyways
}
And I always get the same error message:
04-05 21:06:31.226: W/System.err(1208): org.json.JSONException: A JSONArray text must start with '[' at character 1 of ?[{"test":"success"}]
This is actually the Logcat output saved to log.txt and parsed here, and then all of the sudden there is this question mark. All attempts to trim or format the String before parsing it to the JSONArray have failed. I have absolutely no clue anymore what else to do. In Logcat in Eclipse itself there is no question mark by the way...
Added Log.e output to the catch statements, the error output is exactly the same. Screenshot here:

Problem solved:
Open the PHP file in Notepad++, click on Encoding => Encode in UTF-8 without BOM.
I neglected this because it shows "ANSI as UTF-8" in the status bar which sounded just wrong (I wanted pure UTF-8 and no ANSI mix). Yeah, I'm no Linux fella, so this ANSI/UTF-8 was something I never quite got to be honest, and here it trapped me again.
If someone is interested how I found the solution (credit goes to #Perception, #Raghav-Sood, and #Reuben-Scratton here):
Converted the menuData string to a byte array and found out that the first two chars are not the opening bracket that JSONArray is looking for, but two BOM chars.

Android parse json from url and store it

Hi there i'm creating my first android app and i'm wanting to know what is the best and most efficient way of parsing a JSON Feed from a URL.Also Ideally i want to store it somewhere so i can keep going back to it in different parts of the app. I have looked everywhere and found lots of different ways of doing it and i'm not sure which to go for. In your opinion whats the best way of parsing json efficiently and easily?

I'd side with whatsthebeef on this one, grab the data and then serialize to disk.
The code below shows the first stage, grabbing and parsing your JSON into a JSON Object and saving to disk
// Create a new HTTP Client
DefaultHttpClient defaultClient = new DefaultHttpClient();
// Setup the get request
HttpGet httpGetRequest = new HttpGet("http://example.json");
// Execute the request in the client
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
// Grab the response
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
String json = reader.readLine();
// Instantiate a JSON object from the request response
JSONObject jsonObject = new JSONObject(json);
// Save the JSONOvject
ObjectOutput out = new ObjectOutputStream(new FileOutputStream(new File(getCacheDir(),"")+"cacheFile.srl"));
out.writeObject( jsonObject );
out.close();
Once you have the JSONObject serialized and save to disk, you can load it back in any time using:
// Load in an object
ObjectInputStream in = new ObjectInputStream(new FileInputStream(new File(new File(getCacheDir(),"")+"cacheFile.srl")));
JSONObject jsonObject = (JSONObject) in.readObject();
in.close();

Your best bet is probably GSON
It's simple, very fast, easy to serialize and deserialize between json objects and POJO, customizable, although generally it's not necessary and it is set to appear in the ADK soon. In the meantime you can just import it into your app. There are other libraries out there but this is almost certainly the best place to start for someone new to android and json processing and for that matter just about everyone else.
If you want to persist you data so you don't have to download it every time you need it, you can deserialize your json into a java object (using GSON) and use ORMLite to simply push your objects into a sqlite database. Alternatively you can save your json objects to a file (perhaps in the cache directory)and then use GSON as the ORM.

This is pretty straightforward example using a listview to display the data. I use very similar code to display data but I have a custom adapter. If you are just using text and data it would work fine. If you want something more robust you can use lazy loader/image manager for images.

Since an http request is time consuming, using an async task will be the best idea. Otherwise the main thread may throw errors. The class shown below can do the download asynchronously
private class jsonLoad extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... urls) {
String response = "";
for (String url : urls) {
DefaultHttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
try {
HttpResponse execute = client.execute(httpGet);
InputStream content = execute.getEntity().getContent();
BufferedReader buffer = new BufferedReader(new InputStreamReader(content));
String s = "";
while ((s = buffer.readLine()) != null) {
response += s;
}
} catch (Exception e) {
e.printStackTrace();
}
}
return response;
}
#Override
protected void onPostExecute(String result) {
// Instantiate a JSON object from the request response
try {
JSONObject jsonObject = new JSONObject(result);
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
File file = new File(getApplicationContext().getFilesDir(),"nowList.cache");
try {
file.createNewFile();
FileOutputStream writer = openFileOutput(file.getName(), Context.MODE_PRIVATE);
writer.write(result);
writer.flush();
writer.close();
}
catch (IOException e) { e.printStackTrace(); return false; }
}
}
Unlike the other answer, here the downloaded json string itself is saved in file. So Serialization is not necessary
Now loading the json from url can be done by calling
jsonLoad jtask=new jsonLoad ();
jtask.doInBackground("http:www.json.com/urJsonFile.json");
this will save the contents to the file.
To open the saved json string
File file = new File(getApplicationContext().getFilesDir(),"nowList.cache");
StringBuilder text = new StringBuilder();
try {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
text.append(line);
text.append('\n');
}
br.close();
}
catch (IOException e) {
//print log
}
JSONObject jsonObject = new JSONObject(text);

Android app using data from webservice

I want to write an Android application that can display some data received(polled) from an internet resource.
I guess that I need to write some logic that will periodically call and get data from some endpoint, parse the response and display it. Is there a good tutorial for all this steps?
I know very little about Android programming at the momment and maybe it is better to start with something simpler. I just want to know what to look for while learning an gather some resources on this.

What you want to do is developing a rest api that provides data for your android app. E.g. you website has some content that you want use in your app, then you could write a php script that just returns that data in a specific format.
E.g. mysite.net/rest/fetchAllLocations.php?maybe_some_parameters
This would return locations in e.g. json format, here is an example how that looks like:
[{"id":1,"shop_lng":8.5317153930664,"shop_lat":52.024803161621,"shop_zipcode":33602,"shop_city":"Bielefeld","shop_street":"Arndtstra\u00dfe","shop_snumber":3,"shop_name":"M\u00fcller","shop_desc":"Kaufhaus"}]
Here is an example for a rest api request:
http://shoqproject.supervisionbielefeld.de/public/gateway/gateway/get-shops-by-city/city/Bielefeld
So when you have your rest api set up you can deal with receiving that data with your android phone. I use a static method to get this data:
public class JsonGrabber{
public static JSONArray receiveData(){
String url = "your url";
String result = "";
DefaultHttpClient client = new DefaultHttpClient();
HttpGet method = new HttpGet(url);
HttpResponse res = null;
try {
res = client.execute(method);
} catch (ClientProtocolException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
}
try{
InputStream is = res.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
result = sb.toString();
}catch(Exception e){
Log.e("log_tag", "Error converting result "+e.toString());
}
JSONArray jArray = null;
try{
jArray = new JSONArray(result);
}catch(JSONException e){
Log.e("log_tag", "Error parsing data "+e.toString());
}
return jArray;
}
}
Well thats all, once you have your data in json format you just have to parse it:
JSONArray test = (JSONArray) JsonGrabber.receiveData()
try {
for(int i=0;i<test.length();i++){
JSONObject json_data = test.getJSONObject(i);
int id = json_data.getInt("id");
}
}
The web request should run in another thread, because it can be a time consuming process. So you need to deal with AsyncTask. Here are some resources:
Painless Threading
Multithreading for performance
Hello Android Tutorial

Updateing Android application content via internet?

i want to develop an Android application that will take the content from internet (server) and present it in the application.
(ex. i take the todays weather forecast, put the numbers in SQLite database or .txt file , put the database/txt file on internet server so when i open the application, the app connects&downloads the database via the net and presents me with todays forecast)
If you can references me to some example/video tutorial/book that deals with this issue i will be very thankful!

What you want to do is developing a rest api that provides data for your android app. E.g. you website has some content that you want use in your app, then you could write a php script that just returns that data in a specific format.
E.g. mysite.net/rest/fetchAllLocations.php?maybe_some_parameters
This would return locations in e.g. json format, here is an example how that looks like:
[{"id":1,"shop_lng":8.5317153930664,"shop_lat":52.024803161621,"shop_zipcode":33602,"shop_city":"Bielefeld","shop_street":"Arndtstra\u00dfe","shop_snumber":3,"shop_name":"M\u00fcller","shop_desc":"Kaufhaus"}]
Here is an example for a rest api request:
http://shoqproject.supervisionbielefeld.de/public/gateway/gateway/get-shops-by-city/city/Bielefeld
So when you have your rest api set up you can deal with receiving that data with your android phone. I use a static method to get this data:
public class JsonGrabber{
public static JSONArray receiveData(){
String url = "your url";
String result = "";
DefaultHttpClient client = new DefaultHttpClient();
HttpGet method = new HttpGet(url);
HttpResponse res = null;
try {
res = client.execute(method);
} catch (ClientProtocolException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
}
try{
InputStream is = res.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
result = sb.toString();
}catch(Exception e){
Log.e("log_tag", "Error converting result "+e.toString());
}
JSONArray jArray = null;
try{
jArray = new JSONArray(result);
}catch(JSONException e){
Log.e("log_tag", "Error parsing data "+e.toString());
}
return jArray;
}
}
Well thats all, once you have your data in json format you just have to parse it:
JSONArray test = (JSONArray) JsonGrabber.receiveData()
try {
for(int i=0;i<test.length();i++){
JSONObject json_data = test.getJSONObject(i);
int id = json_data.getInt("id");
}
}
The web request should run in another thread, because it can be a time consuming process. So you need to deal with AsyncTask. Here are some resources:
Painless Threading
Multithreading for performance
Hello Android Tutorial