Develop Reference

Android Http Response is incomplete. Returns an unterminated json object

I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.

Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?

I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.

I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.

Android:Http get fetches old response

Below is the method which I have used to read a response from server by passing a url which retrieved a json object.
There is a very peculiar issue which is still old values has been fetched , though the data has already been updated.
I tried to find out the solution of it but still no success.
Url is of type: http://www.mywebsite.com/svc/user_auth/user_id
Where a user id is the unique integer id of the user which is being passed as a parameter.
public static String getResponse(String url){
String downloadedData = null;
Log.e("getResponse", url);
try {
URL downloadURL = new URL(url);
InputStream inputStream = (InputStream) downloadURL.getContent();
if (null != inputStream) {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
byte[] buffer = new byte[512];
int readCounter = inputStream.read(buffer);
while (readCounter != -1) {
byteArrayOutputStream.write(buffer, 0, readCounter);
readCounter = inputStream.read(buffer);
}
downloadedData = new String(
byteArrayOutputStream.toByteArray());
/*if (null != downloadedData && !"".equals(downloadedData)) {
downloadedJson = new JSONObject(downloadedData);
}*/
}else{
Log.e("getResponse", "Response is null");
}
} catch (Exception e) {
e.printStackTrace();
}
return downloadedData;
}
Any help would be appreciated.

Don't know how is URL.getContent() implemented internally. You can try to use URLConnection, it allows you to control whether to use cache by setUseCaches().

Have you tried opening the url from your browser to see the JSON response? If you see the old value in the browser then the problem is on your server side.
If this is not the case try using the DefaultHttpClient for post and get requests like: (maybe some caching occur with the method you are using, this is not the case for sure with HttpClient)
DefaultHttpClient httpclient = getHttpClient();
HttpPost httpost = new HttpPost(url);
httpost.setEntity(entity); //entity has your params wrapped if you have any...this is just an example for POST request, but for your case you can use GET with URL params..
httpclient.execute(httpost);

Android Authentication scheme ntlm not supported

I am using asynhttpClient for basic authentication
http://loopj.com/android-async-http/
that is looj lib..
below is my code:
usernameRandomPassword = userName + ":" + password;
Log.d("username=",usernameRandomPassword);
Log.d("url=",url);
String authorization = "Basic " + Base64.encodeToString(usernameRandomPassword.getBytes("UTF-8"), Base64.NO_WRAP);
httpClient.addHeader("Authorization",authorization);
httpClient.addHeader("Content-type", "application/json");
httpClient.setTimeout(20000);
httpClient.get( url, new AsyncHttpResponseHandler() {
#Override
public void onStart() {
System.out.println("on satrt");
super.onStart();
}
#Override
public void onSuccess(int statusCode, Header[] headers, byte[] responseBody) {
System.out.println("on onSuccess statusCode="+statusCode);
toastmessgae("onSuccess status code="+statusCode);
super.onSuccess(statusCode, headers, responseBody);
}
#Override
public void onFailure(int statusCode, Header[] headers, byte[] responseBody, Throwable error) {
System.out.println("on onFailure="+statusCode);
toastmessgae("onFailure status code="+statusCode);
super.onFailure(statusCode, headers, responseBody, error);
}
#Override
public void onFinish() {
System.out.println("on onFinish");
super.onFinish();
}
});
} catch (UnsupportedEncodingException e) {
}
but i always receive in console 401, below are logs
Authentication scheme ntlm not supported.
Unable to respond to any of these challenges: {ntlm=WWW-Authenticate: NTLM, negotiate=WWW-Authenticate: Negotiate}
The credentials are correct i checked on direct link.
I have spent already a complete day on this, can any one help me?
If you share some example,it will be really helpful.
Thanks in advance..

here is the answer through code:
add below code to your android file
DefaultHttpClient httpclient = new DefaultHttpClient();
// register ntlm auth scheme
httpclient.getAuthSchemes().register("ntlm", new NTLMSchemeFactory());
httpclient.getCredentialsProvider().setCredentials(
// Limit the credentials only to the specified domain and port
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
// Specify credentials, most of the time only user/pass is needed
new NTCredentials(username, password, "", "")
);
HttpUriRequest httpget = new HttpGet(your_URL);
HttpResponse response = httpclient.execute(httpget);
String responseBody = EntityUtils.toString(response.getEntity());
Log.i(tag,"responseBody =>>>>>>>>>>"+responseBody);
now download lib and java file from
https://github.com/masconsult/android-ntlm
and copy jcifs-1.3.17.jar to your lib folder and
JCIFSEngine and NTLMSchemeFactory to your package. (you can change package if you want..)
Thats it your app is ready to run.
More useful Links:
http://www.developergarden.com/en/marketplace/components/details/cmp/android-ntlm-authentication/

Looks to me that you are possibly behind a proxy? NTLM looks to be a largely undocumented Microsoft protocol:
http://www.innovation.ch/personal/ronald/ntlm.html
You cannot simply use Basic Auth because this is some different authentication scheme required by the server to whom you are speaking, or by a proxy in between you and your destination.

I have same problem. I want to use android asynk http, but i not found ntlm auth
I found solution:
1)use above answer,download and import jcifs-1.3.17.jar
2)then i download https://github.com/loopj/android-async-http (not JAR file) and import in my project,
3) then in file AsynkHttpClient.java
after
httpClient = new DefaultHttpClient(cm, httpParams);
insert
httpClient.getAuthSchemes().register("ntlm", new NTLMSchemeFactory());
httpClient.getCredentialsProvider().setCredentials(
// Limit the credentials only to the specified domain and port
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
// Specify credentials, most of the time only user/pass is needed
new NTCredentials("username", "pass","", "")
);
!!!!!!! then very important
you must comment like this
httpClient.addRequestInterceptor(new HttpRequestInterceptor() {
#Override
public void process(final HttpRequest request, final HttpContext context) throws HttpException, IOException {
AuthState authState = (AuthState) context.getAttribute(ClientContext.TARGET_AUTH_STATE);
CredentialsProvider credsProvider = (CredentialsProvider) context.getAttribute(
ClientContext.CREDS_PROVIDER);
HttpHost targetHost = (HttpHost) context.getAttribute(ExecutionContext.HTTP_TARGET_HOST);
// if (authState.getAuthScheme() == null) {
// AuthScope authScope = new AuthScope(targetHost.getHostName(), targetHost.getPort());
// Credentials creds = credsProvider.getCredentials(authScope);
// if (creds != null) {
// authState.setAuthScheme(new BasicScheme());
// authState.setCredentials(creds);
// }
// }
}
}, 0);
that all you need :)

here is the full fledged working for this.
try
{
DefaultHttpClient httpclient = new DefaultHttpClient();
// register ntlm auth scheme
httpclient.getAuthSchemes().register("ntlm", new NTLMSchemeFactory());
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),
new NTCredentials("username","password"));
//xx = ip address yy = port
HttpPost httpPost = new HttpPost("http://xx.xx.xx.xx:yy/");
Log.e(TAG, "executing request" + httpPost.getRequestLine());
HttpResponse response = httpclient.execute(httpPost);
HttpEntity entity = response.getEntity();
Log.e(TAG, "" + response.getStatusLine());
if (entity != null)
{
Log.e(TAG, "Response content length: " + entity.getContentLength());
}
if (entity != null)
{
Log.e(TAG, "Response stream: " + getMessage(entity.getContent()));
entity.consumeContent();
}
}
catch (Exception e)
{
Log.e(TAG, "" + e.getMessage());
}
please note to include these imports only
import java.net.HttpURLConnection;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.NTCredentials;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.auth.NTLMSchemeFactory;
import org.apache.http.impl.client.DefaultHttpClient;
and do use httpclient-android-4.3.5.1.jar only.

parse Multipart response in Android

I'm sending images and json text from the android client to a tomcat server and the other way around by using Multipart HttpPost's. Sending a Multipart Entity to the server is no big deal, because you can process the parts easily using request.getPart(<name>). But at the client side you can only access the response as a Stream. So I end up appending both, the JSON string and the image to the same ServletOutputStream and have to parse them by hand on the client side. I found apache-mime4j in the web but its hardly documented and I cant find a single example how to use it.
On the server side I build the response like this:
ServletResponse httpResponse = ctx.getResponse();
ResponseFacade rf = (ResponseFacade) httpResponse;
rf.addHeader("Access-Control-Allow-Origin", "*");
rf.addHeader("Access-Control-Allow-Methods", "POST");
rf.addHeader("content-type", "multipart/form-data");
httpResponse.setCharacterEncoding("UTF-8");
MultipartResponse multi = new MultipartResponse((HttpServletResponse) httpResponse);
ServletOutputStream out = httpResponse.getOutputStream();
multi.startResponse("text/plain");
out.println(CMD + "#" + content);
multi.endResponse();
multi.startResponse("image/jpeg");
out.write(data);
multi.endResponse();
multi.finish();
ctx.complete();
And on the client side on Android I want to access the text and the image data:
InputStream is = response.getEntity().getContent();
MimeStreamParser parser = new MimeStreamParser();
MultipartContentHandler con = new MultipartContentHandler();
parser.setContentHandler(con);
try {
parser.parse(is);
String json = con.getJSON(); //get extracted json string
byte[] imgBytes = con.getBytes(); //get extracted bytes
} catch (MimeException e) {
e.printStackTrace();
} finally {
is.close();
}
class MultipartContentHandler implements ContentHandler{
public void body(BodyDescriptor bd, InputStream in) throws MimeException, IOException {
//if MIME-Type is "text/plain"
// process json-part
//else
// process image-part
}
In the method body(BodyDescriptor bd, InputStream in) my whole response is treated as text\plain mime type. So I finally have to parse every byte manually again and the whole apache-mime4j is useless. Can you tell me what I am doing wrong? Thanks!

Ok i finally solved it myself. No here's what i did:
First I need to create a multipart/mixed Response at the server side. It can be done using apache-mime-4j API:
ServletResponse httpResponse = ctx.getResponse();
ResponseFacade rf = (ResponseFacade) httpResponse;
httpResponse.setCharacterEncoding("UTF-8");
httpResponse.setContentType("multipart/mixed");
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE, "SEPERATOR_STRING",Charset.forName("UTF-8"));
entity.addPart("json", new StringBody(CMD + "#" + content, "text/plain", Charset.forName("UTF-8")));
entity.addPart("image", new ByteArrayBody(data, "image/jpeg", "file"));
httpResponse.setContentLength((int) entity.getContentLength());
entity.writeTo(httpResponse.getOutputStream());
ctx.complete();
Now at the client side to access the MIME-Parts of the HttpResponse I use the javax.mail API.
ByteArrayDataSource ds = new ByteArrayDataSource(response.getEntity().getContent(), "multipart/mixed");
MimeMultipart multipart = new MimeMultipart(ds);
BodyPart jsonPart = multipart.getBodyPart(0);
BodyPart imagePart = multipart.getBodyPart(1);
But you can't use the native API, instead take this one http://code.google.com/p/javamail-android/
Now you can proceed handling your individual parts.

It is also possible with apache-mime-4j:
HttpURLConnection conn = ...;
final InputStream is = conn.getInputStream();
try {
final StringBuilder sb = new StringBuilder();
sb.append("MIME-Version: ").append(conn.getHeaderField("MIME-Version")).append("\r\n");
sb.append("Content-Type: ").append(conn.getHeaderField("Content-Type")).append("\r\n");
sb.append("\r\n");
parser.parse(new SequenceInputStream(new ByteArrayInputStream(sb.toString().getBytes("US-ASCII")), is));
} catch (final MimeException e) {
e.printStackTrace();
} finally {
is.close();
}

Sending images using Http Post

I want to send an image from the android client to the Django server using Http Post. The image is chosen from the gallery. At present, I am using list value name Pairs to send the necessary data to the server and receiving responses from Django in JSON. Can the same approach be used for images (with urls for images embedded in JSON responses)?
Also, which is a better method: accessing images remotely without downloading them from the server or downloading and storing them in a Bitmap array and using them locally? The images are few in number (<10) and small in size (50*50 dip).
Any tutorial to tackle these problems would be much appreciated.
Edit: The images chosen from the gallery are sent to the server after scaling it to required size.

I'm going to assume that you know the path and filename of the image that you want to upload. Add this string to your NameValuePair using image as the key-name.
Sending images can be done using the HttpComponents libraries. Download the latest HttpClient (currently 4.0.1) binary with dependencies package and copy apache-mime4j-0.6.jar and httpmime-4.0.1.jar to your project and add them to your Java build path.
You will need to add the following imports to your class.
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntity;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.entity.mime.content.StringBody;
Now you can create a MultipartEntity to attach an image to your POST request. The following code shows an example of how to do this:
public void post(String url, List<NameValuePair> nameValuePairs) {
HttpClient httpClient = new DefaultHttpClient();
HttpContext localContext = new BasicHttpContext();
HttpPost httpPost = new HttpPost(url);
try {
MultipartEntity entity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
for(int index=0; index < nameValuePairs.size(); index++) {
if(nameValuePairs.get(index).getName().equalsIgnoreCase("image")) {
// If the key equals to "image", we use FileBody to transfer the data
entity.addPart(nameValuePairs.get(index).getName(), new FileBody(new File (nameValuePairs.get(index).getValue())));
} else {
// Normal string data
entity.addPart(nameValuePairs.get(index).getName(), new StringBody(nameValuePairs.get(index).getValue()));
}
}
httpPost.setEntity(entity);
HttpResponse response = httpClient.execute(httpPost, localContext);
} catch (IOException e) {
e.printStackTrace();
}
}

Version 4.3.5 Updated Code
httpclient-4.3.5.jar
httpcore-4.3.2.jar
httpmime-4.3.5.jar
Since MultipartEntity has been deprecated. Please see the code below.
String responseBody = "failure";
HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
String url = WWPApi.URL_USERS;
Map<String, String> map = new HashMap<String, String>();
map.put("user_id", String.valueOf(userId));
map.put("action", "update");
url = addQueryParams(map, url);
HttpPost post = new HttpPost(url);
post.addHeader("Accept", "application/json");
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.setCharset(MIME.UTF8_CHARSET);
if (career != null)
builder.addTextBody("career", career, ContentType.create("text/plain", MIME.UTF8_CHARSET));
if (gender != null)
builder.addTextBody("gender", gender, ContentType.create("text/plain", MIME.UTF8_CHARSET));
if (username != null)
builder.addTextBody("username", username, ContentType.create("text/plain", MIME.UTF8_CHARSET));
if (email != null)
builder.addTextBody("email", email, ContentType.create("text/plain", MIME.UTF8_CHARSET));
if (password != null)
builder.addTextBody("password", password, ContentType.create("text/plain", MIME.UTF8_CHARSET));
if (country != null)
builder.addTextBody("country", country, ContentType.create("text/plain", MIME.UTF8_CHARSET));
if (file != null)
builder.addBinaryBody("Filedata", file, ContentType.MULTIPART_FORM_DATA, file.getName());
post.setEntity(builder.build());
try {
responseBody = EntityUtils.toString(client.execute(post).getEntity(), "UTF-8");
// System.out.println("Response from Server ==> " + responseBody);
JSONObject object = new JSONObject(responseBody);
Boolean success = object.optBoolean("success");
String message = object.optString("error");
if (!success) {
responseBody = message;
} else {
responseBody = "success";
}
} catch (Exception e) {
e.printStackTrace();
} finally {
client.getConnectionManager().shutdown();
}

The loopj library can be used straight-forward for this purpose:
SyncHttpClient client = new SyncHttpClient();
RequestParams params = new RequestParams();
params.put("text", "some string");
params.put("image", new File(imagePath));
client.post("http://example.com", params, new TextHttpResponseHandler() {
#Override
public void onFailure(int statusCode, Header[] headers, String responseString, Throwable throwable) {
// error handling
}
#Override
public void onSuccess(int statusCode, Header[] headers, String responseString) {
// success
}
});
http://loopj.com/

I struggled a lot trying to implement posting a image from Android client to servlet using httpclient-4.3.5.jar, httpcore-4.3.2.jar, httpmime-4.3.5.jar. I always got a runtime error. I found out that basically you cannot use these jars with Android as Google is using older version of HttpClient in Android. The explanation is here http://hc.apache.org/httpcomponents-client-4.3.x/android-port.html. You need to get the httpclientandroidlib-1.2.1 jar from android http-client library. Then change your imports from or.apache.http.client to ch.boye.httpclientandroidlib. Hope this helps.

I usually do this in the thread handling the json response:
try {
Bitmap bitmap = BitmapFactory.decodeStream((InputStream)new URL(imageUrl).getContent());
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
If you need to do transformations on the image, you'll want to create a Drawable instead of a Bitmap.