Develop Reference

Android Http Response is incomplete. Returns an unterminated json object

I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.

Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?

I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.

I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.

Getting snapshot image from IP Camera via URL

I am trying to get a picture from a Hikvision IP Camera URL which has a a URL that is more or less formed like this:http://IPaddress:port#/Streaming/channels/1/picture
However as you notice the folder is just named picture with no straight link to an image and its extensions. When I use Picasso library or just normal HttpConnection it fails to get the image / bitmap. How else can I retrieve the image? When I enter the URL in my web browser it loads perfectly and shows me that the picture is .jpeg format.EDIT
Whenever I try to access the snapshot image using InputStream and Bitmap I get a FileNotFoundException error

The problem as pointed out by Dario was the authentication. Status 401 was being returned and that is why the FileNotFoundException was being thrown. The answer he gave me had a lot of deprecated methods my Gradle couldnt even get some of the classes.However I found another way with help from the following forums 1, 2 so I combined with another page for the authentication which I cant find and had the following:
boolean isSaved = false;
try {
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(AppConst.IMAGE_URL));
UsernamePasswordCredentials credentials =
new UsernamePasswordCredentials(AppConst.USERNAME, AppConst.PASSWORD);
BasicScheme scheme = new BasicScheme();
Header authorizationHeader = scheme.authenticate(credentials, request);
request.addHeader(authorizationHeader);
HttpResponse response = client.execute(request);
Log.d("responseType",response!=null? response.getEntity().getContentType().toString():"emptyType");
Log.d("responseCode", response != null ? String.valueOf(response.getStatusLine().getStatusCode()) : "emptyCode");
if((response.getStatusLine().getStatusCode()==200) && (response.getEntity()!=null)){//status OK and not empty
//save stuff
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
String path = APP_DIR+System.currentTimeMillis()+".jpg";
FileOutputStream output = new FileOutputStream(path);
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = instream.read(buffer)) != -1) {
output.write(buffer, 0, len);
}
output.close();
Log.d("Snapshot Filename: ", path);
isSaved = true;
}
}else{
isSaved= false;
}
}catch (Exception ex){
Log.e("saveImageError",ex!=null? ex.getMessage():"error");
}
Had to add the following line in the gradle under the android section: useLibrary 'org.apache.http.legacy'

How to parse HTML full page in android

I am calling a HTML page via a web servise . I need to get hole source code of HTML page.
My problem is that, when I convert the http response to string I am getting only some part of HTML page. How do I can get hole HTML page .Please help me.
//paramString1 = url,paramString = header, paramList = paramiters
public String a(String paramString1, String paramString2, List paramList)
{
String str1 = null;
HttpPost localHttpPost = new HttpPost(paramString1);
localHttpPost.addHeader("Accept-Encoding", "gzip");
InputStream localInputStream = null;
try
{
localHttpPost.setEntity(new UrlEncodedFormEntity(paramList));
localHttpPost.setHeader("Referer", paramString2);
HttpResponse localHttpResponse = this.c.execute(localHttpPost);
int i = localHttpResponse.getStatusLine().getStatusCode();
localInputStream = localHttpResponse.getEntity().getContent();
Header localHeader = localHttpResponse.getFirstHeader("Content-Encoding");
if ((localHeader != null) && (localHeader.getValue().equalsIgnoreCase("gzip")))
{
GZIPInputStream localObject = null;
localObject = new GZIPInputStream(localInputStream);
Log.d("API", "GZIP Response decoded!");
BufferedReader localBufferedReader = new BufferedReader(new InputStreamReader((InputStream)localObject, "UTF-8"));
StringBuilder localStringBuilder = new StringBuilder();
while(true){
String str2 = localBufferedReader.readLine();
if (str2 == null)
break;
localHttpResponse.getEntity().consumeContent();
str1 = localStringBuilder.toString();
localStringBuilder.append(str2);
continue;
}
}
}
catch (IOException localIOException)
{
localHttpPost.abort();
}
catch (Exception localException)
{
localHttpPost.abort();
}
Object localObject = localInputStream;
return (String)str1;

Are you receiving the HTML in the variable paramString1?, in that case, are you encoding the String somehow or its just plane HTML?
Maybe the HTML special characters are breaking your response. Try encoding the String with urlSafe Base64 in your server side, and decoding it in the client side:
You can use the function Base64 of Apache Commons.
Server Side:
Base64 encoder = new Base64(true);
encoder.encode(yourBytes);
Client side:
Base64 decoder = new Base64(true);
byte[] decodedBytes = decoder.decode(paramString1);
HttpPost localHttpPost = new HttpPost(new String(decodedBytes));

You may not get the complete source code in your stringBuilder as it must be exceeding the max size of stringBuilder as StringBuilder is set of arrays. If u want to store that particular sourcecode. You may try this: The inputStream (which contains html source code) data, store directly into a File. Then you will have complete source code in that file and then perform file operation to whatever you require. See if this may help you.

How to post an image to Twitter using statuses/update_with_media in android

I need to post a image to Twitter. I have integrated Twitter in my app. I need to tweet the image as such not as an URL link. I don't want to use TwitPic.
I used the following code to create the multipart entity. It give 404 error.
Bitmap bm = null;
String encodedImage = "";
try {
URL aURL = new URL("http://50.57.227.117/blacksheep/uploaded/Detailed_images/961314275649aladdins.jpg");
URLConnection conn = aURL.openConnection();
conn.connect();
InputStream is = conn.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is, 8192);
bm = BitmapFactory.decodeStream(bis);
bis.close();
is.close();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bm.compress(Bitmap.CompressFormat.PNG, 100, baos);
imageBytes = baos.toByteArray();
encodedImage = Base64.encodeToString(imageBytes, Base64.DEFAULT);
Log.v("encodedImage >>",encodedImage);
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
HttpClient httpClient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost(
"https://api.twitter.com/1.1/statuses/update_with_media.json");
ByteArrayBody bab = new ByteArrayBody(imageBytes, "forest.jpg");
MultipartEntity reqEntity = new MultipartEntity(
HttpMultipartMode.BROWSER_COMPATIBLE);
reqEntity.addPart("media", bab);
reqEntity.addPart("status", new StringBody("test image"));
postRequest.setEntity(reqEntity);
HttpResponse response = httpClient.execute(postRequest);
BufferedReader reader = new BufferedReader(new InputStreamReader(
response.getEntity().getContent(), "UTF-8"));
String sResponse;
StringBuilder s = new StringBuilder();
while ((sResponse = reader.readLine()) != null) {
s = s.append(sResponse);
}
System.out.println("Response: " + s);
// Update status
//twitter4j.Status response = twitter.updateStatus(encodedImage);
// twitter4j.Status response1 = twitter.updateStatus(status);
//Log.d("Status", "> " + response1.getText());
} catch (TwitterException e) {
// Error in updating status
Log.d("Twitter Update Error", e.getMessage());
}

First attempt:
So it seemed like you were using Apache's old http client to make a request external to the library you were using to help integrate with twitter, twitter4j. I assumed you were using a version prior to 3.03 which is latest, and didn't want you to upgrade. You see, update_with_media is quite new, so I didn't think your version had implemented it.
The problem with what you were doing is that twitter uses oauth for authentication. So you'd need to "sign" a request with the access token you'd obtained. Twitter4j, AFAIK, does this for you. You can't use a seperate client to make some calls without reference to your nice helper library without breaking authentication.
The endpoint, ../update_with_media is defined to update a status for the currently authenticating user. I suspect that, since there was no access token and no user in your request, that endpoint doesn't even make sense, so twitter were interpreting it as a 404 (not found) rather than a 401 (unauthorized)- funny.
So the first attempt was not to require you to upgrade to twitter4j. It's a pain to upgrade sometimes! Instead, you can hack with the library as is detailed with this blog. But that wasn't easy as the libraries were different.
So, something else we could try, if you really wanted to make a seperate request to twitter4j, was to actually do the signing, perhaps using scribe to make it easier.... roughly:
final OAuthService myTwitterService = TwitterClient.getTwitterClient().getService();
final OAuthRequest aNiceOAuthRequest = new org.scribe.model.OAuthRequest(
YOURPOST, THATURL);
etc.
Second attempt:
But let's not do all this- turns out you had the latest version of twitter4j anyway. Sorry for going down a cul-de-sac first- I shouldn't have assumed, but I've included the above for help for anybody else should they need it.
It turns out the latest version has implemented this endpoint- documentation here. Except it takes a StatusUpdate object instead. So you want to do something like:
final StatusUpdate statusUpdate = new StatusUpdate("Hallee hallo my status java.lang.String here...");
// now do as you did until:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bm.compress(Bitmap.CompressFormat.PNG, 100, baos);
imageBytes = baos.toByteArray();
encodedImage = Base64.encodeToString(imageBytes, Base64.DEFAULT);
// then flip the stream
byte[] myTwitterUploadBytes = bos.toByteArray();
ByteArrayInputStream bis = new ByteArrayInputStream(myTwitterUploadBytes);
// doo and double check your encoding etc, similar to in your question..
statusUpdate.setMedia("give me a java.lang.String name", bis);
// then continue just using twitter for j- update status as you would
//... get twitter etc...
//
twitter4j.Status response = twitter.updateStatus(statusUpdate);
Haven't currently got a box on me to test- should be about right. If it still gives 404s, what is the error code in the response? Are you authenticated?
If that doesn't work, we can try some of the above too as a back up.
Hope this helps,
best,
Tom.

Using 4.0.3 (perhaps earlier) it's very simple to embed images in a tweet with twitter4j:
Twitter twtobj;
StatusUpdate stsupd;
Status stsres;
twtobj=twitterFactory.getInstance();
twtobj.setOAuthConsumer(csmkey,csmsec);
twtobj.setOAuthAccessToken(new AccessToken(acstkn,acssec));
stsupd=new StatusUpdate(msgtxt);
if(medurls.length>0) {
long[] medidns=new long[medurls.length];
for(int xa=0; xa<medurls.length; xa++) {
String medurl=Util.resolveRelativeUrl(medurls[xa]);
InputStream imgstm=null;
try {
imgstm=new URL(medurl).openConnection().getInputStream();
medidns[xa]=twtobj.uploadMedia(medurl,imgstm).getMediaId(); // this actually uploads the image to Twitter at this point
}
catch(MalformedURLException thr) { throw new ShfFail(Fail.IMAGE_URL ,"The media URL is not valid: " +medurl+" ("+thr.getMessage()+")"); }
catch(IOException thr) { throw new ShfFail(Fail.IMAGE_READ,"The media could not be read: "+medurl+" ("+thr.getMessage()+")"); }
finally { GenUtil.close(imgstm); }
}
stsupd.setMediaIds(medidns);
}
stsres=twtobj.updateStatus(stsupd);
Note that up to 4 images, or 1 animated GIF, or 1 video are allowed, as of 2015-06-10.
Note also that I am capturlng the image streams to close them explicitly in the outer block (not shown). This may be unnecessary, but I can't find positive confirmation of that.
If anyone cares, resolveRelativeUrls is a convenience to allow a relative path to be resolved as a file URL from the current folder:
static public String resolveRelativeUrl(String url) {
if(!TextUtil.stringCT(url,"://")) {
url=new File(url).getAbsoluteFile().toURI().toString();
}
return url;
}
The utility method stringCT is case-insensitive contains.

Android: HttpURLConnection redirect - doInBackground

I'm using HttpURLConnection to do communication with a backend server and im doing so in an async task in the doInBackground method as you should.
Now I need to be able to follow 302 redirects, but I'm having some problems with this. The issue is that the new location usually will be on another host, however when doing the redirect request it seem not to change the URL to a new host hence I get a 404 error saying the specified path does not exits.
Now I know I could set HtppURLConnection.setFollowRedirect but I need to have more control over the redirects so they should not just be followed blindly. The Redirect behavour should be controlled by the object who called the asynctask (when an asynctask object is created you pass the object who creates it in a parameter called _callback).
Heres's my current code:
protected HttpResponse doInBackground(String... req) {
HttpURLConnection urlConnection = null;
try {
urlConnection = (HttpURLConnection) this._url.openConnection();
urlConnection.setConnectTimeout( (int) this._timeout*1000);
String body = req[0];
// set headers / write information to output stream if request is post
// create the response object
HttpResponse responseObject = null;
try
{
// get status, contenttype, charset...
InputStream in = null;
if (urlConnection.getResponseCode() != -1 && urlConnection.getResponseCode() < 300)
{
in = new BufferedInputStream(urlConnection.getInputStream(), 8192);
}
else
{
in = new BufferedInputStream(urlConnection.getErrorStream(), 8192);
}
responseObject = new HttpResponse(in, status, contentType, charset);
// if redirect
if (status == 302 && this._callback.onRedirect(responseObject) == true)
{
// recall
String url = urlConnection.getHeaderField("location");
Log.v("Async Task", "Redirect location: " + url);
this._url = null;
this._url = new URL(url);
urlConnection.disconnect();
urlConnection = null;
responseObject = this.doInBackground(req);
}
} catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
// return the response
return responseObject;
}
// catch some other exceptions
finally
{
if (urlConnection != null)
{
urlConnection.disconnect();
} }
}
And as said the problem is that the redirect request seem to change the path of the URL but not the host. The URL object itself seem to contain the right information so I have no idea why this is happening. (I'm getting HTML as response which is an 404 error page that includes the server name of the old server)
Thanks for any help!
Note: HttpResponse is just an object I created for holding the relevant information about the response.

This was caused by the fact that I sent the same headers and did not change the "host" header of the request which caused Apache to be confused it seems.