Okay, I know this question has been asked before, but all the answers did not help at all. First, let me recap the problem:
I have a (PHP-generated) JSON element, the PHP code looks like this:
if (!headers_sent()) header('Content-type: application/json');
echo "[{\"test\":\"success\"}]";
No other echo, print or anything else before that, so the output really is just
[{"test":"success"}]
and nothing else. The PHP file is saved as pure UTF-8 (checked and double-checked).
Now, my Android code looks as follows:
StringBuilder builder = new StringBuilder();
HttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("http://myurl");
try {
HttpResponse response = client.execute(httpGet);
StatusLine statusLine = response.getStatusLine();
int statusCode = statusLine.getStatusCode();
if (statusCode == 200) {
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(content, "UTF-8"), 8);
String line;
while ((line = reader.readLine()) != null) {
builder.append(line);
}
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
String menuData = builder.getString();
ArrayList<HashMap<String, String>> tempList = new ArrayList<HashMap<String, String>>();
boolean change = false;
try {
JSONArray jsonMenu = new JSONArray(menuData);
for (int i = 0; i < jsonMenu.length(); i++) {
JSONObject jsonObject;
try {
jsonObject = jsonMenu.getJSONObject(i);
HashMap<String, String> menuItem = new HashMap<String, String>();
menuItem.put("test", jsonObject.getString("test"));
tempList.add(menuItem);
change = true;
} catch (JSONException e) {
}
}
} catch (JSONException e) {
}
if(change) {
// here comes the listview update, but never triggered anyways
}
And I always get the same error message:
04-05 21:06:31.226: W/System.err(1208): org.json.JSONException: A JSONArray text must start with '[' at character 1 of ?[{"test":"success"}]
This is actually the Logcat output saved to log.txt and parsed here, and then all of the sudden there is this question mark. All attempts to trim or format the String before parsing it to the JSONArray have failed. I have absolutely no clue anymore what else to do. In Logcat in Eclipse itself there is no question mark by the way...
Added Log.e output to the catch statements, the error output is exactly the same. Screenshot here:
Problem solved:
Open the PHP file in Notepad++, click on Encoding => Encode in UTF-8 without BOM.
I neglected this because it shows "ANSI as UTF-8" in the status bar which sounded just wrong (I wanted pure UTF-8 and no ANSI mix). Yeah, I'm no Linux fella, so this ANSI/UTF-8 was something I never quite got to be honest, and here it trapped me again.
If someone is interested how I found the solution (credit goes to #Perception, #Raghav-Sood, and #Reuben-Scratton here):
Converted the menuData string to a byte array and found out that the first two chars are not the opening bracket that JSONArray is looking for, but two BOM chars.
Related
I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.
Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?
I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.
I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.
I'm building a simple Android application that sends items by json to a python web service for storage in a sqlite database. This part works fine and when recalled the items have their correct 'åäö' characters.
When I put these items in a json to return to the application (using code below) I'm not so sure any more. Instead of 'ö' I get '\xc3\xb6', which I believe would be the utf-8 representation?
connection = sqlite3.connect('database.db')
connection.text_factory = str
cursor = connection.cursor()
cursor.execute("SELECT item, number FROM Items")
rows = cursor.fetchall()
jsobj = []
for row in rows:
jsobj.append({'number':row[1], 'item':row[0]})
When I parse the json object in my app I'm not able to turn the '\xc3\xb6' back into 'ö'
This is the Android code at work:
HttpClient client = new DefaultHttpClient();
HttpGet post = new HttpGet(super.url);
HttpResponse response = client.execute(post);
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(content, "UTF-8"));
JSONArray itemsOnServer = new JSONArray();
itemsOnServer = new JSONArray(reader.readLine());
return itemsOnServer;
Result sent into another function:
ArrayList<Vara> varor = new ArrayList<Vara>();
String vara = "";
int antal;
for (int i = 0; i < json.length(); i++) {
JSONObject object;
try {
object = json.getJSONObject(i);
try {
try {
vara = new String(object.getString("vara").getBytes("ISO-8859-1"), "utf-8");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} System.out.println(vara);
antal = object.getInt("antal");
varor.add(new Vara(vara, antal));
} catch (JSONException e) {
e.printStackTrace();
}
} catch (JSONException e) {
e.printStackTrace();
}
}
return varor;
Everything works fine except the special characters. Please help, I'm going nuts.
b'\xc3\xb6' is a bytestring that can be decoded into Unicode string that contains a single U+00f6 LATIN SMALL LETTER O WITH DIAERESIS Unicode codepoint:
>>> b'\xc3\xb6'.decode('utf-8')
u'\xf6'
By default text_factory is set to unicode for sqlite3 i.e., you should receive Unicode string for TEXT objects if you remove connection.text_factory = str from your code.
Im doing a simple http get,
I see on my result an incomplete response,
what Im doing wrong?
here the code:
class GetDocuments extends AsyncTask<URL, Void, Void> {
#Override
protected Void doInBackground(URL... urls) {
Log.d("mensa", "bajando");
//place proper url
connect(urls);
return null;
}
public static void connect(URL[] urls)
{
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet("http://tiks.document.dev.chocolatecoded.com.au/documents/api/get?type=tree");
// Execute the request
HttpResponse response;
try {
response = httpclient.execute(httpget);
// Examine the response status
Log.d("mensa",response.getStatusLine().toString());
// Get hold of the response entity
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String result= convertStreamToString(instream);
// now you have the string representation of the HTML request
Log.d("mensa", "estratagema :: "+result);
JSONObject jObject = new JSONObject(result);
Log.d("mensa", "resposta jObject::"+jObject);
Log.d("mensa", "alive 1");
JSONArray contacts = null;
contacts = jObject.getJSONArray("success");
Log.d("mensa", "resposta jObject::"+contacts);
Log.d("mensa", "alive");
//instream.close();
}
} catch (Exception e) {}
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
Log.d("mensa", "linea ::"+line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
}
i call it with:
GetDocuments get = new GetDocuments();
URL url = null;
try {
url = new URL("ftp://mirror.csclub.uwaterloo.ca/index.html");
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//URL url = new URL("http://www.google.es");
get.execute(url);
edit 1
I refer to incomplete as the response that gets truncated?
please notice in below image of response how string gets truncated,
is this because of the log size?,
but the other problem is that it doesn't parse?
thanks!
I don't know if this is going to resolve your problem but you can get rid of your method and use simply:
String responseString = EntityUtils.toString(response.getEntity());
I've had exactly the same issue for the last couple of days. I found that my code worked over WiFi but not 3G. In other words I eliminated all the usual threading candidates. I also found that when I ran the code in the debugger and just waited for (say) 10 seconds after client.execute(...) it worked.
My guess is that
response = httpclient.execute(httpget);
is an asynchronous call in itself and when it's slow returns a partial result... hence JSON deserialization goes wrong.
Instead I tried this version of execute with a callback...
try {
BasicResponseHandler responseHandler = new BasicResponseHandler();
String json = httpclient.execute(httpget, responseHandler);
} finally {
httpclient.close();
}
And suddenly it all works. If you don't want a string, or want your own code then have a look at the ResponseHandler interface. Hope that helps.
I have confirmed that this is because size limit of java string. I have checked this by adding the string "abcd" with the ressponse and printed the response string in logcat. But the result is the truncated respose without added string "abcd".
That is
try {
BasicResponseHandler responseHandler = new BasicResponseHandler();
String json = httpclient.execute(httpget, responseHandler);
json= json+"abcd";
Log.d("Json ResponseString", json);
} finally {
httpclient.close();
}
So I put an arrayString to collect the response. To make array, I splitted My json format response by using "}"
The code is given below(This is a work around only)
BasicResponseHandler responseHandler = new BasicResponseHandler();
String[] array=client.execute(request, responseHandler).split("}");
Then you can parse each objects in to a json object and json array with your custom classes.
If you get any other good method to store response, pls share because i am creating custom method for every different json responses );.
Thank you
Arshad
Hi Now I am using Gson library to handle the responses.
http://www.javacodegeeks.com/2011/01/android-json-parsing-gson-tutorial.html
Thanks
Arshad
I cant' comment directly due to reputation, but in response to https://stackoverflow.com/a/23247290/4830567 I felt I should point out that the size limit of a Java String is about 2GB (Integer.MAX_VALUE) so this wasn't the cause of the truncation here.
According to https://groups.google.com/d/msg/android-developers/g4YkmrFST6A/z8K3vSdgwEkJ it is logcat that has a size limit, which is why appending "abcd" and printing in logcat didn't work. The String itself would have had the appended characters. The previously linked discussion also mentioned that size limits with the HTTP protocol itself can occasionally be a factor, but that most servers and clients handle this constraint internally so as to not expose it to the user.
I have a simple Android app that reads some JSON data from a php file on my local server. It's very simple and up until about an hour ago it was working fine. However, after a little break (I didn't even turn my PC off), it's suddenly struggling with what seems like a memory leak, although I'm no expert on the subject.
I'm not sure why all of a sudden this is happening, as I didn't change any code. But when I run the app my logcat slowly starts to fill with messages like this:
"GC_CONCURRENT freed, 910K, 53% free 3167K/6727K....."
The app eventually runs, but only after about 1-2 minutes of waiting. Anyway, I was wondering if someone could check my code or advise me what maybe causing this to happen.
public static JSONArray getJSONfromURL(String url) {
// initialize
InputStream is = null;
JSONArray jArray = null;
String result = "";
// http post
try {
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
is = entity.getContent();
} catch (Exception e) {
Log.e("log_tag", "Error in http connection " + e.toString());
}
// convert response to string
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "utf-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
result = sb.toString();
} catch (Exception e) {
Log.e("log_tag", "Error converting result " + e.toString());
}
// try parse the string to a JSON object
try {
jArray = new JSONArray(result);
} catch (JSONException e) {
Log.e("log_tag", "Error parsing data " + e.toString());
}
return jArray;
}
The JSON data isn't very large at all.
Thanks for any advice.
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, "utf-8"), 8);
You've allocated a buffer size of 8 bytes. That is very small. Unless you really know what you're doing, just use the default size with the alternative single-argument constructor:
public BufferedReader(Reader in)
Well I changed the 'While' loop to this and performance increased dramatically:
int len;
char[] chars = new char[4*1024];
while((len = reader.read(chars))>=0) {
sb.append(chars, 0, len);
}
It's still slower than I'd like, but at least I'm heading in the right direction. I'm going to work on it some more. Thanks everyone.
What's the easiest way to retrieve JSON from a web-service and to parse it? Without using any additional plug-ins?
This is the best way to retreive JSON data and parsing it, i have used it in my project and it works totally well. Check out this Tutorial (source code also available).
If you are using JSON, you will definitely need Google gson library to convert Java Objects into their JSON representation. You can download it from here.
If you want the standard procedure then you need to use JSONObject and JSONArray to parse responses. However, it is fuzzy and cumbersome when your response string contains a complex structure. In order to deal such complex responses its better to off the parsing to Gson, Jackson or any other library.
You should have a look here: http://code.google.com/p/android-query/wiki/AsyncAPI
Everything is explained, with the code samples you need.
The code below should give you a starting point, just use it in an activity/service class. Its gets data from a URL (webservice), and converts it into a JSON object you can use.
StringBuilder stringBuilder = new StringBuilder();
HttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(***Put your web service URL here ***)
try
{
HttpResponse response = client.execute(httpGet);
StatusLine statusLine = response.getStatusLine();
int statusCode = statusLine.getStatusCode();
if (statusCode == 200)
{
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(
new InputStreamReader(content));
String line;
while ((line = reader.readLine()) != null)
{
stringBuilder.append(line);
}
}
}
catch (ClientProtocolException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
//Turn string JSON into object you can process
JSONArray jsonArray = new JSONArray(stringBuilder.toString());
for (int i = 0; i < jsonArray.length(); i++)
{
//Get Each element
JSONObject jsonObject = jsonArray.getJSONObject(i);
//Do stuff
}