Develop Reference

Android - large data json download

I'm facing a problem in my app Android that consists in a failure to recover large json data from web service. The JSON data is large because contains images which are recovered from a table in my database.
Some days ago everything was working fine, but the number of registers in this table grew up fastly and then the problem rises.
This problem is not happening on iOS or on Android emulator, just on real Android device.
The code is stopping the download suddenly and throwing the error
org.json.JSONException: Unterminated string at character <cccc> of <json data>
That is, the code is not downloading the entire JSON data. Besides, the download stops in a different point always I run the code.
Someone know why that is happening?
This is the function which tries to recover the data from web service:
protected static JSONObject executeJSONQuery(ContentValues values,
WebServiceResolverCode code, Context context)
throws ContratoInativoException, JSONException {
URL url;
HttpURLConnection conn;
JSONObject jArray = null;
// build the string to store the response text from the server
String response = "";
try {
url = new URL(WebServiceResolver.getWebServiceSufix(code, context));
String param = "";
for (String key : values.keySet()) {
Object value = values.get(key);
param += key+"="+ URLEncoder.encode(value!=null?values.getAsString(key):"", "UTF-8")+"&";
}
conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setFixedLengthStreamingMode(param.getBytes().length);
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
// send the POST out
OutputStreamWriter out = new OutputStreamWriter(conn.getOutputStream());
out.write(param);
out.flush();
out.close();
// start listening to the stream
Scanner inStream = new Scanner(conn.getInputStream());
while (inStream.hasNextLine()){
response += (inStream.nextLine());
}
inStream.close();
// process the stream and store it in StringBuilder
jArray = new JSONObject(response);
} catch (IOException ex) {
ex.printStackTrace();
Log.d("ERROR JSON STRING", response);
}
return jArray;
}

Firstly, do not keep an image in a table. Keep it in your server space and keep the URL to the image in your table. When you need to load the image, load it with the URL kept in the table.
Secondly, JSON is a light weight data interchange format. You should limit the number of items it carries. Its advisable not to let it carry up to a hundred rows in your table(when converting to JSON). If you have to, then load a little at first, then load another little, till it gets to the end. That's the idea behind infinite scrolling.
That way, it loads your data efficiently.

Looks like your json string object is not forming correctly. It may be missing s double quote at the end of a string. Check the param string object.

Parsing json object from google api returning null

I am making a simple application where i scan the barcode of a book and fetch its title and author from Google APIs,
Now, this is the url for json(for a particular book i am scanning)
https://www.googleapis.com/books/v1/volumes?q=isbn:9788120305960
using this code to get json in a string
HttpURLConnection urlConnection = (HttpURLConnection)url.openConnection();
InputStream inputStream = urlConnection.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream,"iso-8859-1"));
String line = "";
while ((line=bufferedReader.readLine())!=null)
{
response+=line;
}
bufferedReader.close();
inputStream.close();
urlConnection.disconnect();
Log.d("Info",response);
return response;
I store the result in a string and use this code to parse through
(json_response is a string)
JSONObject rootObject = new JSONObject(json_response);
JSONArray items = rootObject.getJSONArray("items");
JSONObject items_object = items.getJSONObject(0);
JSONObject volume_info = items_object.getJSONObject("volumeInfo");
book.setTitle(volume_info.getString("title"));
JSONArray authors = volume_info.getJSONArray("authors");
Log.d("Info","authors array length: "+authors.length());
String author="";
for (int i =0;i<authors.length();i++)
{
author+=authors.getString(i)+", ";
}
book.setAuthor(author);
The exception is:
Value null of type org.json.JSONObject$1 cannot be converted to JSONObject
also I used logcat to see what is contained in json_response it looks something like this
null{ "kind": "books#volumes", "totalItems": 1, "items":...
The null here is probably causing the problem, so... any insights how to deal with this???
PS: I am a student , dealing first time with json and android, code is unprofessional, please pardon :)

Having
null{ "kind": "books#volumes", "totalItems": 1, "items":...
means that the response value has not been initialised.
You should therefore initialise it to empty string.

Getting json response from servlet to Android

I have a servlet that has the following purpose:
Receive data via the URL (that is, using get). Then returns a message, based on this input, back to the caller. I am new to this stuff, but have come to learn that using json (actually, Gson) is suitable for this.
My question now is, how do I retrieve this json message? What URL do I target? The relevant lines in the servlet are:
String json = new Gson().toJson(thelist);
response.setContentType("application/json");
response.setCharacterEncoding("UTF-8");
response.getWriter().println(json);
This is how I try to retrieve the json:
try{
DefaultHttpClient defaultClient = new DefaultHttpClient();
HttpGet httpGetRequest = new HttpGet("http://AnIPno:8181/sample/response?first=5&second=92866");
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
String json = reader.readLine();
JSONObject jsonObject = new JSONObject(json);
} catch(Exception e){
e.printStackTrace();
}
But apparently this does not work, as I have found jsonObject has a size of 0 (it should be an array with three elements).
Previously, I had a write() instead of println() in the servlet. I'm not sure if that matters in this case. But I'm assuming I've misunderstood something about how the json object is retrieved. Is it not enough to point it towards the URL of the servlet?

Reading an InputStream whether from a File on the file system or from an HTTP request is, in most cases, the same.
What you have is correct only if your servlet wrote a single line. If the Gson object toString() method returns multiple lines, you're going to have to read multiple lines from the InputStream. I like to use the Scanner class for reading from an InputStream.
try {
DefaultHttpClient defaultClient = new DefaultHttpClient();
HttpGet httpGetRequest = new HttpGet("http://localhost:8080/cc/jsonyeah");
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
Scanner scanner = new Scanner(httpResponse.getEntity().getContent(), "UTF-8");
while(scanner.hasNextLine()) { // scanner looks ahead for an end-of-line
json += scanner.nextLine() + "\n"; // read the full line, you can append a \n
}
// do your serialization
} catch(Exception e) {
e.printStackTrace();
}
So we've done the same thing we would've done if we were reading from a file. Now the json object contains the json you received from the servlet, as a String.
For the serialization, you have a few options.
A Gson object has an overloaded method fromJson() that can take a String or a Reader, among other things.
From where we are with the code above, you can do
MyClass instance = new Gson().fromJson(json, MyClass.class);
where MyClass is the type you are trying to create. You will have to use a TypeToken for generic classes (such as a list). TypeToken is an abstract class, so generate an anonymous class and call getType()
Type type = new com.google.gson.reflect.TypeToken<List<String>>(){}.getType();
List<MyClass> list = new Gson().fromJson(json, type);
Another option is to use the overloaded method that takes a Reader directly instead of reading line by line from the InputStream:
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
MyClass instance = new Gson().fromJson(reader , MyClass.class);
You'll get to skip a step.
Don't forget to close your streams.

I have this function to readJsonData from a a request to a JSON String. You can use this function to retrieve the JSON, then use GSON to parse it to the object that you like. It works for my application. Hope it works for you too.
protected String readJson(HttpResponse resp)
throws IOException {
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(
resp.getEntity().getContent()));
StringBuffer buffer = new StringBuffer();
int read;
char[] chars = new char[1024];
while ((read = reader.read(chars)) != -1)
buffer.append(chars, 0, read);
} finally {
if (reader != null)
reader.close();
}
return buffer.toString();
}
So based on your code. I guess this should work:
String jsonData = readJson(httpResponse);
YourObject obj = new Gson().fromJson(jsonData, YourObject.class);
Before trying this, make sure your servlet prints out the JSON data that you want. I suggest using these Chrome Extensions: Postman - REST Client and JSON Formatter, to test your data from servlet. It's pretty helpful.

Parsing JSON remote call and store it locally

I need to parse a remote response from a URL that retrieves a big JSON object and store it locally so we can perform searches for example, in an activity in Android.
I've been trying to look it up but I can't find any approach about how to do it.
Any suggestion/help please?
Thanks a lot in advance!

You can use Gson parsing to parse the JSON from url.
For Storing the result use sqlite Database.
Here is the good example for GSON parsing
For sqlite follow this link

the retrieved JSON response is stored in the BufferReader i.e.,
urlClientStream = openHttpClientPostConnection(searchURL,
urlParams.toString());
BufferedReader reader = new BufferedReader(
new InputStreamReader(urlClientStream), 8192);
String line = null;
while ((line = reader.readLine()) != null) {
buffer.append(line);
}
JSONObject mainObj = new JSONObject(buffer.toString())
.getJSONObject("RESPONSE");
Hope this helps you

Setting TextView's text as non-English text received from JSON response

I'm getting back names of (Foursquare) venues from a server call where the names of the venues returned can be in English or non-English.
Assume the venue name is in a JSON object as follows:
{...
"name":"venue name which can be in any language"
...}
I'm creating a JSONObject from this response and then pulling out the name of the venue as follows:
String name = jsonObject.getString("name");
Lastly, I'm setting the TextView's text to show the name of the venue as follows:
myTextView.setText(name);
I'm finding however for Arabic names that where the Arabic characters are joined in the original JSON object (as they should be), the characters that show in the app (i.e. in the TextView) are disjoint. (I'm not too familiar with other languages so can't really tell if they're showing incorrectly too.)
Is there something additional I should be doing to pull out non-English names correctly from the JSON object and setting it as the text of a TextView, or is it down to the phone to decide how the text will be displayed?
Edit: I've tried parsing the server response (as suggested by #bbedward) explicitly specifying the content encoding as UTF-8 as follows...
HttpEntity httpEntity = httpResponse.getEntity();
String responseMessage = EntityUtils.toString(myHttpEntity, "UTF-8");
JSONObject jsonObject = new JSONObject(responseMessage);
... but still no joy. (Arabic characters appear, as before, disjoint in words where they should be joint up.) Could it be a phone thing or is there something extra needing to be done myself to get the words/characters to show proper in non-English languages? Perhaps the server needs to explicitly specify a "Content-Type" header with value "UTF-8"?

I'm going to answer anyway, I'm guessing you aren't getting your json in UTF-8 as i had a similar problem, I believe json won't come any other way.
Complete Example
The only things to concern yourself with this is setting the encoding for the InputStreamReader and creating the JSONObject
private DefaultHttpClient httpclient = new DefaultHttpClient(new BasicHttpParams());
HttpPost httppost = new HttpPost("http://myjsonurl.com/search?type=json");
// Depending on your web service
httppost.setHeader("Content-type", "application/json");
try
{
String result = null;
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
InputStream inputStream = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, "utf-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null)
{
sb.append(line + "\n");
}
JSONObject myJObject = new JSONObject(sb.toString();
}
catch (Exception e)
{
}
finally
{
try{if(inputStream != null)inputStream.close();}catch(Exception none){}
}

add this line when you connect to mysql:
mysql_set_charset('utf8', $con);
ex:
$con = mysql_connect(DB_HOST, DB_USER, DB_PASSWORD);
mysql_set_charset('utf8', $con);
mysql_select_db(DB_DATABASE);