Hi there i'm creating my first android app and i'm wanting to know what is the best and most efficient way of parsing a JSON Feed from a URL.Also Ideally i want to store it somewhere so i can keep going back to it in different parts of the app. I have looked everywhere and found lots of different ways of doing it and i'm not sure which to go for. In your opinion whats the best way of parsing json efficiently and easily?
I'd side with whatsthebeef on this one, grab the data and then serialize to disk.
The code below shows the first stage, grabbing and parsing your JSON into a JSON Object and saving to disk
// Create a new HTTP Client
DefaultHttpClient defaultClient = new DefaultHttpClient();
// Setup the get request
HttpGet httpGetRequest = new HttpGet("http://example.json");
// Execute the request in the client
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
// Grab the response
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
String json = reader.readLine();
// Instantiate a JSON object from the request response
JSONObject jsonObject = new JSONObject(json);
// Save the JSONOvject
ObjectOutput out = new ObjectOutputStream(new FileOutputStream(new File(getCacheDir(),"")+"cacheFile.srl"));
out.writeObject( jsonObject );
out.close();
Once you have the JSONObject serialized and save to disk, you can load it back in any time using:
// Load in an object
ObjectInputStream in = new ObjectInputStream(new FileInputStream(new File(new File(getCacheDir(),"")+"cacheFile.srl")));
JSONObject jsonObject = (JSONObject) in.readObject();
in.close();
Your best bet is probably GSON
It's simple, very fast, easy to serialize and deserialize between json objects and POJO, customizable, although generally it's not necessary and it is set to appear in the ADK soon. In the meantime you can just import it into your app. There are other libraries out there but this is almost certainly the best place to start for someone new to android and json processing and for that matter just about everyone else.
If you want to persist you data so you don't have to download it every time you need it, you can deserialize your json into a java object (using GSON) and use ORMLite to simply push your objects into a sqlite database. Alternatively you can save your json objects to a file (perhaps in the cache directory)and then use GSON as the ORM.
This is pretty straightforward example using a listview to display the data. I use very similar code to display data but I have a custom adapter. If you are just using text and data it would work fine. If you want something more robust you can use lazy loader/image manager for images.
Since an http request is time consuming, using an async task will be the best idea. Otherwise the main thread may throw errors. The class shown below can do the download asynchronously
private class jsonLoad extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... urls) {
String response = "";
for (String url : urls) {
DefaultHttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
try {
HttpResponse execute = client.execute(httpGet);
InputStream content = execute.getEntity().getContent();
BufferedReader buffer = new BufferedReader(new InputStreamReader(content));
String s = "";
while ((s = buffer.readLine()) != null) {
response += s;
}
} catch (Exception e) {
e.printStackTrace();
}
}
return response;
}
#Override
protected void onPostExecute(String result) {
// Instantiate a JSON object from the request response
try {
JSONObject jsonObject = new JSONObject(result);
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
File file = new File(getApplicationContext().getFilesDir(),"nowList.cache");
try {
file.createNewFile();
FileOutputStream writer = openFileOutput(file.getName(), Context.MODE_PRIVATE);
writer.write(result);
writer.flush();
writer.close();
}
catch (IOException e) { e.printStackTrace(); return false; }
}
}
Unlike the other answer, here the downloaded json string itself is saved in file. So Serialization is not necessary
Now loading the json from url can be done by calling
jsonLoad jtask=new jsonLoad ();
jtask.doInBackground("http:www.json.com/urJsonFile.json");
this will save the contents to the file.
To open the saved json string
File file = new File(getApplicationContext().getFilesDir(),"nowList.cache");
StringBuilder text = new StringBuilder();
try {
BufferedReader br = new BufferedReader(new FileReader(file));
String line;
while ((line = br.readLine()) != null) {
text.append(line);
text.append('\n');
}
br.close();
}
catch (IOException e) {
//print log
}
JSONObject jsonObject = new JSONObject(text);
Related
I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.
Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?
I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.
I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.
Im doing a simple http get,
I see on my result an incomplete response,
what Im doing wrong?
here the code:
class GetDocuments extends AsyncTask<URL, Void, Void> {
#Override
protected Void doInBackground(URL... urls) {
Log.d("mensa", "bajando");
//place proper url
connect(urls);
return null;
}
public static void connect(URL[] urls)
{
HttpClient httpclient = new DefaultHttpClient();
// Prepare a request object
HttpGet httpget = new HttpGet("http://tiks.document.dev.chocolatecoded.com.au/documents/api/get?type=tree");
// Execute the request
HttpResponse response;
try {
response = httpclient.execute(httpget);
// Examine the response status
Log.d("mensa",response.getStatusLine().toString());
// Get hold of the response entity
HttpEntity entity = response.getEntity();
// If the response does not enclose an entity, there is no need
// to worry about connection release
if (entity != null) {
// A Simple JSON Response Read
InputStream instream = entity.getContent();
String result= convertStreamToString(instream);
// now you have the string representation of the HTML request
Log.d("mensa", "estratagema :: "+result);
JSONObject jObject = new JSONObject(result);
Log.d("mensa", "resposta jObject::"+jObject);
Log.d("mensa", "alive 1");
JSONArray contacts = null;
contacts = jObject.getJSONArray("success");
Log.d("mensa", "resposta jObject::"+contacts);
Log.d("mensa", "alive");
//instream.close();
}
} catch (Exception e) {}
}
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the BufferedReader.readLine()
* method. We iterate until the BufferedReader return null which means
* there's no more data to read. Each line will appended to a StringBuilder
* and returned as String.
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
Log.d("mensa", "linea ::"+line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
}
i call it with:
GetDocuments get = new GetDocuments();
URL url = null;
try {
url = new URL("ftp://mirror.csclub.uwaterloo.ca/index.html");
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//URL url = new URL("http://www.google.es");
get.execute(url);
edit 1
I refer to incomplete as the response that gets truncated?
please notice in below image of response how string gets truncated,
is this because of the log size?,
but the other problem is that it doesn't parse?
thanks!
I don't know if this is going to resolve your problem but you can get rid of your method and use simply:
String responseString = EntityUtils.toString(response.getEntity());
I've had exactly the same issue for the last couple of days. I found that my code worked over WiFi but not 3G. In other words I eliminated all the usual threading candidates. I also found that when I ran the code in the debugger and just waited for (say) 10 seconds after client.execute(...) it worked.
My guess is that
response = httpclient.execute(httpget);
is an asynchronous call in itself and when it's slow returns a partial result... hence JSON deserialization goes wrong.
Instead I tried this version of execute with a callback...
try {
BasicResponseHandler responseHandler = new BasicResponseHandler();
String json = httpclient.execute(httpget, responseHandler);
} finally {
httpclient.close();
}
And suddenly it all works. If you don't want a string, or want your own code then have a look at the ResponseHandler interface. Hope that helps.
I have confirmed that this is because size limit of java string. I have checked this by adding the string "abcd" with the ressponse and printed the response string in logcat. But the result is the truncated respose without added string "abcd".
That is
try {
BasicResponseHandler responseHandler = new BasicResponseHandler();
String json = httpclient.execute(httpget, responseHandler);
json= json+"abcd";
Log.d("Json ResponseString", json);
} finally {
httpclient.close();
}
So I put an arrayString to collect the response. To make array, I splitted My json format response by using "}"
The code is given below(This is a work around only)
BasicResponseHandler responseHandler = new BasicResponseHandler();
String[] array=client.execute(request, responseHandler).split("}");
Then you can parse each objects in to a json object and json array with your custom classes.
If you get any other good method to store response, pls share because i am creating custom method for every different json responses );.
Thank you
Arshad
Hi Now I am using Gson library to handle the responses.
http://www.javacodegeeks.com/2011/01/android-json-parsing-gson-tutorial.html
Thanks
Arshad
I cant' comment directly due to reputation, but in response to https://stackoverflow.com/a/23247290/4830567 I felt I should point out that the size limit of a Java String is about 2GB (Integer.MAX_VALUE) so this wasn't the cause of the truncation here.
According to https://groups.google.com/d/msg/android-developers/g4YkmrFST6A/z8K3vSdgwEkJ it is logcat that has a size limit, which is why appending "abcd" and printing in logcat didn't work. The String itself would have had the appended characters. The previously linked discussion also mentioned that size limits with the HTTP protocol itself can occasionally be a factor, but that most servers and clients handle this constraint internally so as to not expose it to the user.
I have a servlet that has the following purpose:
Receive data via the URL (that is, using get). Then returns a message, based on this input, back to the caller. I am new to this stuff, but have come to learn that using json (actually, Gson) is suitable for this.
My question now is, how do I retrieve this json message? What URL do I target? The relevant lines in the servlet are:
String json = new Gson().toJson(thelist);
response.setContentType("application/json");
response.setCharacterEncoding("UTF-8");
response.getWriter().println(json);
This is how I try to retrieve the json:
try{
DefaultHttpClient defaultClient = new DefaultHttpClient();
HttpGet httpGetRequest = new HttpGet("http://AnIPno:8181/sample/response?first=5&second=92866");
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
String json = reader.readLine();
JSONObject jsonObject = new JSONObject(json);
} catch(Exception e){
e.printStackTrace();
}
But apparently this does not work, as I have found jsonObject has a size of 0 (it should be an array with three elements).
Previously, I had a write() instead of println() in the servlet. I'm not sure if that matters in this case. But I'm assuming I've misunderstood something about how the json object is retrieved. Is it not enough to point it towards the URL of the servlet?
Reading an InputStream whether from a File on the file system or from an HTTP request is, in most cases, the same.
What you have is correct only if your servlet wrote a single line. If the Gson object toString() method returns multiple lines, you're going to have to read multiple lines from the InputStream. I like to use the Scanner class for reading from an InputStream.
try {
DefaultHttpClient defaultClient = new DefaultHttpClient();
HttpGet httpGetRequest = new HttpGet("http://localhost:8080/cc/jsonyeah");
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
Scanner scanner = new Scanner(httpResponse.getEntity().getContent(), "UTF-8");
while(scanner.hasNextLine()) { // scanner looks ahead for an end-of-line
json += scanner.nextLine() + "\n"; // read the full line, you can append a \n
}
// do your serialization
} catch(Exception e) {
e.printStackTrace();
}
So we've done the same thing we would've done if we were reading from a file. Now the json object contains the json you received from the servlet, as a String.
For the serialization, you have a few options.
A Gson object has an overloaded method fromJson() that can take a String or a Reader, among other things.
From where we are with the code above, you can do
MyClass instance = new Gson().fromJson(json, MyClass.class);
where MyClass is the type you are trying to create. You will have to use a TypeToken for generic classes (such as a list). TypeToken is an abstract class, so generate an anonymous class and call getType()
Type type = new com.google.gson.reflect.TypeToken<List<String>>(){}.getType();
List<MyClass> list = new Gson().fromJson(json, type);
Another option is to use the overloaded method that takes a Reader directly instead of reading line by line from the InputStream:
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
MyClass instance = new Gson().fromJson(reader , MyClass.class);
You'll get to skip a step.
Don't forget to close your streams.
I have this function to readJsonData from a a request to a JSON String. You can use this function to retrieve the JSON, then use GSON to parse it to the object that you like. It works for my application. Hope it works for you too.
protected String readJson(HttpResponse resp)
throws IOException {
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(
resp.getEntity().getContent()));
StringBuffer buffer = new StringBuffer();
int read;
char[] chars = new char[1024];
while ((read = reader.read(chars)) != -1)
buffer.append(chars, 0, read);
} finally {
if (reader != null)
reader.close();
}
return buffer.toString();
}
So based on your code. I guess this should work:
String jsonData = readJson(httpResponse);
YourObject obj = new Gson().fromJson(jsonData, YourObject.class);
Before trying this, make sure your servlet prints out the JSON data that you want. I suggest using these Chrome Extensions: Postman - REST Client and JSON Formatter, to test your data from servlet. It's pretty helpful.
I want to write an Android application that can display some data received(polled) from an internet resource.
I guess that I need to write some logic that will periodically call and get data from some endpoint, parse the response and display it. Is there a good tutorial for all this steps?
I know very little about Android programming at the momment and maybe it is better to start with something simpler. I just want to know what to look for while learning an gather some resources on this.
What you want to do is developing a rest api that provides data for your android app. E.g. you website has some content that you want use in your app, then you could write a php script that just returns that data in a specific format.
E.g. mysite.net/rest/fetchAllLocations.php?maybe_some_parameters
This would return locations in e.g. json format, here is an example how that looks like:
[{"id":1,"shop_lng":8.5317153930664,"shop_lat":52.024803161621,"shop_zipcode":33602,"shop_city":"Bielefeld","shop_street":"Arndtstra\u00dfe","shop_snumber":3,"shop_name":"M\u00fcller","shop_desc":"Kaufhaus"}]
Here is an example for a rest api request:
http://shoqproject.supervisionbielefeld.de/public/gateway/gateway/get-shops-by-city/city/Bielefeld
So when you have your rest api set up you can deal with receiving that data with your android phone. I use a static method to get this data:
public class JsonGrabber{
public static JSONArray receiveData(){
String url = "your url";
String result = "";
DefaultHttpClient client = new DefaultHttpClient();
HttpGet method = new HttpGet(url);
HttpResponse res = null;
try {
res = client.execute(method);
} catch (ClientProtocolException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
}
try{
InputStream is = res.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
result = sb.toString();
}catch(Exception e){
Log.e("log_tag", "Error converting result "+e.toString());
}
JSONArray jArray = null;
try{
jArray = new JSONArray(result);
}catch(JSONException e){
Log.e("log_tag", "Error parsing data "+e.toString());
}
return jArray;
}
}
Well thats all, once you have your data in json format you just have to parse it:
JSONArray test = (JSONArray) JsonGrabber.receiveData()
try {
for(int i=0;i<test.length();i++){
JSONObject json_data = test.getJSONObject(i);
int id = json_data.getInt("id");
}
}
The web request should run in another thread, because it can be a time consuming process. So you need to deal with AsyncTask. Here are some resources:
Painless Threading
Multithreading for performance
Hello Android Tutorial
i want to develop an Android application that will take the content from internet (server) and present it in the application.
(ex. i take the todays weather forecast, put the numbers in SQLite database or .txt file , put the database/txt file on internet server so when i open the application, the app connects&downloads the database via the net and presents me with todays forecast)
If you can references me to some example/video tutorial/book that deals with this issue i will be very thankful!
What you want to do is developing a rest api that provides data for your android app. E.g. you website has some content that you want use in your app, then you could write a php script that just returns that data in a specific format.
E.g. mysite.net/rest/fetchAllLocations.php?maybe_some_parameters
This would return locations in e.g. json format, here is an example how that looks like:
[{"id":1,"shop_lng":8.5317153930664,"shop_lat":52.024803161621,"shop_zipcode":33602,"shop_city":"Bielefeld","shop_street":"Arndtstra\u00dfe","shop_snumber":3,"shop_name":"M\u00fcller","shop_desc":"Kaufhaus"}]
Here is an example for a rest api request:
http://shoqproject.supervisionbielefeld.de/public/gateway/gateway/get-shops-by-city/city/Bielefeld
So when you have your rest api set up you can deal with receiving that data with your android phone. I use a static method to get this data:
public class JsonGrabber{
public static JSONArray receiveData(){
String url = "your url";
String result = "";
DefaultHttpClient client = new DefaultHttpClient();
HttpGet method = new HttpGet(url);
HttpResponse res = null;
try {
res = client.execute(method);
} catch (ClientProtocolException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
}
try{
InputStream is = res.getEntity().getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
result = sb.toString();
}catch(Exception e){
Log.e("log_tag", "Error converting result "+e.toString());
}
JSONArray jArray = null;
try{
jArray = new JSONArray(result);
}catch(JSONException e){
Log.e("log_tag", "Error parsing data "+e.toString());
}
return jArray;
}
}
Well thats all, once you have your data in json format you just have to parse it:
JSONArray test = (JSONArray) JsonGrabber.receiveData()
try {
for(int i=0;i<test.length();i++){
JSONObject json_data = test.getJSONObject(i);
int id = json_data.getInt("id");
}
}
The web request should run in another thread, because it can be a time consuming process. So you need to deal with AsyncTask. Here are some resources:
Painless Threading
Multithreading for performance
Hello Android Tutorial