Develop Reference

Questions about retrieving/refreshing data using Volley (and SimpleXML) on Android

All the data my Android app displays is taken from a XML file. I'm using Volley to retrieve the data and SimpleXML to map the XML data to objects using a Volley custom Request: SimpleXmlRequest.
Every time I refresh the content, it reads the whole XML file and creates every single object again and I have the feeling that this is not the best way to do it, cause this file can be quite large. So I have a few questions about that:
Is there a way to "subscribe" to the XML file to retrieve only the
new items added on it, avoiding to read the whole file every time? I know Volley can cache but doesn't help so much here.
Would it be possible to make a SimpleXmlArrayRequest using
SimpleXML framework? Would it help? Cause I really don't want to
parse the file by myself...
If not, would it worth to switch from XML to Json so I could use
JsonArrayRequest or would I have to read the whole file and refresh
every single item anyway? I've never used it before.
Many items from the source file are displayed as a list using
RecyclerView but, again, when I refresh, I have to remove the items
it had before and add them all again as I'm getting all the items at
once and consequently can't use
RecyclerView.notifyDataSetChanged().
I've been reading a lot, even until finding out the RxJava and RxAndroid which seems perfect but it looks like too hard to integrate Volley and SimpleXML on it... so I hope you can help me :)
Thanks!

I always work with Json apart from being very light, with libraries like GSON its fun to work with.
To get only new new data, my practice is to include two fields in the database tables, 1 for isDeleted (boolean) and 1 for lastModified (long epoc time).
The isDeleted field applies only if the data is accessed by multiple users.

Decode complex JSON string to various object(s)

I was just wondering if anyone could recommend a better alternative method than org.json for decoding a complex JSON string. For reference, this will be coming from a web server down to Android (& iOS, but that's the other dev's problem!) devices, it doesn't have to go back up.
The string is of the following nature...
{"header":"value","count":value, ..., "messages":[
{"messagetype":1,"name":"value"},
{"messagetype":2,"name":"value","name":value},
{"messagetype":1,"name":"value"},
{"messagetype":3,"name":"value","subvalues":["value",value,value]},
...
{"messagetype":4,"name":value,"name":"value","name":value}
]}
Basically, there are some header fields which I can always rely on but then there will be an "array" of messages, variable in count, content and order.
I've been researching this for a few days now and have dismissed GSON and a few others because that either need to know the exact structure in advance and/or don't deal well with the embedded types (the contained messages).
Answer three in this question pointed me to using the org.json library and I know I can use that to parse through the string but I guess one of that answer's replies ("That's super old school and nobody uses that library anymore in the real world") has made me question my approach.
Can anyone suggest a library/approach which would handle this problem better? If anyone else has used an alternative approach to dealing with this type of complex and variable structure, I'd really appreciate your input.
Thanks in advance.

I really do not agree with the opinion about org.json libray: "That's super old school and nobody uses that library anymore in the real world", since parsing json by using this library is pretty straightforward. Besides, how complex can json get?, I mean, is all about key/value pairs, nothing that can't be solved with a few lines of code, for instance I will illustrate you a few cases, so that you'll get convinced that is pretty simple to do:
Suppose you have a response from the server containing all info you need formatted in a json array, then you can do something like this to parse the String:
JsonArray arrayJson = new JsonArray(response);
But now you want to access arrayJson childs:
for (int i = 0; i < arrayJson.length() - 1; i++)
{
JsonObject json = arrayJson.getJSONObject(i);
}
And now assume you have another array of json's inside those you retrieved in the for loop:
Then you'll get them this way:
for (int i = 0; i < arrayJson.length() - 1; i++)
{
JsonObject json = arrayJson.getJSONObject(i);
JSONArray anotherArray = json.getJSONArray("key");
}
....., more nestings you can handle them the same way, so I think I established my point. Remember that sometimes, struggling on finding easier ways to do things, can get them even harder to do.

Android performance getJSONObject while parsing

I'm currently parsing JSON and got the following piece of code:
boolean suf = list.getJSONObject(i).getBoolean("sufficient");
String grade = list.getJSONObject(i).getString("grade");
String id= list.getJSONObject(i).getString("id");
I'm wondering if multiple times calling getJSONObject creates overhead resulting in increasing processing time.
Would this be faster and/or better for example?
JSONObject object = list.getJSONObject(i);
boolean suf = object.getBoolean("sufficient");
String grade = object).getString("grade");
String id= object.getString("id");
This does introduce a new object, but will the next 3 calls make the tradeoff worth it?
Since I'm showing a dialog to inform the user something is loading (and thus they can't undertake any action), I'd like to minimize the wait time for the user.

2nd option is how I usually do. But you will hardly see any notice in performance.
list.getJSONObject(i).getBoolean("sufficient"); creates a temporary object and gets the value. Now a days, compilers are smart enough to store that temporary objects just in case them. Even if they don't, unless you are handling some millions of jsonobjs in your "list", I don't see any performance impact here.

android: gson performance

I am trying to use gson to do my object mapping on the android emulator.
It has been ridiculously slow when processing json data around 208 kb. I do not have any hierarchies in my json.
After the object mapping is done, i can see it that gson created around 500 records.
It is taking it over 3 minutes on the android emulator to map the input json.
I have annotated my entity which comprises of strings and couple of floats.
An I missing something?
Any ideas, best practices would greatly help.
Are there any ways of quickly object mapping the json data?
URL myURL = new URL(url);
/* Open a connection to that URL. */
URLConnection ucon = myURL.openConnection();
/*
* Define InputStreams to read from the URLConnection.
*/
InputStream is = ucon.getInputStream();
InputStreamReader reader = new InputStreamReader(is);
long tickCount = System.currentTimeMillis();
Policy[] policies = new Gson().fromJson(reader, Policy[].class);
long endCount = System.currentTimeMillis() - tickCount;
Log.d("Time to pull policies in milliseconds", "" + endCount);

I've seen questions like this come up before, and the general consensus is that Jackson is much faster than Gson. See the following links for more information:
Jackson Vs. Gson
Replace standard Android JSON parser for better performance?
http://www.cowtowncoder.com/blog/archives/2009/12/entry_345.html
https://stackoverflow.com/questions/338586/a-better-java-json-library
Here is one which specifically discusses Android: http://ubikapps.net/?p=525

Have you tried the mixing the GSON streaming parser with the Gson object? http://sites.google.com/site/gson/streaming (look for the Mixed read example).
This approach may help since Gson reads in an entire parse tree and then acts on it. With a large array list, reading in all elements and attempting to parse may cause lot of memory swaps (or thrashing). This approach will read in one element at a time.
Hope this helps.

You'd probably get better performance if you wrapped that InputStream in a BufferedInputStream with a nice big buffer...
3 minutes is insane. I seldom run the emulator but I have an app with a ~1.1MB JSON asset and that takes around 5 seconds to load and process on hardware.
(Which is still far too long, but still).

I've found that I can speed up gson.fromJSON quite considerably by not modelling all the elements in the JSON that I won't need. GSON will happily fill in only what is specified in your response classes.

I have found that CREATING a Gson instance is a very expensive operation, both in terms of CPU used and memory allocated.
Since Gson instances are thread-safe, constructing and reusing a single static instance pays off, especially if you are serializing / deserializing often.

Parse JSON into a ListView friendly output

So I have this JSON, which then my activity retrieves to a string:
{"popular":
{"authors_last_month": [
{
"url":"http://activeden.net/user/OXYLUS",
"item":"OXYLUS",
"sales":"1148",
"image":"http://s3.envato.com/files/15599.jpg"
},
{
"url":"http://activeden.net/user/digitalscience",
"item":"digitalscience",
"sales":"681",
"image":"http://s3.envato.com/files/232005.jpg"
}
{
...
}
],
"items_last_week": [
{
"cost":"4.00",
"thumbnail":"http://s3.envato.com/files/227943.jpg",
"url":"http://activeden.net/item/christmas-decoration-balls/75682",
"sales":"43",
"item":"Christmas Decoration Balls",
"rating":"3",
"id":"75682"
},
{
"cost":"30.00",
"thumbnail":"http://s3.envato.com/files/226221.jpg",
"url":"http://activeden.net/item/xml-flip-book-as3/63869",
"sales":"27",
"item":"XML Flip Book / AS3",
"rating":"5",
"id":"63869"
},
{
...
}],
"items_last_three_months": [
{
"cost":"5.00",
"thumbnail":"http://s3.envato.com/files/195638.jpg",
"url":"http://activeden.net/item/image-logo-shiner-effect/55085",
"sales":"641",
"item":"image logo shiner effect",
"rating":"5",
"id":"55085"
},
{
"cost":"15.00",
"thumbnail":"http://s3.envato.com/files/180749.png",
"url":"http://activeden.net/item/banner-rotator-with-auto-delay-time/22243",
"sales":"533",
"item":"BANNER ROTATOR with Auto Delay Time",
"rating":"5",
"id":"22243"},
{
...
}]
}
}
It can be accessed here as well, although it because it's quite a long string, I've trimmed the above down to display what is needed.
Basically, I want to be able to access the items from "items_last_week" and create a list of them - originally my plan was to have the 'thumbnail' on the left with the 'item' next to it, but from playing around with the SDK today it appears too difficult or impossible to achieve this, so I would be more than happy with just having the 'item' data from 'items_last_week' in the list.
Coming from php I'm struggling to use any of the JSON libraries which are available to Java, as it appears to be much more than a line of code which I will need to deserialize (I think that's the right word) the JSON, and they all appear to require some form of additional class, apart from the JSONArray/JSONObject script I have which doesn't like the fact that items_last_week is nested (again, I think that's the JSON terminology) and takes an awful long time to run on the Android emulator.
So, in effect, I need a (preferably simple) way to pass the items_last_week data to a ListView. I understand I will need a custom adapter which I can probably get my head around but I cannot understand, no matter how much of the day I've just spent trying to figure it out, how to access certain parts of a JSON string..

originally my plan was to have the
'thumbnail' on the left with the
'item' next to it, but from playing
around with the SDK today it appears
too difficult or impossible to achieve
this
It is far from impossible, but it will be tedious to get right, unless you use something that already wraps up that pattern for you (and that hopefully is reasonably "right"). On the Web, performance/bandwidth issues were the user's problem -- in mobile, they're your problem.
as it appears to be much more than a
line of code which I will need to
deserialize (I think that's the right
word) the JSON
new JSONObject(data) is one line of code. Now, fetching the JSON, which I presume you are doing from the aforementioned URL, will be several lines of code. Neither the parsing of the JSON nor the fetching of it off the Internet is unique to Android -- all of that would look the same on a desktop Java app, or a Java servlet, or whatever.
apart from the JSONArray/JSONObject
script I have which doesn't like the
fact that items_last_week is nested
I have not had a problem parsing JSON with structures like your file exhibits. Moreover, this is hardly unique to Android -- the JSON parser is used in many other Java-based projects.
and takes an awful long time to run on
the Android emulator
The speed of the emulator is tied to the speed of your development machine. For me, the emulator is usually slower than actual phone hardware...and my desktop is a quad-core. Bear in mind that the emulator is pretending to be an ARM chipset running on your PC, converting ARM opcodes into x86 opcodes on the fly, so it's not going to be fast and won't leverage multiple cores very well.
So, in effect, I need a (preferably
simple) way to pass the
items_last_week data to a ListView.
There is nothing really built into Android to take an arbitrary JSON structure, with arbitrary data, and directly pour it into a ListView. This is not unique to JSON -- XML would exhibit similar phenomenon.
Your choices are:
Create a custom ListAdapter that wraps the parsed JSON.
Convert the parsed JSON into a MatrixCursor (think 2D array of data) and use a SimpleCursorAdapter.
Convert the parsed JSON into an ArrayList<String> and use an ArrayAdapter.
For the short term, option #3 is probably the simplest.
I understand I will need a custom
adapter which I can probably get my
head around but I cannot understand,
no matter how much of the day I've
just spent trying to figure it out,
how to access certain parts of a JSON
string..
And that question is too vague for much in the way of assistance. You might consider opening up a separate question, tagged for Java and JSON, where you get into the details of where you are having problems with the json.org parser.