Develop Reference

Give me a suggestion to improve my app performance, on retrieving huge data(nearly more than 10,000 records) from server?

On performance view, JSON parsing take huge time for retrieving Data.In my app i need to get nearly 10,000 records from Server.On emulator,it gets data immediately and works efficiently.But in my android phone,it takes more than 2 minutes to retrieve all data.Kindly,give me a suggestion for improve the performance on phone.

The emulator has access to your host machine's resources and is therefore not a good way to test performance.
I have used the Jackson streaming JSON parser with large data sets and it works well for me. However, I run this process in the background and am able to accept long fetch/parse times. Depending on the size of the data and the speed of the device you're running on, 2 minutes does not seem extraordinarily long to me.
Maybe you could fetch a smaller subset of the data first, and then display it while you fetch the rest in the background. You're probably going to have to do some kind of optimization like this in order to improve performance.

I think you can parse the complex JSON response using GSON. Please check these tutorial http://www.javacodegeeks.com/2011/01/android-json-parsing-gson-tutorial.html
You just create the model classes and use the proper annotations then the data will be parsed to model objects directly.

The question is, what causes this slowdown. Because of everything goes in the rmulator like charm, it is probably the network speed. You can help this if you find a solution to compress the json data.
It is a text, with a lot of repeat, it is very, very good compressable. And http supports compression.
You need to set it in your http server.
If you find this a promising direction, I suggest to make a new question, giving your http server version. Good luck!

Handling large XML files in Android

I'm new to Android development and want to create a Google map app. I have to use 2 large XML files with geo data (latlons), 1 is 2mb and the other 8 mb big.
What is the fastest way to access and parse this data? Is it maybe beter to store this into a database? A webservice is not a good idea I think, because everytime the app will start I have to load 10 mb... And that will process very slow I guess...
Has anybody some good advice?
Thanks!

Keep the whole data in the server and access it from device using webservices. Better to go for SAX parser as the data is too large. If possible better to implement search functionality from server end or to implement pagination from server end.

Maybe you can create some kind of meta data to go along with the large xml file. This meta data can point to specific parts of the large xml file. And each small part can be a dom object instead of a sax parser.
I suggest using sax, creating a summary of the data you need in some kind of hashtable (simple key value pairs) and using it. May be using SharedPreferences for data that does not change.
Also you need to think on how / what data will be used and how frequently. May be you can keep a producer consumer architecture, where a thread prepares the data for use, and the consumer uses it when its ready on a notify or something.
You need to think from 3 angles
Not too much serial processing
Not redoing what you have already parsed
Parsing using the right quantum of data. Infact think of dom for small parts to make your design easier to start with.
And I dont recommend DOM for the entire tree. You need to use a combination of SAX and DOM.
First get it working, and then record response/performance delays and work your way up to improve it from the worst to best.

Is there an easy to use XML parser for remote database connection?

I am making an Android app, and I need to connect to a remote database with a webservice.
I will receive an XML file from the webservice with the results of my SELECT query (various rows in some cases).
I don't know much about XML or web services, I only know that I will receive an XML file and that I have to parse it to obtain the data.
Is there an XML parser for this purpose that is easy to add to my app and configure?

I would suggest not using XML and instead use JSON. JSON is much cleaner and much less in kilobytes. Then I would use Jackson to deserialize the JSON string to an object. You really don't need to do any work. Just point to the URL and you get an object back. You just have to make sure on the webservice you also generate correct JSON, which you can also use Jackson.
Here is how easy it is:
ObjectMapper mapper = new ObjectMapper(); // can reuse, share globally
User user = mapper.readValue(new File("user.json"), User.class); // can use File, URL, String!

Designing XML parsers are not very difficult. You can probably google for xml parsers and then change them to look for the tags you need or values.
Google is your friend.

You don't need an in depth knowledge about XML but you should know a bit about it. A recommended read would be this article. You should also at least know how the server is handling the communication. Is it a unidirectional communication (your application only fetching data) or is it a bidirectional communication (you are sending requests to the server). If the later how is the server handling them and so on. A bit of background knowledge is required.
I always recommend using the SAX parser as it seems so be the most efficient one due to its concept (being event based). A good read about dealing with XML files on Android can be found here. And also don't forget to search for similar questions here on StackOverflow as it is quite a popular question with some quality answers.

How to parse a complex SOAP response in Android

Since two days I am trying to consume a WCF (.NET) Soap Service and serialize it's response without success. I am getting a correct response (I had to put it on pastebin: SOAP Response Example), but KSOAP2 is not able to handle .NET Datasets correctly.
I already consultated various articles about this specific problem, but none has a .NET Dataset to handle. The main article which gave guideance was an
article by IBM "Working with XML on Android"
I tried following steps to parse my data without success:
Parsing with SAX (android) -> Seems not to work with this complex document because of the different namespaces.
with DOM Object(android) -> NullPointerExeption (dooh!)
with Digester (dom4j) -> NullPointerException (arggh!)
method suggested by helloandroid.com "Using ksoap2 for android, and parsing output data"
Some questions:
- The returned response is a normal XML, but actually it includes a .NET Dataset. Has anyone had success to parse data out of such a response?
- Is there a way to make KSOAP2 not to trying to parse the data? It returns a rubbish SoapObject, which is unreadable. I would like just the contents of the SOAP body. Is there a way to intercept that?
- Do you have any other hint?

With ksoap2 you can set the envelope debugging to true and then get the response dump, which will contain the full xml.
However what makes you think that the SoapObject returned is unreadable. Check it out in a debugger and you will find that everything is there and you can just parse it out using getProperty("propname") and getAttribute("attribute) which in turn are either SoapObjects again if they are nested or contain actual values if they are leaf nodes.
Check out some of the links on the wiki to http://code.google.com/p/ksoap2-android/

I have used DOM and SAX so far to parse XML documents. DOM had a few issues on Android and I had to handle some bugs in the API. SAX seems to be better (and leaner if you only read). I have not used ksoap, but did everything hand crafted. Though I am not sure where the problem with that .NET thing is, I wouldn't see an issue using SAX or DOM. Can you comment why you think SAX won't work because of namespaces?
A.

One of my solutions was: http://vtd-xml.sourceforge.net/
I hope someone will find something more suitable.

I have the same problem. First I used vtd-xml. It was working without problems but it is a bit slow. Now I switched to standard Java SAX (not Android SAX implementation) and it works ok.

Consuming a WCF REST service from Android is very slow

I have a WCF REST service built with C# and it returns an image as part of a CPU intensive operation. The client is running on Android (Java) By default, it will return a text JSON object that looks something like this:
{"d",[9,0,77,12,11,...]}
Those are they bytes of the image. Fine. However, all the solutions for decoding this JSON are intolerably slow. I've tried Gson, Jackson, and the built-in Android JSONObject class. I have no idea why they are so slow.
As an alternative solution, I have my REST service return a GUID, and then that GUID can be used by the Android client to go to a regular URL that serves up the image as a regular binary stream, via an MVC controller.
This works well, and it fast, and is pretty easy to handle on the Android side. However, it does feel like a bit of kludge and kind of a violation of the REST design principles.
Am I missing something here? Is there a better way to do this?

How about your REST service return a Redirect 303 with a Location header that has an URL that points to the image? Or why not just return the bytes directly from the first URL?
As far as RESTful or not, returning a JSON encoded image is not exactly in the spirit of the REST self-descriptive constraint.
Just make sure the endpoint that returns the image stream of bytes, actually uses an image/* media type in the content header.

Well, on of your main problems is trying to transmit binary data using a text format.
Most if not all java json libraries will try to recognize the type of the field. It'll take a long time if there's a lot of fields.
Yeah, streaming it directly is a lot faster. Maybe you can use XML since it supports binary or blob data.

As Darrel wrote above, if the URL computes and returns an Image, simply return that Image with an appropriate content-type, for e.g., as a PNG image. Transmitting the image encoded within JSON is a strange choice, to say the least.

There is a great talk about developing rest client application on android form Google IO 2010.
http://www.youtube.com/watch?v=xHXn3Kg2IQE
This session will present architectural considerations for developing RESTful applications on the Android platform. It focuses on design patterns, platform integration and performance issues specific to the Android platform.
A great resource and must watch.