has anybody used delta compression for json data in android - android

I have been trying to reduce the json data footprint that we are using in our android native application the file sizes for which are generally huge.
This has led me to believe that to achieve optimisations for json the following can be done
1) json minimization
2) compression algorithm like gzip or lzw (not sure which one will be better)
3) horizontal delta compression wherein we send only the difference/delta between two subsequent json files that greatly reduces the amount of data we send over the wire.
Please share your views if you guys have done or experimented with any of these techniques for json compression. Also we use node.js as the backend for the server side apis.

As for delta compression, you may want to look at http://jsonpatch.com
JSON Patch is a format for describing changes to a JSON document. It can be used to avoid sending a whole document when only a part has changed. When used in combination with the HTTP PATCH method it allows partial updates for HTTP APIs in a standards compliant way.
As you're using Node, you can take a look at JSON-Patch which will not only apply patches to a document/object but also has the ability to observe an object and generate a patch for you once it's been modified.

Related

Is a good idea for performance issue use ASN.1 protocol (or other bit conversion) than json+gzip?

Suppose you have a common communication protocol between your app and webservice. This protocol never changes and has a high demand.
I was thinking, to get a better performance use ASN.1(or other in-house bit code protocol) protocolo to code all information inside Android before send to webservice.
Today, I use json encode, and after I gzip all information before send to the webservice. I have the cost of zipping and unzipping the data, more json_decode and encode for both sides.
I know that to code to ASN.1 I will have bit operations to code and decode, but generally they are really fast. And other point that I am planning to NOT zip this information, once I am using bit coding the size will be reduced naturally.
An important observation is that I want to get a better perfomance to the server side. The client performance issue is not a problem here.

what is the purpose of using JSON in android?

I need to know the purpose of using JSON in android ?
Please anyone tell me in a simple way...
Thanks
The same reason you'd use it on any platform. JSON is a way of storing and expressing information. It uses attribute-value pairs in a hierarchical structure. In Android specifically, you may need to download some information from a database, which could be stored in JSON and then read by your app. Alternatively, you could store data locally in JSON but there are probably better and more efficient ways to do that if you're not sending data across a network.
https://en.wikipedia.org/wiki/JSON
JSON is very light weight, structured, easy to parse and much human readable. JSON is best alternative to XML when your android app needs to interchange data with your server
For example, you can get data Json if you work with database. Or if you work with some API's then you can get data in format Json.
For example an app could fetch data from a server. When using JSON to get the data, the traffic is quite small and the app can easily work with it.
For example you have a server with a database with recipes, and your app displays recipes, the app could ask the server for recipes, and it gets a JSON in return. for example:
{
name: 'Cookies'
ingredients: { 'Butter', 'Eggs', ... /* I don't know, I'm not a chef :D */
...
}
The app can then just read the properties and display them in a neat list ;)
JSON (JavaScript Object Notation) is a lightweight format that is used for data interchanging. It is also a subset of JavaScript's Object Notation (the way objects are built in JavaScript
Pls go through this link: http://www.copterlabs.com/blog/json-what-it-is-how-it-works-how-to-use-it/
JSON stands for JavaScript Object Notation
JSON is lightweight text-data interchange format
JSON is language independent *
JSON is "self-describing" and easy to understand
* JSON uses JavaScript syntax for describing data objects, but JSON is still language and platform independent. JSON parsers and JSON libraries exists for many different programming languages.
Using JSON in Android is not different than using it on any other platform. The main advantage of the format (in comparison to XML for example) is the small size of the data. This is very important for mobile devices due to the scarce resource those application use - i.e. your mobile app should be able to run with little memory usage, slow internet connection and so on.
Besides Android's framework has built-in tools for parsing / creating JSON objects. Thus it is both easy and efficient to use JSON rather than XML. If you have any project specific reason to prefer another data presentation format - don't worry. It is perfectly fine NOT to use JSON as long as some other format is more suitable for your project.
In short JSON is usually the right choice due to its small footprint and easy of use.

Is it better to add accept encoding gzip for HTTP request headers in android?

Just wanted to know if it is better to have "Accept-Encoding", "gzip" in request headers while making HTTP request in android?
If not then in which cases it is not preferable? Also can it be used for requests returning XML or JSON as response?
Thanks in advance!
The gzip encoding is used to cut down the size of the data being transmitted over the netwrok. It compresses it into a zip and gives you a smaller compressed version which in turn reduces bandwidth usage and makes for faster programs. So i would say there is no harm in adding the "gzip" tag as it would definitely speed up the get process in most cases. A good site you should probably look at:
Is this
And yes. It can be used with xml.
Some of the issues are
Most of the images are already highly compressed and cant be
compressed further.
Some of the older versions have problems with compression but this
should not be an issue for you.

best method for xml data storage

I am a php/mysql developer learning android. I am creating an android app that receives info from my php app to create list views of different products which will open a web view of that product's detail.
Currently my php cms web application outputs xml lists for an iphone app.... (also, separately outputs html). I have full control of the php app so if there is a better way to output the data for the android app please let me know.
I have created code that reads the xml from the web and creates the list view. The list can be refreshed daily, so the data does not need to be read from the online xml every time the app starts.
So I was thinking to store the data retrieved locally to improve my apps responsiveness. there may be up to 500 product descriptions to be stored at any given time in up to 30 different xml lists. I am starting development with one xml list with about 30 products.
For best performance should i store the product info in a sqlLite db or should i store the actual xml file in the cache/db or some other method like application cache.
I also was think to create the update of the data as a service, would this be a good idea?
The most efficient way to store data is RAM. But if you want to cache it, then the most efficient way is Database.
I recommend you store your data in sqlite android database.
You could also consider zipping you xml for faster network transfer and unzipping through java.util.zip package classes. You could even consider a simpler format for transmitting data, less verbose than xml, using a datainput/outputstream.
(I do that in of my apps and it works great)
Here are some details on data input / output stream method :
imagine a proprietary protocol for your data, only what you need. No tags, no attributes, just raw values in order.
on the client side, get an input stream on your data using URL.getContent() and cast it in input stream.
on the client side still, build a data input stream encapsulating your socket input stream and read data in order. Use readInt, readDouble, readUTF, and so on.
on the client side, from php, you need to find a way to save your data in a format that is compatible with the data format expected by the client. I can't tell much about PHP, I only program using java.
The advantage of this technique is that you save bandwith as there is only data and no verbose decoration due to xml. You should read about java specs to understand how double, int, strings are written in data output stream. But it can be hard using two languages to get the data right.
If php can't save format in a suitable way, use xml, it will be much simpler. First try with just plain xml, then give a try using a zip or tarball or xml file.
But all this is about speed gain during network connection.
The second part of what you have to do is to store each row of your list in a SQL table. Then you can retrieve it pretty fast using a CursorAdapter for your list view (it breaks the charming MVC model but it is quite fast !).
Sorry about this, but it became too long to write as a comment. This is not intended to be an answer to your question, because in my opinion Stéphane answered very well. The best solution is indeed to store the data in an sqlite database. Then you need to create the class to be used as a connection between the data, the database and the app. I don't want to take credit for what is said here already (I, too, voted it up).
I'm concerned with the other suggestion (use of low level raw streams for data manipulation, the list steps on that answer). I strongly recommend you to avoid creating your own proprietary protocol. It goes like this:
I need to exchange data.
I don't want to deal with the hassle of integrating external APIs into my code.
I know I can write two 5 minute routines to read and write the data back and forth.
Therefore, I just created my own proprietary format for exchanging data!
It makes me cry whenever I need to deal with unknown, obscure and arbitrary sequence of data blobs. It's always good to remember why we should not use unknown formats:
Reinventing the wheel is counter-productive. It seems not, but on the middle term it is. You can adapt your project to other mediums (server-side, other platforms) easily.
Using off-the-shelf components help you scale your code later.
Whenever you need to adapt your solution to other technologies and mediums, you'll work faster. Otherwise, you would probably end up with ad hoc code solutions that are not (easily) extensible and interoperable.
Using off the shelf components enables you to leverage advances in that particular technology. That's particularly important when you are using Android APIs, as they are frequently optimized for performance later down the road (See Android's Designing for Performance). Rolling your own standards may result in a performance penalty.
Unless you document your protocol, it's extremely easy to forget the protocol you created yourself. Just give it enough time and it will happen: you'll need to relearn/remember. If you document, then you are just wasting the computational time of your brain.
You think you don't need to scale your work, but chances are you will most of the time.
When you do, you will wish you had learned how to easily and seamlessly integrate well known formats.
The learning curve is needed anyway. In my experience, when you learn, you actually integrate well known formats faster than imagining your own way of doing things.
Finally, trust your data to geniuses that take their lives into creating cohesive and intelligent standards. They know it better!
Finally, if the purpose is to avoid the typical verbosity of XML, for whatever reasons, you have several options. Right now I can think of CSV, but I'm no expert in data storage, so if you're not confortable with it, I'm sure you can find good alternatives with plenty of ready to use APIs.
Good luck!

Consuming a WCF REST service from Android is very slow

I have a WCF REST service built with C# and it returns an image as part of a CPU intensive operation. The client is running on Android (Java) By default, it will return a text JSON object that looks something like this:
{"d",[9,0,77,12,11,...]}
Those are they bytes of the image. Fine. However, all the solutions for decoding this JSON are intolerably slow. I've tried Gson, Jackson, and the built-in Android JSONObject class. I have no idea why they are so slow.
As an alternative solution, I have my REST service return a GUID, and then that GUID can be used by the Android client to go to a regular URL that serves up the image as a regular binary stream, via an MVC controller.
This works well, and it fast, and is pretty easy to handle on the Android side. However, it does feel like a bit of kludge and kind of a violation of the REST design principles.
Am I missing something here? Is there a better way to do this?
How about your REST service return a Redirect 303 with a Location header that has an URL that points to the image? Or why not just return the bytes directly from the first URL?
As far as RESTful or not, returning a JSON encoded image is not exactly in the spirit of the REST self-descriptive constraint.
Just make sure the endpoint that returns the image stream of bytes, actually uses an image/* media type in the content header.
Well, on of your main problems is trying to transmit binary data using a text format.
Most if not all java json libraries will try to recognize the type of the field. It'll take a long time if there's a lot of fields.
Yeah, streaming it directly is a lot faster. Maybe you can use XML since it supports binary or blob data.
As Darrel wrote above, if the URL computes and returns an Image, simply return that Image with an appropriate content-type, for e.g., as a PNG image. Transmitting the image encoded within JSON is a strange choice, to say the least.
There is a great talk about developing rest client application on android form Google IO 2010.
http://www.youtube.com/watch?v=xHXn3Kg2IQE
This session will present architectural considerations for developing RESTful applications on the Android platform. It focuses on design patterns, platform integration and performance issues specific to the Android platform.
A great resource and must watch.

Categories

Resources