In my android application, user need to load about >100 rows populated in a Listview. To update new data everyday without updating app, I store my data in a XML file, put it in server and when user open app, my app load new XML file from server and parse them into my Listview.
Does my practice is a good method? How about it performance compare to JSON? And when I need to use MySQL server?
Your approach is fine but JSON would be a bit better because it is more compact. Refer this answer: JSON and XML comparison
To cache your data use SQLite database, it will be much faster to read data from database on your device than making a network request.
This is a static approach. You must edit the XML yourself every time you want to change the result.
A dynamic approach would make you create a database to store the data (MySQL or any other) and you would write a script that communicates with the database and query for data and then you need there to choose between the formats XML or Json. I think Json is faster and more readable than XML and JSON is also more compact.
You can use Protocol Buffers
According to the site:
Protocol buffers are Google's language-neutral, platform-neutral,
extensible mechanism for serializing structured data – think XML, but
smaller, faster, and simpler. You define how you want your data to be
structured once, then you can use special generated source code to
easily write and read your structured data to and from a variety of
data streams and using a variety of languages.
Benefits of Protocol Buffers:
Protocol buffers have many advantages over XML for serializing
structured data. Protocol buffers:
are simpler
are 3 to 10 times smaller
are 20 to 100 times faster
are less ambiguous
generate data access classes that are easier to use programmatically
Related
I am making an Android App for marking Attendance of students. I have an SQLite database in the app and a MySql remote database. The app sends and receives data between the two databases and i am thinking of using JSON for parsing data. Is there any other method for parsing data between the databases other than JSON that is more efficient and good. Thanks! Cheers!
There are a plethera of other data choices other than JSON.
JSON has the advantage of being easy to read, which makes debugging the wire protocol easier. It can be kind of verbose though, and there are more compact encodings which will save bandwidth.
I personally really like Avro. However, other good choices are Thrift and Protocol Buffers. All three offer a way to convert POJOs into a compact binary form. Thrift is basically a redo of protocol buffers by guys that were hired away from Google by Facebook. Avro takes a slightly different approach in that it has Schema resolution, which can make some schema evolution tasks easier to accomplish.
We developing an app that will store data locally on phone, on server and share it between users. The access to store and retrieve data will be approximately 20 times a day for each user, it will consist mostly of strings. The app will be cross platform Android/iOS. So before beginning developing DB what you suggest Json vs SQlite or use HTTP Methods: GET, POST.
What are advantages and disadvantages of each, and what do you suggest.
I do not agree that sql is slow. It's not like these devices are using spinning disks. In my app, i've implemented a RESTful API that passes JSON data back. This is easy because on iOS, json can be easily converted to the iOS Native NSDictionary object.
I recommend storing data using sqlite or Core Data because you'll be happy for it later on if you need to query the data. That's what databases are used for, storing and retrieving data easily.
As an alternative, you can also consider building the SQLite database on the server, and actually downloading that file to the devices. This has it's benefits because you won't need to do the actual sqlite processing on device.
I recommend staying away from XML, not because it's bad, but because passing JSON data back has become standard, and a lot of large companies are moving away from XML formats because of the processing power required.
In designing your RESTful API, it's always good practice to use the already available HTTP request methods. Use GET to retrieve data, POST to create new data, PUT to update data and DELETE to remove data. Once you have a beautifully designed REST api, all things things should just fall into place and you'll have an app that's maintainable and scalable on the client and on the server side.
This question really depends on your own area of expertise. If you have good understanding of JSON, then use that. XML is another great way via SOAP (or REST) to distribute data to your platforms, Server -> Device -> Server. My own experience is that JSON with GZIP is small and fast and there exists lots of parsers that is fast. XML has the downside that it gets big quickly but is often easier to maintain over time since the syntax is set in the "namespace" of your XML. SQLite is slow, since its disk based and should only be used for persistent storage of data. Access it as little as possible during runtime. My suggestion: JSON+GZIP, REST-api on server/device, store as much as possible in memory on devices, SQLite for persistent storage on devices and lastly MySQL on server for persistent storage (free, fast enough if done correctly with views and methods).
But as said, this is all up to the implementation.
I am looking to use an XML file to store the data my Android app generates. With that in mind, I have two questions for the community:
Is XML the best way to store data on Android and most efficient in cases where data may be added or altered every second or less then a second.
If XML is indeed the best for the scenario described in #1, how do I go about setting it up?
1.) Is XML the best way to database data on android and most efficient in cases where data may be added or altered every second or less then a second.
Definitely not.
2.) If XML is indeed the best for the scenario described in #1, how do I go about setting it up?
If you plan to store data just locally, the best way would be SQLite which works as a local database on every device.
If you later plan to synchronize this data with a central database, you may do this asynchronously within an AsyncTask or a Thread which would run periodically, but writing each second into a XML file is a bad idea as far as performance goes.
It's probably also a bad idea synchronizing a remote database at each insert/modification/deletion operation as if you had many users you could collapse the remote database.
I think the best approach is (as previously said) having a local database where you would store that data, and implement a webservice in the remote side if needed and use it to periodically synchronize both databases.
I would use JSON over XML and I would highly consider using GSON from Google. You maybe want to consider writing directly to a database with it's own structure and use transactions and sets. Is there are reason you want to go through JSON/XML?
XML is one of the worst ideas to keep local data in Android.
Most common used is SQLite available on the Android platform, but it all depends on what data and how you want to use.
In many mobile applications you don't need the relational database for one of the following reasons:
You have no relational data (i.e. settings) => no point in making relational tables with 1 record each
You have small, and dynamically changed data (like cache for downloaded content)
You don't need to search for data (using indexes etc.)
What alternatives can be used?
Shared preferences - simple key/value storage of primitive objects
Data serialization - for your consideration - binary (native java), JSON, parcelable (can be combined with the shared preferences)
For most of my app I'm currently using the binary serialization for "local storage".
- It's fast enough (usually much faster than starting the local SQLite engine)
- It's extremely easy and quick to implement, especially when you are using it for json/xml downloaded data parsed to POJO objects. All you need to do is just put "extends serializable" and put few lines of code to serialize/deserialize whole structure
- You can use those same classes for keeping data locally and communication with backend
Of course - it all depends from the situation - if you want to keep locally log of data from some sensor, or allow others apps to use this data, have to quick filter 1k+ records, or you really like to write few hundreds lines of code SQLite will be the best option for you. But most of mobile applications has no clear reason to use the relational (and trust me - not perfect one) engine.
I have a few years of exp with webdev using .Net and C#... and one feature I really enjoy there is the linq-expressions for quering for data.. is there anything that simulair for Android using to query against JSONs or against the SqlLite database? (I guess im looking for a typed expression/query-framework).
And for my next question.. Which one of the JSON approach and the SQLLite approach am I supposed to use.. and when? I know this has been asked a million times.. but there doesnt seem to be any strict answers..
Is SQLLite for more complex querying and when Im having a bigger ammount of data, while JSON is used while having a smaller ammount of data and not very much querying is needed?
Could I store all my JSON data in the memmory for faster read/write access?..since I have heard that storing it in the storage/on disk might be a slow process.
But on the other hand I also have heard that SQLLite is slow in generall...
And finally.. when we are speaking of "slow" is it slow like, a single read/write looks up the entire application for a few milisecs or is it slow like compared to a in-memory-database that executes queries and stuff fast as lightning
When to use SQLite
SQLite is one form of how to persist your data, how to secure your data. Generally if you want to store larger quantum of data and have quick access to them(and also if these data are "sensitive"), SQLite is great choice. Hence i disagree with your opition that SQLite is slow, definitely not i guess.
I have SQLite database with approximately with 650 000 records and still smart performance(an usage of indexes is sometimes necessary).
When to use JSON
JSON is lightweight data structure designed for human-readable data interchange and what is main is language-independent. It's very good choice(maybe the best) if you want send data via network(send data to remote server etc.) but i think not very good for data persisting also it's not safe as SQLite is.
Generally you should compare JSON with XML but you can't compare SQLite and JSON which are two different things.
General Idea
SqlLite : For Local Storage
JSON : For Server SideStore
It doesn't make sense to compare SQLite and JSON. One is a storage solution, the other one is a data format.
You should probably read the official guide before deciding which storage solution to choose (SQLite, preferences, file system,...): http://developer.android.com/guide/topics/data/data-storage.html
If you want to go for SQLite persistence you can use some third-party ORM to make queries easier (e.g ORMLite is the most popular one)
I am a php/mysql developer learning android. I am creating an android app that receives info from my php app to create list views of different products which will open a web view of that product's detail.
Currently my php cms web application outputs xml lists for an iphone app.... (also, separately outputs html). I have full control of the php app so if there is a better way to output the data for the android app please let me know.
I have created code that reads the xml from the web and creates the list view. The list can be refreshed daily, so the data does not need to be read from the online xml every time the app starts.
So I was thinking to store the data retrieved locally to improve my apps responsiveness. there may be up to 500 product descriptions to be stored at any given time in up to 30 different xml lists. I am starting development with one xml list with about 30 products.
For best performance should i store the product info in a sqlLite db or should i store the actual xml file in the cache/db or some other method like application cache.
I also was think to create the update of the data as a service, would this be a good idea?
The most efficient way to store data is RAM. But if you want to cache it, then the most efficient way is Database.
I recommend you store your data in sqlite android database.
You could also consider zipping you xml for faster network transfer and unzipping through java.util.zip package classes. You could even consider a simpler format for transmitting data, less verbose than xml, using a datainput/outputstream.
(I do that in of my apps and it works great)
Here are some details on data input / output stream method :
imagine a proprietary protocol for your data, only what you need. No tags, no attributes, just raw values in order.
on the client side, get an input stream on your data using URL.getContent() and cast it in input stream.
on the client side still, build a data input stream encapsulating your socket input stream and read data in order. Use readInt, readDouble, readUTF, and so on.
on the client side, from php, you need to find a way to save your data in a format that is compatible with the data format expected by the client. I can't tell much about PHP, I only program using java.
The advantage of this technique is that you save bandwith as there is only data and no verbose decoration due to xml. You should read about java specs to understand how double, int, strings are written in data output stream. But it can be hard using two languages to get the data right.
If php can't save format in a suitable way, use xml, it will be much simpler. First try with just plain xml, then give a try using a zip or tarball or xml file.
But all this is about speed gain during network connection.
The second part of what you have to do is to store each row of your list in a SQL table. Then you can retrieve it pretty fast using a CursorAdapter for your list view (it breaks the charming MVC model but it is quite fast !).
Sorry about this, but it became too long to write as a comment. This is not intended to be an answer to your question, because in my opinion Stéphane answered very well. The best solution is indeed to store the data in an sqlite database. Then you need to create the class to be used as a connection between the data, the database and the app. I don't want to take credit for what is said here already (I, too, voted it up).
I'm concerned with the other suggestion (use of low level raw streams for data manipulation, the list steps on that answer). I strongly recommend you to avoid creating your own proprietary protocol. It goes like this:
I need to exchange data.
I don't want to deal with the hassle of integrating external APIs into my code.
I know I can write two 5 minute routines to read and write the data back and forth.
Therefore, I just created my own proprietary format for exchanging data!
It makes me cry whenever I need to deal with unknown, obscure and arbitrary sequence of data blobs. It's always good to remember why we should not use unknown formats:
Reinventing the wheel is counter-productive. It seems not, but on the middle term it is. You can adapt your project to other mediums (server-side, other platforms) easily.
Using off-the-shelf components help you scale your code later.
Whenever you need to adapt your solution to other technologies and mediums, you'll work faster. Otherwise, you would probably end up with ad hoc code solutions that are not (easily) extensible and interoperable.
Using off the shelf components enables you to leverage advances in that particular technology. That's particularly important when you are using Android APIs, as they are frequently optimized for performance later down the road (See Android's Designing for Performance). Rolling your own standards may result in a performance penalty.
Unless you document your protocol, it's extremely easy to forget the protocol you created yourself. Just give it enough time and it will happen: you'll need to relearn/remember. If you document, then you are just wasting the computational time of your brain.
You think you don't need to scale your work, but chances are you will most of the time.
When you do, you will wish you had learned how to easily and seamlessly integrate well known formats.
The learning curve is needed anyway. In my experience, when you learn, you actually integrate well known formats faster than imagining your own way of doing things.
Finally, trust your data to geniuses that take their lives into creating cohesive and intelligent standards. They know it better!
Finally, if the purpose is to avoid the typical verbosity of XML, for whatever reasons, you have several options. Right now I can think of CSV, but I'm no expert in data storage, so if you're not confortable with it, I'm sure you can find good alternatives with plenty of ready to use APIs.
Good luck!