I need to download onto my devices some data in multiple files.
Then this data will be copied to application's local db (this is SQLite db, however in future this may be Compact SQL on WInPhone).
What is the best format for such files?
I am considering such possibilities:
SQLited db file - possibly this will be easy to copy to my db. My current prefferance.
JSON format. Maybe not enough compact because column name will be repeating.
CSV - it allows to store only one table but I would prefer have few tables in one file
XML - I do not see any prefferaces over json.
JSON is the most popular, human-readable, easy to use format. There's tons of supporting libraries, native and not, for all OSs. It's fast and reliable. You can easily update the data you pass with it without updating the apps (which you cannot by passing an SQLite database and would be difficult with a CSV file). XML is being slowly deprecated for data communications... but if you see some special advantage with XML (parsing the XML directly, which is not as effective with JSON yet, for example), go for it. I'd choose JSON anyway, it's the current standard and will still be for a long time.
Related
I have been looking at different ways to hold onto some predefined character data, however I am having a hard time nailing down which would be the best solution.
An example of data would be 10 strings, 5 int arrays (of size 10 each). There would be 10+ set's of this data. The application would load in the information and inflate generic "character" objects.
Possible solutions:
XML: Due to Android's structured XML requirements it can be hard to use without making a different XML file for every character, and even then it would have ID overlapping for similar named data values.
SQLite: Wouldn't be a huge database, but databases are ugly version controlled unless it is done with a create-database script which has its own downsides (such as making sure DB is up to date between builds).
Hard-coded Objects: By far my least favorite solution, using polymorphism to hardcode all of the objects. Too dirty, not nearly as dynamic as it should be.
I would like to consider things such as version controlling the files, ease of updating (due to them only being inflated, never changed by the app).
If this data is baked i would suggest to use harcoded data.
Reasons.
In those three solution you save the data in the application.
If you use XML-data, you have to consume the time while code parsing inside the code. And you have to write the code that parses your xml.
If you use SQLite, your data will be doubled because of you have to store this database in raw or assest directory, copied in the /data/data folder. Futhermore, if you use Strings and SQLite by default the data will be doubled again (due to UTF-16 encoding).
If think, if only you manage the data this is more usefull to store directly inside the code. Obviously, if you do not use tons of content:)
You might want to use the Realm framework, which is comparatively faster than SQLite and easy to implement inside in your current code.
It handles large data too and it feels like you're using only native android classes.
I have read through the Android Storage Options and I have a question that I haven't been able to find the answer to:
Should I use SQLite to store my data or should I use a JSON object that is written to a file?
Requirements:
Store (up to) a few hundred instances of the same object. Each instance will be somewhat complex, storing reference to images, smaller objects, etc. The data will be stored locally, with the option of cloud backup. All the data will be loaded on startup and saved when manipulated by the user.
The reason I ask this is because I don't have a lot of data to store - for a SQLite database there will probably never be more than a few 100 rows, which makes me think SQL is overkill.
Also, exporting my data to a JSON file will allow me to easily import/export from different device platforms (I already do this on iOS).
Or, maybe there's a better option? If there was an NSCoding type library for Android I would probably use that.
Any opinions are helpful.
Thanks!
From the presented so far, storing in files will be more advantageous.
Considering that each "unit" is less than 16 attributes, a json file with short identifiers will likely generate a larger file representation than the SQL representation equivalent.
However, the local file manipulation will allow for easier interactions, as well as easier backing up/down.
Also, the File class is simple enough to generate less issues when compared to SQL.
Finally, given the choices, you are going to have to evaluate the operations used.
If you are going to compare the data, then SQL is likely to go faster, but if you are just inputting/outputting each data as a separate object, than files are going to be as fast as SQL.
Finally, please, particionate your objects, do not create just 1 file with all the info.
I have read through the Android Storage Options and I have a question
that I haven't been able to find the answer to:
Should I use SQLite to store my data or should I use a JSON object
that is written to a file?
You need to analyse your requirement again.
maybe there's a better option?
It depends upon your requirement.
if Your requirement is fixed to simply storing and retrieving then you can have a look on tinnyDB, which is basically using the SharedPreferences as storage mechanism. But if you need case base based selection/query of data then you should go with SQLite.
I am about to implement backup for my Android app, and my issue is: The data resides in an sqlite db. Some of the data are just there for user convenience, and can be recreated from other sources. So in order to minimize the size, I wish to export relevant data only (the limit for backup using the Google API is 1mb).
All the data has class equivalents, which are populated via my SQLiteOpenHelper implementation. This means I can implement serialization.
So far I can see the following options:
Serialization using Java Serializable, and write all objects into a binary chunk and pass it to writeEntityData()
Serialization using XML or JSON, perhaps together with the zip API and dump the file as a binary chunk
Clone the database with relevant objects only. Probably a lot of work.
So far, using XML or JSON seems to be the best option, as I can reuse that for data sharing across users/devices. Java Serializable seems to bloat the size..
Would like to hear your opinions on this !
I recommend you to use vacuum() in order to shrink the db size, or use the auto_vacuum pragma.
If your db is something big, you can try to zip it.
Delete all unnecessary libs (compat_v7, for example, if you don't need it).
Try to compress images with optipng.
Try to convert your wav or mp3 to aac.
And... we are talking about how many Mb?
I am looking to use an XML file to store the data my Android app generates. With that in mind, I have two questions for the community:
Is XML the best way to store data on Android and most efficient in cases where data may be added or altered every second or less then a second.
If XML is indeed the best for the scenario described in #1, how do I go about setting it up?
1.) Is XML the best way to database data on android and most efficient in cases where data may be added or altered every second or less then a second.
Definitely not.
2.) If XML is indeed the best for the scenario described in #1, how do I go about setting it up?
If you plan to store data just locally, the best way would be SQLite which works as a local database on every device.
If you later plan to synchronize this data with a central database, you may do this asynchronously within an AsyncTask or a Thread which would run periodically, but writing each second into a XML file is a bad idea as far as performance goes.
It's probably also a bad idea synchronizing a remote database at each insert/modification/deletion operation as if you had many users you could collapse the remote database.
I think the best approach is (as previously said) having a local database where you would store that data, and implement a webservice in the remote side if needed and use it to periodically synchronize both databases.
I would use JSON over XML and I would highly consider using GSON from Google. You maybe want to consider writing directly to a database with it's own structure and use transactions and sets. Is there are reason you want to go through JSON/XML?
XML is one of the worst ideas to keep local data in Android.
Most common used is SQLite available on the Android platform, but it all depends on what data and how you want to use.
In many mobile applications you don't need the relational database for one of the following reasons:
You have no relational data (i.e. settings) => no point in making relational tables with 1 record each
You have small, and dynamically changed data (like cache for downloaded content)
You don't need to search for data (using indexes etc.)
What alternatives can be used?
Shared preferences - simple key/value storage of primitive objects
Data serialization - for your consideration - binary (native java), JSON, parcelable (can be combined with the shared preferences)
For most of my app I'm currently using the binary serialization for "local storage".
- It's fast enough (usually much faster than starting the local SQLite engine)
- It's extremely easy and quick to implement, especially when you are using it for json/xml downloaded data parsed to POJO objects. All you need to do is just put "extends serializable" and put few lines of code to serialize/deserialize whole structure
- You can use those same classes for keeping data locally and communication with backend
Of course - it all depends from the situation - if you want to keep locally log of data from some sensor, or allow others apps to use this data, have to quick filter 1k+ records, or you really like to write few hundreds lines of code SQLite will be the best option for you. But most of mobile applications has no clear reason to use the relational (and trust me - not perfect one) engine.
In my app I have to download JSON data from numerous web services. The data classes I use are fairly complex ones (lots of properties, quite deep inheritance tree, etc.).
I intend to do caching, using a single db table, where I'd store the downloaded JSON data in a VARCHAR column (along with other meta-data containing columns). JSON serialization is being done with the Gson library.
It seems quite convenient to just dump the instances into JSON, and parse them again later when I need them. No need to create custom tables for every class, or write loads of custom serialization code. Also, I can do queries on the cache table this way.
The question: Is this approach an anti-pattern by any means?
There is absolutely nothing wrong with this approach; however, I am going to recommend that you instead use the built in caching storage. See the section called "Saving cache files" in Data Storage for more details.This way you don't hog any precious space if your JSON objects are large in the event of a low memory situation.