I am about to implement backup for my Android app, and my issue is: The data resides in an sqlite db. Some of the data are just there for user convenience, and can be recreated from other sources. So in order to minimize the size, I wish to export relevant data only (the limit for backup using the Google API is 1mb).
All the data has class equivalents, which are populated via my SQLiteOpenHelper implementation. This means I can implement serialization.
So far I can see the following options:
Serialization using Java Serializable, and write all objects into a binary chunk and pass it to writeEntityData()
Serialization using XML or JSON, perhaps together with the zip API and dump the file as a binary chunk
Clone the database with relevant objects only. Probably a lot of work.
So far, using XML or JSON seems to be the best option, as I can reuse that for data sharing across users/devices. Java Serializable seems to bloat the size..
Would like to hear your opinions on this !
I recommend you to use vacuum() in order to shrink the db size, or use the auto_vacuum pragma.
If your db is something big, you can try to zip it.
Delete all unnecessary libs (compat_v7, for example, if you don't need it).
Try to compress images with optipng.
Try to convert your wav or mp3 to aac.
And... we are talking about how many Mb?
Related
I have been looking at different ways to hold onto some predefined character data, however I am having a hard time nailing down which would be the best solution.
An example of data would be 10 strings, 5 int arrays (of size 10 each). There would be 10+ set's of this data. The application would load in the information and inflate generic "character" objects.
Possible solutions:
XML: Due to Android's structured XML requirements it can be hard to use without making a different XML file for every character, and even then it would have ID overlapping for similar named data values.
SQLite: Wouldn't be a huge database, but databases are ugly version controlled unless it is done with a create-database script which has its own downsides (such as making sure DB is up to date between builds).
Hard-coded Objects: By far my least favorite solution, using polymorphism to hardcode all of the objects. Too dirty, not nearly as dynamic as it should be.
I would like to consider things such as version controlling the files, ease of updating (due to them only being inflated, never changed by the app).
If this data is baked i would suggest to use harcoded data.
Reasons.
In those three solution you save the data in the application.
If you use XML-data, you have to consume the time while code parsing inside the code. And you have to write the code that parses your xml.
If you use SQLite, your data will be doubled because of you have to store this database in raw or assest directory, copied in the /data/data folder. Futhermore, if you use Strings and SQLite by default the data will be doubled again (due to UTF-16 encoding).
If think, if only you manage the data this is more usefull to store directly inside the code. Obviously, if you do not use tons of content:)
You might want to use the Realm framework, which is comparatively faster than SQLite and easy to implement inside in your current code.
It handles large data too and it feels like you're using only native android classes.
I have read through the Android Storage Options and I have a question that I haven't been able to find the answer to:
Should I use SQLite to store my data or should I use a JSON object that is written to a file?
Requirements:
Store (up to) a few hundred instances of the same object. Each instance will be somewhat complex, storing reference to images, smaller objects, etc. The data will be stored locally, with the option of cloud backup. All the data will be loaded on startup and saved when manipulated by the user.
The reason I ask this is because I don't have a lot of data to store - for a SQLite database there will probably never be more than a few 100 rows, which makes me think SQL is overkill.
Also, exporting my data to a JSON file will allow me to easily import/export from different device platforms (I already do this on iOS).
Or, maybe there's a better option? If there was an NSCoding type library for Android I would probably use that.
Any opinions are helpful.
Thanks!
From the presented so far, storing in files will be more advantageous.
Considering that each "unit" is less than 16 attributes, a json file with short identifiers will likely generate a larger file representation than the SQL representation equivalent.
However, the local file manipulation will allow for easier interactions, as well as easier backing up/down.
Also, the File class is simple enough to generate less issues when compared to SQL.
Finally, given the choices, you are going to have to evaluate the operations used.
If you are going to compare the data, then SQL is likely to go faster, but if you are just inputting/outputting each data as a separate object, than files are going to be as fast as SQL.
Finally, please, particionate your objects, do not create just 1 file with all the info.
I have read through the Android Storage Options and I have a question
that I haven't been able to find the answer to:
Should I use SQLite to store my data or should I use a JSON object
that is written to a file?
You need to analyse your requirement again.
maybe there's a better option?
It depends upon your requirement.
if Your requirement is fixed to simply storing and retrieving then you can have a look on tinnyDB, which is basically using the SharedPreferences as storage mechanism. But if you need case base based selection/query of data then you should go with SQLite.
I want to give backup facility in my android application. So for that purpose i don't know which which format will be suitable. I am thinking either XML or CSV. Please tell me which is efficient.
It's my opinion that you're probably better off using JSON, as it has many great advantages as listed below and given your data, wont be considerably larger than CSV or Binary.
Take a look at this post for details on how to implement it:
How to parse JSON in Android
The following is a general breakdown of the different data format options:
XML
This format is the least efficient (file size and time to parse), but comes with the advantage that it can be easily debugged or modified/read by a person. In general, use this if you are going to be reading the content, displaying it in some other program or the file will be small enough that it's size and processing disadvantages don't have a significant effect.
JSON
More concise than XML, while still maintaining it's human readability. It's syntax isn't quite as simple as XML but it's still very simple. I would recommend this over XML.
CSV
This format is much more efficient than XML, but is prone to errors if modified manually and can be very hard to read. You will likely require special care in dealing with the delimiting character so you'll want to find yourself a simple CSV library. It's disadvantages are that although
Binary
These formats will be read/written to a file as bytes. They are structured in such a way that only your specific application/reader will know how how the bytes are structured. This format is the most concise and has the best read/write performance, but of course, it's practically impossible to modify or read.
Edit: Also worth considering is your ability to modify the format of the data, for the purposes of supporting future version changes. Using JSON or XML allows you to easily add new fields or ignore existing ones and so can be easier to maintain backwards compatibility for existing applications without breaking their functionality. A similar solution for CSV or Binary would require that you store and check the data format version number with the files, and then manually switch between loaders in code.
I'd go (and I use them in my apps) for CSV files, since the data are crude and concise (i.e.: small file size and fast to read/write).
I won't choose XML files, which put a lot of garbage in the file, bloating them ridicolously.
I need to download onto my devices some data in multiple files.
Then this data will be copied to application's local db (this is SQLite db, however in future this may be Compact SQL on WInPhone).
What is the best format for such files?
I am considering such possibilities:
SQLited db file - possibly this will be easy to copy to my db. My current prefferance.
JSON format. Maybe not enough compact because column name will be repeating.
CSV - it allows to store only one table but I would prefer have few tables in one file
XML - I do not see any prefferaces over json.
JSON is the most popular, human-readable, easy to use format. There's tons of supporting libraries, native and not, for all OSs. It's fast and reliable. You can easily update the data you pass with it without updating the apps (which you cannot by passing an SQLite database and would be difficult with a CSV file). XML is being slowly deprecated for data communications... but if you see some special advantage with XML (parsing the XML directly, which is not as effective with JSON yet, for example), go for it. I'd choose JSON anyway, it's the current standard and will still be for a long time.
In my app I have to download JSON data from numerous web services. The data classes I use are fairly complex ones (lots of properties, quite deep inheritance tree, etc.).
I intend to do caching, using a single db table, where I'd store the downloaded JSON data in a VARCHAR column (along with other meta-data containing columns). JSON serialization is being done with the Gson library.
It seems quite convenient to just dump the instances into JSON, and parse them again later when I need them. No need to create custom tables for every class, or write loads of custom serialization code. Also, I can do queries on the cache table this way.
The question: Is this approach an anti-pattern by any means?
There is absolutely nothing wrong with this approach; however, I am going to recommend that you instead use the built in caching storage. See the section called "Saving cache files" in Data Storage for more details.This way you don't hog any precious space if your JSON objects are large in the event of a low memory situation.