I'll begin with explaining how I stumbled upon SQLite Asset Helper library. I am trying to build a small android application which is basically shows the meaning of words. And to do so I intend to keep everything offline (no dependence of internet connectivity). Now, as far as I can think of, there are 2 ways of achieving that:
1. Using String array, which I believe will be a tedious task and a memory hog.
2. By providing a pre-populated database, using which I can easily establish relations between words and their meanings and do more (searching, sorting, etc).
Now, the problem I am facing is supplying a pre-populated database (or words and meanings) with the app itself. And for doing that I came across SQLite Asset Helper which does the job.
I have read a number of articles related to SQlite Asset Helper but not many which confirm its implementation on latest iterations of Android. Also, is the only possible solution to deliver a pre-populated database to the user (without needing to go online)? Is it acceptable method? Any other better alternative up for suggestion would be great!
I have read a number of articles related to SQlite Asset Helper but not many which confirm its implementation on latest iterations of Android
It works on the latest iterations of Android.
Also, is the only possible solution to deliver a pre-populated database to the user (without needing to go online)?
You are welcome to roll your own implementation. I do not know what you would gain by this.
Is it acceptable method?
I am not aware of anything better.
I've been using ORMLite in my application and I was considering whether to move to greenDAO.
Performance is a huge part of that decision, and greenDAO's Features page says:
For the same given entity, greenDAO inserts and updates entities over
2 times faster, and loads entities 4.5 times faster for loading
entities than ORMLite.
...
(Figures and chart updated 10-23-2011)
I thought ORMLite's config file generation step should remove the need for reflection at runtime.
The ORMLite changlog suggests that the greenDAO benchmark was done after the config file feature was released, but the greenDAO features page doesn't explicitly say if a static config file was generated for the test.
4.26: 9/26/2011 (svn r1978)
* ANDROID: Added field configuration utility to avoid Dao creation performance problems with annotations.
There have also been ORMLite performance fixes since then, e.g.
4.40: 4/16/2012 (svn r2516)
* ANDROID: Big performance bug fix when creating DAOs. Foreign fields always used reflection instead of table configs.
Can anybody confirm if there is still a big performance difference between greenDAO and ORMLite? Thanks!
We've just published a Github project that we used to compare the performance of ORMLite and GreenDao to raw SQLite:
https://github.com/daj/android-orm-benchmark
The project also allows you to compare the performance of an in-memory database to an on disk one.
The headline results are:
GreenDao is much faster than ORMLite. It is roughly:
3X faster at writing large numbers of objects.
50% faster at reading in all 10000 entries in a single table.
2X to 3X faster at an indexed read of a single row (though both were very fast).
15X faster at doing a LIKE search for 100 records in a 10000 entry table.
The project contains both a naive raw SQLite benchmark, and an optimized SQLite benchmark.
GreenDao vs unoptimized raw SQLite
GreenDao is 2X faster for the write benchmark.
GreenDao is 25% slower for the read benchmark.
GreenDao vs optimized raw SQLite
GreenDao is 50% slower for the read and write benchmarks.
For detailed results please see the Github repository above.
Of course we may have bugs in our benchmarking code...if you find any please fork, fix and submit a pull request! :-)
Disclaimer: make sure you do your own research before choosing GreenDao over ORMLite.
I'm writing an application which needs to store data. A single pack of data is about 4 classes with many dependencies between them. For example, class A has a list of objects B and B has a list of objects C and few more dependencies...
And I wonder what would be better. Keep them in SQLite db or serialize each pack separately and store them in serialized files?
For me the only right solution would be to save the data inside a database especially if there are any dependencies. For beginners it might be hard at the beginning to get into database creation. but after you have created a database in the right form you just have to insert the data and you won't have any problems in the future if you want to change something or expand your app. With simple serialisation the logic has to be solved inside the app and might cause more problems especially if you have any dependencies.
If you need a good tutorial for saving data you should look at this tutorial
http://thenewboston.org/watch.php?cat=6&number=111
For other different solutions for saving data there are also some tutorials on the website, Nr. 108 - 110 of Android programming
IT depends on the usage of the data. You may do well choosing JSON/GSON serialization and avoid the overhead of doing ORM over SQLite. (Overhead meaning additional coding to marshall to/from the db) However, if your data is subject to growth or something that would be better managed by a db (a lot of non-sequential or random access across a larger data set) then go for SQLite and ORM. In the end it comes down to what type of data you are trying to manage. Again if your data set is something that could grow and involves a lot of random access it may be worth using SQLite.
I want to share data across multiple applications, instead of having a ContentResolver->ContentProvider mechanism, I can just define a client library which talks to the process which does the SQLite DB operations right?
What does the ContentProvider brings in here which we cannot achieve by having a Process expose the data?
You can find answer Exact Difference between “Content-Provider” and “SQLite Database”.
But I like to explain this..
What does the ContentProvider brings in here which we cannot achieve by have a Process expose the data?
There is one particular SQLite limitation you should be aware of and that is that SQLite is single-user only. What this really means is that you will need to guard your database from being accessed from multiple threads at the same time. This is generally not a problem in a content provider, since they almost always have a single-threaded implementation.
Also It's good practice to provide the extra level of abstraction over your data to make it easier to change internally. What if you decide to change the underlying database structure at a later time? If you use a ContentProvider you can contain all the structural changes within it, where as if you don't use one, you are forced to change all areas of the code that are affected by the structural changes. Besides, it's nice to be able to re-use the same standard API for accessing data rather than littering your code with low-level access to the database.
You can check the thread below:
Exact Difference between "Content-Provider" and "SQLite Database"
We're designing an Android app that has a lot of data ("customers", "products", "orders"...), and we don't want to query SQLite every time we need some record. We want to avoid to query the database as most as we can, so we decided to keep certain data always in memory.
Our initial idea is to create two simple classes:
"MemoryRecord": a class that will contain basically an array of objects (string, int, double, datetime, etc...), that are the data from a table record, and all methods to get those data in/out from this array.
"MemoryTable": a class that will contain basically a Map of [Key,MemoryRecord] and all methods to manipulate this Map and insert/update/delete record into/from database.
Those classes will be derived to every kind of table we have in the database. Of course there are other useful methods not listed above, but they are not important at this point.
So, when starting the app, we will load those tables from an SQLite database to memory using those classes, and every time we need to change some data, we will change in memory and post it into the database right after.
But, we want some help/advice from you. Can you suggest something more simple or efficient to implement such a thing? Or maybe some existing classes that already do it for us?
I understand what you guys are trying to show me, and I thank you for that.
But, let's say we have a table with 2000 records, and I will need to list those records. For each one, I have to query other 30 tables (some of them with 1000 records, others with 10 records) to add additional information in the list, and this while it's "flying" (and as you know, we must be very fast at this moment).
Now you'll be going to say: "just build your main query with all those 'joins', and bring all you need in one step. SQLite can be very fast, if your database is well designed, etc...".
OK, but this query will become very complicated and sure, even though SQLite is very fast, it will be "too" slow (2 a 4 seconds, as I confirmed, and this isn't an acceptable time for us).
Another complicator is that, depending on user interaction, we need to "re-query" all records, because the tables involved are not the same, and we have to "re-join" with another set of tables.
So, an alternative is bring only the main records (this will never change, no matter what user does or wants) with no join (this is very fast!) and query the other tables every time we want some data. Note that on the table with 10 records only, we will fetch the same records many and many times. In this case, it is a waste of time, because no matter fast SQLite is, it will always be more expensive to query, cursor, fetch, etc... than just grabbing the record from a kind of "memory cache". I want to make clear that we don't plan to keep all data in memory always, just some tables we query very often.
And we came to the original question: What is the best way to "cache" those records? I really like to focus the discussion on that and not "why do you need to cache data?"
The vast majority of the apps on the platform (contacts, Email, Gmail, calendar, etc.) do not do this. Some of these have extremely complicated database schemas with potentially a large amount of data and do not need to do this. What you are proposing to do is going to cause huge pain for you, with no clear gain.
You should first focus on designing your database and schema to be able to do efficient queries. There are two main reasons I can think of for database access to be slow:
You have really complicated data schemas.
You have a very large amount of data.
If you are going to have a lot of data, you can't afford to keep it all in memory anyway, so this is a dead end. If you have complicated structures, you would benefit in either case with optimizing them to improve performance. In both cases, your database schema is going to be key to good performance.
Actually optimizing the schema can be a bit a of a black art (and I am no expert on it), but some things to look out for are correctly creating indices on rows you will query, designing joins so they will take efficient paths, etc. I am sure there are lots of people who can help you with this area.
You could also try looking at the source of some of the platform's databases to get some ideas of how to design for good performance. For example the Contacts database (especially starting with 2.0) is extremely complicated and has a lot of optimizations to provide good performance on relatively large data and extensible data sets with lots of different kinds of queries.
Update:
Here's a good illustration of how important database optimization is. In Android's media provider database, a newer version of the platform changed the schema significantly to add some new features. The upgrade code to modify an existing media database to the new schema could take 8 minutes or more to execute.
An engineer made an optimization that reduced the upgrade time of a real test database from 8 minutes to 8 seconds. A 60x performance improvement.
What was this optimization?
It was to create a temporary index, at the point of upgrade, on an important column used in the upgrade operations. (And then delete it when done.) So this 60x performance improvement comes even though it also includes the time needed to build an index on one of the columns used during upgrading.
SQLite is one of those things where if you know what you are doing it can be remarkably efficient. And if you don't take care in how you use it, you can end up with wretched performance. It is a safe bet, though, if you are having performance issues with it that you can fix them by improving how you are using SQLite.
The problem with a memory cache is of course that you need to keep it in sync with the database. I've found that querying the database is actually quite fast, and you may be pre-optimizing here. I've done a lot of tests on queries with different data sets and they never take more than 10-20 ms.
It all depends on how you're using the data, of course. ListViews are quite well optimized to handle large numbers of rows (I've tested into the 5000 range with no real issues).
If you are going to stay with the memory cache, you may want have the database notify the cache when it's contents change and then you can update the cache. That way anyone can update the database without knowing about the caching. Also, if you build a ContentProvider over your database, you can use the ContentResolver to notify you of changes if you register using registerContentObserver.