I'm curious if anyone has done any performance testing on querying a ContentProvider via ContentResolver vs querying a SQLiteDatabase object in the same process. I'm guessing that a ContentResolver query passes back a Cursor that communicates with the database through a Binder (Android IPC). This means that if I read the contents of 100 records through the Cursor that would result 100 Binder method calls. Are my guesses correct and if so would that be significantly slower than accessing the database in the same process?
I have not done exactly that meassure. What I did was to meassure the performance of multiple inserts via a ContentProvider or directly via a SQLite database.
I inserted around 1000 items (one by one). It was much slower to insert via a ContentProvider. In my test almost 10% slower.
There's no definitive answer and results depends on what and how you do it.
For example, I want to share preferences between apps so ContentProvider seems the perfect answer. Yes if I don't mind a longer delay on first read as just connecting to a ContentProvider takes 120ms on a S10+ these days!
So if you have a UI depends on those settings, you'd better copy the preference file between apps (using a ContentProvider) and then read from the file directly otherwise the UI will be delayed before showing with appropriate theme. Fact is the onStart() will have already been called for the newly created activity.
On the opposite side doing DB operations (if done right) will not change much of the results, unless you need to re-connect frequently as it will add a non neglectable overhead.
Related
I am creating an Android app for which I need to create a SQLite DB and pre-populate it with some values.
The Android documentation says this about what to do in "onCreate" of the SQLiteOpenHelper:
Called when the database is created for the first time. This is where the creation of tables and the initial population of the tables should happen.
Reference - http://developer.android.com/reference/android/database/sqlite/SQLiteOpenHelper.html#onCreate(android.database.sqlite.SQLiteDatabase)
I am doubtful about the following 2 things -
What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
Any help is appreciated.
1) The later. It is done on the first read or write to the DB.
Your fear might be correct, this is why you can ship your app with a database that's already populated. Or you can launch an AsyncTask with a simple SELECT 1 FROM anytable query. More about shipping with DB here. (60 rows is nothing to fear about tho, and you can safely just keep using AsyncTasks).
2) Yes it is fine. The onCreate logic will run when you first read/write the DB, so it if you always use AsyncTasks onCreate will run in an AsyncTask also.
What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
It happens when you first query from database in general term. After that only Upgrade method is called that too when you change the db version.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
60 rows insertion is not a big task. More you can read about beginTransaction(),commitTransaction and endTransaction for insertion. It will make your insertion task lighting fast.
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
It good you are doing you Db operation in AsyncTask and its completely fine.
Speaking of DB operations:
Performing DB operations in AsyncTask is not a good approach, generally. As you might encounter a problem called "memory leak", and it might come as a silent assassin in the night.
There's lot written on this issue. Just google "asynctask leak context" and here you go.
So how to perform DB operations?
Using Loader API in conjunction with ContentProvider is considered good approach for querying database. Loader asynchronously queries your database and delivers the result to specified subscribers. Configuration changes or other sudden stuff does not bother it.
And it is really convenient to query your data using loader API once you know how to do it.
Single inserts/updates/deletes might be done directly from the main thread via ContentResolver. These calls will be blocking (synchronous), but I bet you user would never notice anything while the amount of data is not large.
If you're operating on a large dataset, and you fear you'll be significantly blocking UI thread, I'd suggest using IntentService or any custom Service capable of doing operations in background (note that by default Service operates on main UI thread and you have to specify background operation yourself or use IntentService)
Speaking of DB initialisation:
You might create a one-time IntentService, if you're initialising a large set of data. It will handle your request asynchronously and, for example, perform a broadcast that the application is set up and ready, so you might stop a "wait a sec, performing app initialisation" screen and show user your data.
There's also nothing wrong with shipping your database along with application, though it appears to be a bit hackish solution.
Either way, you choose what is more suitable for you.
I am planning on writing an application that saves a fair amount of data. Historically, I have simply written data directly to a server, and only used some simple key/value storage with shared preferences for local storage.
I am considering this time, instead, using SQLite to save the information at first, and sync the data to the server in the background later. This will benefit the user in a few ways: 1) can use the app offline 2) don't have to worry about data being saved right away, it happens when ever it can 3) more reliability.
My approach will be to get/set data from SQLite during UI usage, and use a background process to find new rows and put them on the server, flagging them as synced when it happens.
Does this sound reasonable?
You can use SQLIte for your scenario. But, while implementing, you can follow any one of this approach.
Approach #1: Use an Abstract Factory to Instantiate the SQLiteOpenHelper.
Approach #2: Wrap the SQLiteDatabase in a ContentProvider
Refer to this link for how to implement these 2 approaches. http://www.androiddesignpatterns.com/2012/05/correctly-managing-your-sqlite-database.html
Key points to be noted while using SQLite
Sqlite takes care of the file level locking.
Many threads can read,one can write. The locks prevent more than one
writing.
Android implements some java locking in SQLiteDatabase to help keep
things straight.
If we handle the database incorrectly from many threads and mess up the code, your
database will not be corrupted. Only few updates will be lost.
How "Multiple Threads - DB access" can be used for your scenario
The SqliteOpenHelper object holds on to one database connection.
If you try to write to the database from actual distinct connections (multiple threads) at the same time, one will fail. It will not wait till the first is done and then write. It will simply not write your change. Worse, if you don’t call the right version of insert/update on the SQLiteDatabase, you won’t get an exception. You’ll just get a message in your LogCat, and that will be it.
So recommended to write using single thread and read from multiple threads if necessary for faster access.
Does this sound reasonable?
Yes. Note that the synchronization process can get tricky (e.g., what happens if the server hiccups halfway through?), but that has mostly to do with synchronization and little to do with SQLite.
We implemented a solution that used a SQLite db on the device to sync data via a web service to the master database. We did this for a couple reasons: offline, poor connection, manual sync.
For our solution we had a flag on the table that determined if the data was pushed to the web service. Our web service also provided data back to our application to let us know if the data was received and processed correctly. This allowed us to clean up the data on the device, send notifications if there were failures, and resubmit the data if there were previous failures.
You can use push notifications as well if you have fixed the issues on the backend and have the device resend the data to the web service. This worked really well for us.
I am trying to understand the possible ways to work with SQLite when there can be multiple threads work on DB.
Based on various responses in stackoverflow and other sites, it appears that there will be locking issue when same sqlitehelper instance is used from multiple threads. In a typical java application, I would expect instance to mean single object of type sqlite helper to be used by different threads of application.In such cases, the locks ,I guess, are a matter of correctly using the synchronized blocks. [Correct me here as I am not comfortable with this way of looking at sqliethelper instance here]
My concern is with sharing same data base : when one instantiate sqlite helper in different threads [ie each thread has its own object instance] but working on same Database [this I guess is more inline with having same db instance].
In such cases I'm getting frequent database lock errors. This occurs even when the threads are working on different tables of database.
In my application database can be updated by user interaction through application or by getting data through server [periodic synchronization]. And some time when synchronization process and user activity overlaps, I get the lock issues. As this pattern of data processing seems to be common in application synchronizing with server, would like to know how do lock issue due the concurrency is to be handled.
I would like to understand this since if this is bound to happen always then probably need to make only one handler over database and implement queue over that to avoid lock. But that will mean the complete application needs to be aware that the database may not get updated immediately and they need to implement listener to know when the data is actually updated in database.
thanks
pradeep
As far as I know sqlite is intended for single process usage. No matter what you will always need to access the database from one thread at a time. You can do selects from multiple clients but can only write from one at a time. And other readers and writers will ahve to lock in the mean time.
As a side note - database access can hardly ever be considered instantaneous.
I have a certain update method in my Android App, which generates quite an amount of data - up to hundreds of database entries.
I also have a background service, aside from the UI thread. Both threads have to execute the update method, sometimes even at the same time - basically, this is about generating and caching data to display. Both the UI and background service need this data.
Currently, I have wrapped the method's execution in an ORMLite transaction, which maps to an ordinary SQLite transaction. However, I am afraid that this will bite me in the butt one day, when some race condition screws up the data cache.
The question: Do SQLite transactions protect me from concurrent execution, or should I rather implement some kind of worker thread which is spawned when the generator-method shall start, or blocking if the generator-method is already running?
UPDATE:
I have decided to not rely on SQLite logic for the protection of my high-level Java method. The solution was for me as follows:
Wrap the generating part of the method with synchronized
Introduce a variable which tracks the last time of executing the method (set at the end of the method, so it is the marker of execution END)
First thing in the synchronized section, check if the last execution is in a specific threshold (e.g. <= 100ms in the past)
If yes, skip generation
If no, perform generation
In this way, duplicate generation should not take place, since when the method is accessed from two threads at the same time, the first will generate, but the second will not. The most important part for me here is that it is still blocking, since both threads rely on the generation having taken place after they have called the method.
EDIT:
It seems I'm wrong in my below statement: The SQLite implementation is, according to many, thread safe. I have, however, bitterly experienced threading issues, especially when testing database access, but that must have been caused by other factors in my code then, I assume.
Sorry for the misleading answer.
ORIGIN:
Good question!
You should be very careful here because the standard Android database access objects (such as SQLiteDatabase, Cursor etc) are not thread-safe by default. Not even ContentProvider's seem to give you a complete protection unless you explicitly write them with multithreading in mind.
According to Android documentation on ContentProvider's and threading (almost at the end of the page):
"Because these methods [update() is one of the functions] might be called from any number of threads at the same time, they too must be implemented to be thread-safe."
I don't know if there is any explicit locking mechanism to SQLiteDatabases (as in locking the actual database file). I would assume that a transaction itself would lock, at least the very handle you access your database with. I don't know what is true for the case where you have multiple handles to your database.
Maybe you could try to implement some singleton object (A ContentProvider maybe?) to access your database with, but even then you'd have to manage some sort of "request queue" I suppose.
You should also consider not to make any calls to the file system (the database is on the file system) from the UI-thread, what-so-ever. There is no guarantee that the database will answer in time and you're likely to end up with an ANR (especially as you write "...which generates quite an amount of data").
I'm developing an Android application. It has multiple threads reading from and writing to the Android SQLite database. I am receiving the following error:
SQLiteException: error code 5: database is locked
I understand the SQLite locks the entire db on inserting/updating, but these errors only seem to happen when inserting/updating while I'm running a select query. The select query returns a cursor which is being left open quite a wile (a few seconds some times) while I iterate over it. If the select query is not running, I never get the locks. I'm surprised that the select could lock the db. Is this possible, or is something else going on?
What's the best way to avoid such locks?
You are probably opening and closing multiple database connections in your various threads. This is a bad idea. Just open a single database connection, and reuse it everywhere; SQLite will then ensure that concurrent accesses are serialized correctly.
As with jcwenger's answer, using a ContentProvider is another way of achieving this, but will require much more intrusive changes to your code.
By avoiding leaving cursors open for "quite a while". If you can afford to have all your data in memory all at once, then do so.
If you can't, then try increasing the busy timeout.
Migrate to a ContentProvider rather than directly accessing the DB. ContentResolver marshals away all the threading issues for you and also allows for other useful features like sharing data between apps or syncing with a server.
The api overhead of ContentResolver is minimal. You just need to define an AUTHORITY string (A unique string identifying the "kind" of your data -- use a "com.example.myapp.contacts" type of string) and use ContentResolver.bla rather than db.bla.
Its caused by beginTransaction() function.Look at your code, the problem is solved for my app to making a comment line this function(beginTransaction) line