My app runs some very complicated queries using joins and other things which the Android ContentResolver is not equipped to handle. Some of these queries, inserts, etc can block the UI thread, so I run them in the background. As you might guess, this can cause a SQLiteException (database locked) when the timing of two sql requests coincide with one another (the most common cause of this is data loading into my db from our server in the background while the user requests to view data from the db on the screen).
The best way to fix this issue is to keep all DB activity on one thread, but like I said above some inserts in particular would cause some big time UI thread blockage, so that's not really realistic. The other option I've seen is synchronizing all the methods of the ContentResolver (they all should share a common member variable like a DatabaseHelper). I've done this, but again, this is not enough, because for some methods I have to use the SQLiteDatabase object's rawQuery() method to pull off the sql I need to get the data.
Is anyone aware of a strategy I could use to synchronize both ContentResolver methods and SQLiteDatabase methods? Thanks a lot!
I would make the long inserts in a transaction
db.beginTransaction();
try {
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
If you don't want that the requests that a user makes during the insert returns empty content, implement a logic of retry.
Would not this work ?
Related
I am creating an Android app for which I need to create a SQLite DB and pre-populate it with some values.
The Android documentation says this about what to do in "onCreate" of the SQLiteOpenHelper:
Called when the database is created for the first time. This is where the creation of tables and the initial population of the tables should happen.
Reference - http://developer.android.com/reference/android/database/sqlite/SQLiteOpenHelper.html#onCreate(android.database.sqlite.SQLiteDatabase)
I am doubtful about the following 2 things -
What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
Any help is appreciated.
1) The later. It is done on the first read or write to the DB.
Your fear might be correct, this is why you can ship your app with a database that's already populated. Or you can launch an AsyncTask with a simple SELECT 1 FROM anytable query. More about shipping with DB here. (60 rows is nothing to fear about tho, and you can safely just keep using AsyncTasks).
2) Yes it is fine. The onCreate logic will run when you first read/write the DB, so it if you always use AsyncTasks onCreate will run in an AsyncTask also.
What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
It happens when you first query from database in general term. After that only Upgrade method is called that too when you change the db version.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
60 rows insertion is not a big task. More you can read about beginTransaction(),commitTransaction and endTransaction for insertion. It will make your insertion task lighting fast.
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
It good you are doing you Db operation in AsyncTask and its completely fine.
Speaking of DB operations:
Performing DB operations in AsyncTask is not a good approach, generally. As you might encounter a problem called "memory leak", and it might come as a silent assassin in the night.
There's lot written on this issue. Just google "asynctask leak context" and here you go.
So how to perform DB operations?
Using Loader API in conjunction with ContentProvider is considered good approach for querying database. Loader asynchronously queries your database and delivers the result to specified subscribers. Configuration changes or other sudden stuff does not bother it.
And it is really convenient to query your data using loader API once you know how to do it.
Single inserts/updates/deletes might be done directly from the main thread via ContentResolver. These calls will be blocking (synchronous), but I bet you user would never notice anything while the amount of data is not large.
If you're operating on a large dataset, and you fear you'll be significantly blocking UI thread, I'd suggest using IntentService or any custom Service capable of doing operations in background (note that by default Service operates on main UI thread and you have to specify background operation yourself or use IntentService)
Speaking of DB initialisation:
You might create a one-time IntentService, if you're initialising a large set of data. It will handle your request asynchronously and, for example, perform a broadcast that the application is set up and ready, so you might stop a "wait a sec, performing app initialisation" screen and show user your data.
There's also nothing wrong with shipping your database along with application, though it appears to be a bit hackish solution.
Either way, you choose what is more suitable for you.
I have an Android Application that holds a fairly large Database which is accessed from multiple threads. I am starting to get Database locked Exceptions. My question, is there a way of creating a database Queue (similar to FMDataBase.Queue in iOS)?
I have done some research and I don't want to create a helper class as a lot of my database queries and inserts are different so creating a method for each query is not feasible.
I could put
if(!db.isLocked)
{
//exec(BLAH)
}
else
{
//try{thread.sleep(1000);}... then some retry code
}
on every database function but there must be a better way than this.
moreover if the thread is locked after is has slept for 1000ms, it will still crash and if the db is locked for 10ms I will be waiting for 990ms for the query to run, which is not great for user experience.
Is is possible to create a Queue so that any commands sent to the db will be executed once the the db becomes unlocked?
Any advice would be greatly appreciated
My question, is there a way of creating a database Queue
Move all your database writes to an IntentService, a regular service hosting a LinkedBlockingQueue, a LinkedBlockingQueue held by your singleton SQLiteOpenHelper, etc.
a lot of my database queries and inserts are different so creating a method for each query is not feasible.
The overhead to wrap a database I/O in a method not especially much, less than what you were proposing doing in your question.
if the db is locked for 10ms I will be waiting for 990ms for the query to run, which is not great for user experience
Tune your database access (e.g., EXPLAIN keyword, then set up appropriate indices). Use Traceview to determine exactly where your problems lie. If your problems are in a database transaction (e.g., you open your own transaction and do a ton of work inside of it), consider using yieldIfContendedSafely().
I am wondering when or if its ok to use a cursor to get data from a sqlite database on the ui thread. Basically I use a cursorloader to get a cursor. And then I want to do something with that cursor. Is it ok to do something with the cursor from the main ui thread? Really I just need to read one row from the cursor.
Retrieving a row from a Cursor is not an expensive operation... querying the data from the database, on the other hand, could be time-consuming depending on the amount of data you are requesting and/or the complexity of the query. CursorLoaders query for Cursors asynchronously so your approach is fine.
It's OK. It's not a network operation.
EDIT: by "it's OK" I mean that Android 4.x won't force-close your app for doing that, like if would for performing network I/O on the main thread. Depending on the query time, it might or might not be appropriate to stall the UI thread. For a "get a single row from a single table by its primary key" kind of query, it's perfectly fine. For a hairy query that takes a second or more, consider a background thread and a progress dialog.
I have an application that I can insert data into a SQLite database. There's a service thread running every sixty seconds checking if a condition is true and raising an alarm if necessary. Think "calendar" application.
I have lots of activities with a reference to their own SQLiteOpenHelper, and their own SQLiteDatabase object.
The application was working pretty well, but one of the main update applications I decided needed a progress dialog subject - that's another subject, but if you can tell me how I can get the spinner to spin that'd be great. It does however display.
But to get it to display, I needed to put it in a thread.
Creating this thread, I started to get loads of "database is locked" messages in the Log.
I figured the issue may have been related to the fact I had a task in the background running every sixty seconds - even though it visually only took a couple of seconds to do the update, so I placed the main write transactions for the updater function in a beginTransaction block. That's the only place I used one.
Everything seemed good, but then I started getting more spurious errors about the database being locked all over the place.
It's encouraged me to do some tidying up, but it seems after I do the first beginTransaction, any further database modifications I want to make fail with a report that the database is locked, despite that fact I do call db.close. I even close the database helper object and cursor just to be on the safe side.
I was able to clear up most of the warnings with regards to cursors still being open, eg
E/Database( 678): android.database.sqlite.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here
I couldn't figure out what was locking the database as everything was being closed.
I then decided to remove the "beginTransaction" which stabilised things - I could now navigate my program again. But Im still getting a few random locking issues.
E/Database( 1304): Failure 5 (database is locked) on 0x26e5d8 when executing 'INSERT INTO
Is it possible to figure out what is locking the database? I can see what is grumbling about the lock, but not what is locking it.
What is the best way to handle concurrent database updates/reads like this? I've read a lot about synchronized and ContentProviders but I must admit, it is a little bit big for me.
Any pointers on the "best" way of doing things?
And can I find out what is locking the database?
Thanks
Simon
class NAME extends SQLiteOpenHelper
add:
#Override
public void onOpen(SQLiteDatabase db) {
super.onOpen(db);
if (!db.isReadOnly()) {
db.execSQL("PRAGMA foreign_keys=ON;");
Cursor c = db.rawQuery("PRAGMA foreign_keys", null);
if (c.moveToFirst()) {
int result = c.getInt(0);
}
if (!c.isClosed()) {
c.close();
}
}
}
You may want to look at the latest version of Berkeley DB as an option. It provides a SQLite3-compatible SQL API (so that your application can stay the same) and it provides a finer granularity of locking and therefore better concurrency. Berkeley DB supports multiple concurrent read and write threads to actively access the database at the same time. You can find out additional information about Berkeley DB on our website.
I've got a kind of heavy Android application and want to make sure not never ANR it.
I've moved my database queries away from the UI thread (to AsyncTasks), but I still read from the cursor in the ui thread since I suppose the returned cursor is stored somewhere in memory, i.e. the actual reading of it does not go the entire way to the database. Is that correct or do i really need to move all reads of the cursor to non UI threads as well?
To be more specific:
Does for example http://developer.android.com/reference/android/database/Cursor.html#getInt(int) read from memeory, or does it aquire some kind of read lock from the actual SQLite database.
I guess the Cursor implementation in my case is an SQLiteCursor since the ContentProvider is implemented using an SQLite database.
When you call query() or rawQuery() on a SQLiteDatabase, a Cursor is returned immediately, because the actual query itself is delayed until you start using data. Any call that manipulates the Cursor or needs data that implies that the query is executed (e.g., getCount()) will actually execute the query. Hence, it is best to "touch" the Cursor in doInBackground() while you are on the background thread.
From that point forward, though, the entire result set is in memory, for result sets under 1MB.