I have an application that I can insert data into a SQLite database. There's a service thread running every sixty seconds checking if a condition is true and raising an alarm if necessary. Think "calendar" application.
I have lots of activities with a reference to their own SQLiteOpenHelper, and their own SQLiteDatabase object.
The application was working pretty well, but one of the main update applications I decided needed a progress dialog subject - that's another subject, but if you can tell me how I can get the spinner to spin that'd be great. It does however display.
But to get it to display, I needed to put it in a thread.
Creating this thread, I started to get loads of "database is locked" messages in the Log.
I figured the issue may have been related to the fact I had a task in the background running every sixty seconds - even though it visually only took a couple of seconds to do the update, so I placed the main write transactions for the updater function in a beginTransaction block. That's the only place I used one.
Everything seemed good, but then I started getting more spurious errors about the database being locked all over the place.
It's encouraged me to do some tidying up, but it seems after I do the first beginTransaction, any further database modifications I want to make fail with a report that the database is locked, despite that fact I do call db.close. I even close the database helper object and cursor just to be on the safe side.
I was able to clear up most of the warnings with regards to cursors still being open, eg
E/Database( 678): android.database.sqlite.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here
I couldn't figure out what was locking the database as everything was being closed.
I then decided to remove the "beginTransaction" which stabilised things - I could now navigate my program again. But Im still getting a few random locking issues.
E/Database( 1304): Failure 5 (database is locked) on 0x26e5d8 when executing 'INSERT INTO
Is it possible to figure out what is locking the database? I can see what is grumbling about the lock, but not what is locking it.
What is the best way to handle concurrent database updates/reads like this? I've read a lot about synchronized and ContentProviders but I must admit, it is a little bit big for me.
Any pointers on the "best" way of doing things?
And can I find out what is locking the database?
Thanks
Simon
class NAME extends SQLiteOpenHelper
add:
#Override
public void onOpen(SQLiteDatabase db) {
super.onOpen(db);
if (!db.isReadOnly()) {
db.execSQL("PRAGMA foreign_keys=ON;");
Cursor c = db.rawQuery("PRAGMA foreign_keys", null);
if (c.moveToFirst()) {
int result = c.getInt(0);
}
if (!c.isClosed()) {
c.close();
}
}
}
You may want to look at the latest version of Berkeley DB as an option. It provides a SQLite3-compatible SQL API (so that your application can stay the same) and it provides a finer granularity of locking and therefore better concurrency. Berkeley DB supports multiple concurrent read and write threads to actively access the database at the same time. You can find out additional information about Berkeley DB on our website.
Related
I am creating an Android app for which I need to create a SQLite DB and pre-populate it with some values.
The Android documentation says this about what to do in "onCreate" of the SQLiteOpenHelper:
Called when the database is created for the first time. This is where the creation of tables and the initial population of the tables should happen.
Reference - http://developer.android.com/reference/android/database/sqlite/SQLiteOpenHelper.html#onCreate(android.database.sqlite.SQLiteDatabase)
I am doubtful about the following 2 things -
What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
Any help is appreciated.
1) The later. It is done on the first read or write to the DB.
Your fear might be correct, this is why you can ship your app with a database that's already populated. Or you can launch an AsyncTask with a simple SELECT 1 FROM anytable query. More about shipping with DB here. (60 rows is nothing to fear about tho, and you can safely just keep using AsyncTasks).
2) Yes it is fine. The onCreate logic will run when you first read/write the DB, so it if you always use AsyncTasks onCreate will run in an AsyncTask also.
What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
It happens when you first query from database in general term. After that only Upgrade method is called that too when you change the db version.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
60 rows insertion is not a big task. More you can read about beginTransaction(),commitTransaction and endTransaction for insertion. It will make your insertion task lighting fast.
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
It good you are doing you Db operation in AsyncTask and its completely fine.
Speaking of DB operations:
Performing DB operations in AsyncTask is not a good approach, generally. As you might encounter a problem called "memory leak", and it might come as a silent assassin in the night.
There's lot written on this issue. Just google "asynctask leak context" and here you go.
So how to perform DB operations?
Using Loader API in conjunction with ContentProvider is considered good approach for querying database. Loader asynchronously queries your database and delivers the result to specified subscribers. Configuration changes or other sudden stuff does not bother it.
And it is really convenient to query your data using loader API once you know how to do it.
Single inserts/updates/deletes might be done directly from the main thread via ContentResolver. These calls will be blocking (synchronous), but I bet you user would never notice anything while the amount of data is not large.
If you're operating on a large dataset, and you fear you'll be significantly blocking UI thread, I'd suggest using IntentService or any custom Service capable of doing operations in background (note that by default Service operates on main UI thread and you have to specify background operation yourself or use IntentService)
Speaking of DB initialisation:
You might create a one-time IntentService, if you're initialising a large set of data. It will handle your request asynchronously and, for example, perform a broadcast that the application is set up and ready, so you might stop a "wait a sec, performing app initialisation" screen and show user your data.
There's also nothing wrong with shipping your database along with application, though it appears to be a bit hackish solution.
Either way, you choose what is more suitable for you.
I have an Android Application that holds a fairly large Database which is accessed from multiple threads. I am starting to get Database locked Exceptions. My question, is there a way of creating a database Queue (similar to FMDataBase.Queue in iOS)?
I have done some research and I don't want to create a helper class as a lot of my database queries and inserts are different so creating a method for each query is not feasible.
I could put
if(!db.isLocked)
{
//exec(BLAH)
}
else
{
//try{thread.sleep(1000);}... then some retry code
}
on every database function but there must be a better way than this.
moreover if the thread is locked after is has slept for 1000ms, it will still crash and if the db is locked for 10ms I will be waiting for 990ms for the query to run, which is not great for user experience.
Is is possible to create a Queue so that any commands sent to the db will be executed once the the db becomes unlocked?
Any advice would be greatly appreciated
My question, is there a way of creating a database Queue
Move all your database writes to an IntentService, a regular service hosting a LinkedBlockingQueue, a LinkedBlockingQueue held by your singleton SQLiteOpenHelper, etc.
a lot of my database queries and inserts are different so creating a method for each query is not feasible.
The overhead to wrap a database I/O in a method not especially much, less than what you were proposing doing in your question.
if the db is locked for 10ms I will be waiting for 990ms for the query to run, which is not great for user experience
Tune your database access (e.g., EXPLAIN keyword, then set up appropriate indices). Use Traceview to determine exactly where your problems lie. If your problems are in a database transaction (e.g., you open your own transaction and do a ton of work inside of it), consider using yieldIfContendedSafely().
My app runs some very complicated queries using joins and other things which the Android ContentResolver is not equipped to handle. Some of these queries, inserts, etc can block the UI thread, so I run them in the background. As you might guess, this can cause a SQLiteException (database locked) when the timing of two sql requests coincide with one another (the most common cause of this is data loading into my db from our server in the background while the user requests to view data from the db on the screen).
The best way to fix this issue is to keep all DB activity on one thread, but like I said above some inserts in particular would cause some big time UI thread blockage, so that's not really realistic. The other option I've seen is synchronizing all the methods of the ContentResolver (they all should share a common member variable like a DatabaseHelper). I've done this, but again, this is not enough, because for some methods I have to use the SQLiteDatabase object's rawQuery() method to pull off the sql I need to get the data.
Is anyone aware of a strategy I could use to synchronize both ContentResolver methods and SQLiteDatabase methods? Thanks a lot!
I would make the long inserts in a transaction
db.beginTransaction();
try {
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
If you don't want that the requests that a user makes during the insert returns empty content, implement a logic of retry.
Would not this work ?
I was looking for a solution for making my slqite database, accessing syncronized and avoid problem with open/close connection, and I falled in this good answer:
In this solution, however, there is not a explicit closing of database.
many argue that this is not a problem, but I would know how can I close the connection after the application is stopped.
note: In the aforementioned link the author of the answer suggest this:
With my implementation the singleton lives for the life of the
application. Therefore I did not implement an explicit destructor,
instead, in the app shutdown code (or crash handler) the database is
explicitly closed. See Thread.setDefaultUncaughtExceptionHandler()
which I used in ApplicationContext.onCreate() after I posted my sample
code.
Can be this a good solution? If yes, how can I manage Thread.setDefaultUncaughtExceptionHandler() ?
EDIT:
I want also, that if the database resource is not available, the requiring thread will wait until the database is released. Can I use this code? the isDbLockedByOtherThreads() is deprecated
private myDatabase(){
final DbHelper mDbHelper =new DbHelper(ApplicationFramework.getContext());
mDB = mDbHelper.getWritableDatabase();
while(mDB.isDbLockedByCurrentThread() || mDB.isDbLockedByOtherThreads()) {
//db is locked, keep looping
}
}
You need not to worry about closing database connection after the application is stopped it automatically gets closed.
Actually there is only one connection internally, you make its different instance thats a different thing. Thats why many people says closing of database connection does'nt make any sense.
Always have a single static object of database connection it will be safe and less exception prone, and don't worry about closing it.
If you try to write to the database from actual distinct connections at the same time, one will deifnately get failed . It will not wait till the first is done and then write. It will simply not write your change. Worse, if you don’t call the right version of insert/update on the SQLiteDatabase, you won’t get an exception. You’ll just get a message in your LogCat, and that will be it.
Answer for updated query
Use only Single static instance of your db connection no need to make others threads, all operations will be automatically synchronised.
I'm developing an Android application. It has multiple threads reading from and writing to the Android SQLite database. I am receiving the following error:
SQLiteException: error code 5: database is locked
I understand the SQLite locks the entire db on inserting/updating, but these errors only seem to happen when inserting/updating while I'm running a select query. The select query returns a cursor which is being left open quite a wile (a few seconds some times) while I iterate over it. If the select query is not running, I never get the locks. I'm surprised that the select could lock the db. Is this possible, or is something else going on?
What's the best way to avoid such locks?
You are probably opening and closing multiple database connections in your various threads. This is a bad idea. Just open a single database connection, and reuse it everywhere; SQLite will then ensure that concurrent accesses are serialized correctly.
As with jcwenger's answer, using a ContentProvider is another way of achieving this, but will require much more intrusive changes to your code.
By avoiding leaving cursors open for "quite a while". If you can afford to have all your data in memory all at once, then do so.
If you can't, then try increasing the busy timeout.
Migrate to a ContentProvider rather than directly accessing the DB. ContentResolver marshals away all the threading issues for you and also allows for other useful features like sharing data between apps or syncing with a server.
The api overhead of ContentResolver is minimal. You just need to define an AUTHORITY string (A unique string identifying the "kind" of your data -- use a "com.example.myapp.contacts" type of string) and use ContentResolver.bla rather than db.bla.
Its caused by beginTransaction() function.Look at your code, the problem is solved for my app to making a comment line this function(beginTransaction) line