I'm developing an Android application. It has multiple threads reading from and writing to the Android SQLite database. I am receiving the following error:
SQLiteException: error code 5: database is locked
I understand the SQLite locks the entire db on inserting/updating, but these errors only seem to happen when inserting/updating while I'm running a select query. The select query returns a cursor which is being left open quite a wile (a few seconds some times) while I iterate over it. If the select query is not running, I never get the locks. I'm surprised that the select could lock the db. Is this possible, or is something else going on?
What's the best way to avoid such locks?
You are probably opening and closing multiple database connections in your various threads. This is a bad idea. Just open a single database connection, and reuse it everywhere; SQLite will then ensure that concurrent accesses are serialized correctly.
As with jcwenger's answer, using a ContentProvider is another way of achieving this, but will require much more intrusive changes to your code.
By avoiding leaving cursors open for "quite a while". If you can afford to have all your data in memory all at once, then do so.
If you can't, then try increasing the busy timeout.
Migrate to a ContentProvider rather than directly accessing the DB. ContentResolver marshals away all the threading issues for you and also allows for other useful features like sharing data between apps or syncing with a server.
The api overhead of ContentResolver is minimal. You just need to define an AUTHORITY string (A unique string identifying the "kind" of your data -- use a "com.example.myapp.contacts" type of string) and use ContentResolver.bla rather than db.bla.
Its caused by beginTransaction() function.Look at your code, the problem is solved for my app to making a comment line this function(beginTransaction) line
Related
I am planning on writing an application that saves a fair amount of data. Historically, I have simply written data directly to a server, and only used some simple key/value storage with shared preferences for local storage.
I am considering this time, instead, using SQLite to save the information at first, and sync the data to the server in the background later. This will benefit the user in a few ways: 1) can use the app offline 2) don't have to worry about data being saved right away, it happens when ever it can 3) more reliability.
My approach will be to get/set data from SQLite during UI usage, and use a background process to find new rows and put them on the server, flagging them as synced when it happens.
Does this sound reasonable?
You can use SQLIte for your scenario. But, while implementing, you can follow any one of this approach.
Approach #1: Use an Abstract Factory to Instantiate the SQLiteOpenHelper.
Approach #2: Wrap the SQLiteDatabase in a ContentProvider
Refer to this link for how to implement these 2 approaches. http://www.androiddesignpatterns.com/2012/05/correctly-managing-your-sqlite-database.html
Key points to be noted while using SQLite
Sqlite takes care of the file level locking.
Many threads can read,one can write. The locks prevent more than one
writing.
Android implements some java locking in SQLiteDatabase to help keep
things straight.
If we handle the database incorrectly from many threads and mess up the code, your
database will not be corrupted. Only few updates will be lost.
How "Multiple Threads - DB access" can be used for your scenario
The SqliteOpenHelper object holds on to one database connection.
If you try to write to the database from actual distinct connections (multiple threads) at the same time, one will fail. It will not wait till the first is done and then write. It will simply not write your change. Worse, if you don’t call the right version of insert/update on the SQLiteDatabase, you won’t get an exception. You’ll just get a message in your LogCat, and that will be it.
So recommended to write using single thread and read from multiple threads if necessary for faster access.
Does this sound reasonable?
Yes. Note that the synchronization process can get tricky (e.g., what happens if the server hiccups halfway through?), but that has mostly to do with synchronization and little to do with SQLite.
We implemented a solution that used a SQLite db on the device to sync data via a web service to the master database. We did this for a couple reasons: offline, poor connection, manual sync.
For our solution we had a flag on the table that determined if the data was pushed to the web service. Our web service also provided data back to our application to let us know if the data was received and processed correctly. This allowed us to clean up the data on the device, send notifications if there were failures, and resubmit the data if there were previous failures.
You can use push notifications as well if you have fixed the issues on the backend and have the device resend the data to the web service. This worked really well for us.
In my content provider I create and maintain 3 SQLiteDatabase objects. They are created like this:
private ContentProviderHelper helper;
#Override
public boolean onCreate() { // that's the ContentProvider onCreate()
SQLiteDatabase dbLog = new DbLog(getContext()).getWritableDatabase();
SQLiteDatabase dbSession = new DbSession(getContext()).getWritableDatabase();
SQLiteDatabase dbLocation = new DbLocation(getContext()).getWritableDatabase();
helper = new ContentProviderHelper(UriManager.getAuthority(getContext()));
helper.addDb(dbLog, DbLog.TABLE_NAME, UriManager.LOG, SQLiteDatabase.CONFLICT_REPLACE);
helper.addDb(dbSession, DbSession.TABLE_NAME, UriManager.SESSION, SQLiteDatabase.CONFLICT_REPLACE);
helper.addDb(dbLocation, DbLocation.TABLE_NAME, UriManager.LOCATION, SQLiteDatabase.CONFLICT_REPLACE);
the ContentProviderHelper stores those SQLiteDatabase in an ArrayList indexed with the UriMatcher.
The <provider> is properly registered in the manifest and my app have SD-card permission.
It runs fine for most of our 500.000 user base, but every once in a while I get a SQLiteCantOpenDatabaseException: unable to open database file from the Google Play
the relevant stack track is:
Caused by: android.database.sqlite.SQLiteCantOpenDatabaseException: unable to open database file
at android.database.sqlite.SQLiteDatabase.dbopen(Native Method)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:1013)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:986)
at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:1051)
at android.app.ContextImpl.openOrCreateDatabase(ContextImpl.java:787)
at android.content.ContextWrapper.openOrCreateDatabase(ContextWrapper.java:221)
at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:157)
most often than not those error reports come from generic brandless poor quality devices.
Any help on how to properly avoid those errors will be much appreciated.
edit:
a different SQL exceptions that I'm getting on the same code.
Caused by: android.database.sqlite.SQLiteDiskIOException: disk I/O error: COMMIT;
at android.database.sqlite.SQLiteDatabase.native_execSQL(Native Method)
at android.database.sqlite.SQLiteDatabase.execSQL(SQLiteDatabase.java:1763)
at android.database.sqlite.SQLiteDatabase.endTransaction(SQLiteDatabase.java:583)
at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:137)
at ***.***.***.data.ContentManager.onCreate(ContentManager.java:26)
so is there any other way of handling this besides try{} catch(){} the hell out of it ?
edit:
Extra info regarding cursors usage:
In general cursors are kept opened for the minimum amount of time as possible. All 3 SQLiteDatabase have a Wrapper that do the actual Cursor calls, read the data from them and close them. There's only 1 instance where the cursor is passed back to the calling object, but it is an IntentService, again, single thread, single instance, single process (just like ContentProvider) that loops through the cursor and close it.
This doesn't look like a programming fault on your (Java) side, neither of the Android SQLite wrapping classes.
The provided stacktraces and code doesn't provide much information to proof my guess, but I think this should be caused by the unreliable nature of those SDCards (you already mentioned that the observed traces usually come from low quality phones).
The exception itself is generated in the native sqlite code, and while I havn't looked up the C/C++ part of the JNI interface, this should come directly from the underlying sqlite3_open call. For non obvious reasons the actuall errorcode is not included in the thrown exception, so you are basically out of luck here to find the root cause.
As this comes directly from the native layer it's some kind of filesystem/hardware problem. The card may be broken, the cardsocket may be broken/dirty/whatever or anything in between could be messed up (most likely physically).
To give you a possible solution for a problem you can't fix: Don't use the (sometimes unreliable) SDCard. If your databases are small enough (less than a few MBs) you should be fine storing them on the internal flash. This has to be reliable storage as if thouse writes or queries fail the user has more serious problems anyway.
If this is not an option, you could write "most recent data" (I don't know which kind of data you store) internally and migrate this data reguallary on the SDCard. This way you could at least try to store it without flodding your code with try/catch (if you try this in a seperate thread which catches any exception at the top level), but it doesn't solve the read part of the problem.
Another, while a little ugly, thing I can imagine: Just let the exceptions fly right through and catch them in the Application class (you can overload it), push it to disk (internal, obviously) and rethrow it to crash your app. On the next start you could then lookup if you've been crashed by an SQLite Exception and present a msg to the user which states something like "Sorry, your SDCard seems to be broken as we couldn't use it. Consider buying a new one of order a better device". ACRA may help you with this, it does what I tried to describe pretty well the last time I used it (1 year ago).
What I suppose is that you have some problem with concurrency and you are trying to get DB object more than once.
Moreover - having 3 different DB (probably) makes the application run 3 DB engines at a time (memory problems).
There is no single and simple solution while the problem is located in other place than exception is thrown, but some steps that you can do:
Put 3 databases into one file (if possible).
Make a simple singleton to keep DB object in memory and avoid concurrent attempts to the stored file. Remember that SQLite is claimed as "thread safe" not as "multithread" so the IO error can be just a natural behavior to protect DB corruption.
Make sure that cursors are closed as quick as possible (i.e. after query the object read the cursor and map data to some POJO objects.
Workaround - use try/catch while opening db, in case of exception try to do it after some time (1 second).
I am trying to understand the possible ways to work with SQLite when there can be multiple threads work on DB.
Based on various responses in stackoverflow and other sites, it appears that there will be locking issue when same sqlitehelper instance is used from multiple threads. In a typical java application, I would expect instance to mean single object of type sqlite helper to be used by different threads of application.In such cases, the locks ,I guess, are a matter of correctly using the synchronized blocks. [Correct me here as I am not comfortable with this way of looking at sqliethelper instance here]
My concern is with sharing same data base : when one instantiate sqlite helper in different threads [ie each thread has its own object instance] but working on same Database [this I guess is more inline with having same db instance].
In such cases I'm getting frequent database lock errors. This occurs even when the threads are working on different tables of database.
In my application database can be updated by user interaction through application or by getting data through server [periodic synchronization]. And some time when synchronization process and user activity overlaps, I get the lock issues. As this pattern of data processing seems to be common in application synchronizing with server, would like to know how do lock issue due the concurrency is to be handled.
I would like to understand this since if this is bound to happen always then probably need to make only one handler over database and implement queue over that to avoid lock. But that will mean the complete application needs to be aware that the database may not get updated immediately and they need to implement listener to know when the data is actually updated in database.
thanks
pradeep
As far as I know sqlite is intended for single process usage. No matter what you will always need to access the database from one thread at a time. You can do selects from multiple clients but can only write from one at a time. And other readers and writers will ahve to lock in the mean time.
As a side note - database access can hardly ever be considered instantaneous.
I would like to read some data from the webview.db database that is created when your app uses a WebView. However, that database is managed by classes that are mostly package level or do not provide much in the way of access through apis.
Since I can't share their database objects or locks, how can I safely read content from that database without getting "database is locked" errors or similar exceptions?
I only need to read, I do not need to write.
If there is no way to safely aquire sql queries/cursors on it, is there a way to safely copy the actual webview.db file in a thread safe manner? Is there any danger of getting a corrupt copy?
Thanks much
I think all the locks are at the sqlite level, not in the java code, so you should be able to get away with just opening the db and reading from it without confusing the webview.
However, for the same reason you should be prepared for lock errors and retry your queries until the lock is gone. The db will be locked when the webview does data altering statements.
Hopefully the webview doesn't put an exclusive lock on all the db for the whole time it's running, but i don't see a reason for that.
I have a an application that has 2 parts.
A service which creates content.
An application that uses the content
Each of these run as different processes. The problem is that both of them share a database. And I frequently get database locked error, both when the service tries to write something and the UI is reading data. Also vice versa.
How do go about this?
The class used to access DB is a singleton class. But since both UI & the service are 2 different processes, there are 2 singletons I presume. So that doesn't help.
Even synchronise won't help I suppose, since again because of 2 different processes.
Content Providers maybe an option, but since I use complex queries to dig info, it would be really hard to use that too.
How do I get the two processes share the database.
Any cues would be greatly appreciated.
Using a content provider is one option. Another is to take a look at Berkeley DB. The BDB SQL API is SQLite compatible and the BDB lock manager allows multiple threads and/or processes to read/write to the database concurrently.
close the connection after each operation
catch the database locked error and try to reconnect after 50ms
or let the service handle the database and the activity ask the service for data
may be there is isDatabaseInUseMethod ?
You should use a content provider to funnel your database queries through one source. Inside of the content provider you can use any locking mechanisms you would like to ensure you're not having concurrent access. You may also think about using content observers to coordinate service actions with changes to the database.
The following is a great article on how locking works with SQLite on Android and what things to be aware of: http://kagii.squarespace.com/journal/2010/9/10/android-sqlite-locking.html
I would think you'll find some answers there :)