My code:
SQLiteDatabase db = ...;
db.beginTransaction();
try{
db.update(...);
db.setTransactionSuccessful();
}finally{
db.endTransaction();
}
Now the problem is that endTransaction occasionally throws SQLiteDatabaseLockedException, and I don't know reason, or how to repeat same exception.
From SQLiteDatabaseLockedException I read:
Thrown if the database engine was unable to acquire the database locks
it needs to do its job.
And from beginTransaction I read:
Begins a transaction in EXCLUSIVE mode.
From SQLite manual I read:
An EXCLUSIVE lock is needed in order to write to the database file.
Only one EXCLUSIVE lock is allowed on the file and no other locks of
any kind are allowed to coexist with an EXCLUSIVE lock. In order to
maximize concurrency, SQLite works to minimize the amount of time that
EXCLUSIVE locks are held.
So how can DB lock not be acquired in endTransaction when I hold exclusive lock from beginTransaction?
Android version where this happens is 4.0.4 (I have crash report, but not able to repeat this).
Need to say that I enabled SQLiteDatabase.enableWriteAheadLogging on the DB, maybe it matters? My app accessess DB in multiple threads.
Anyway, I'd like to get clear explanation, and make simple example that can repeat conditions repeating the problem, so that I can make real fix.
Thanks.
IMO your code is correct in a singular thread app, so it must be an enableWriteAheadLogging related issue. Maybe this can help:
SOURCE
... The maximum number of connections used to execute queries in
parallel is dependent upon the device memory and possibly other
properties.
...
Writers should use beginTransactionNonExclusive() or
beginTransactionWithListenerNonExclusive(SQLiteTransactionListener) to
start a transaction. Non-exclusive mode allows database file to be in
readable by other threads executing queries.
...
As I understand in serializable (default) mode of sqlite the locks refer not to different threads but to connections (this mode even knows nothing about threads). So if you are using the same connection (and all SqliteDatabase objects, produced by one sqliteOpenHelper instance share the same connection) across multiple threads you are totally unprotected.
Use either connection per thread or non-database synchronization locks, if you insist on multithreaded use of Sqlite. I would prefer a wrapper singleton with a single thread lock protecting its every transaction-like method. But this depends on your app specifics.
Please read this extensive answer and links in it for details and best practices for Sqlite multithreading.
Related
I am very much confused regarding access to SQLiteDatabase. Either it should be one connection or multiple connections to have access from multiple threads. I have read many articles including following two.
https://stackoverflow.com/a/3689883/3027124
http://touchlabblog.tumblr.com/post/24474398246/android-sqlite-locking
Both of these recommends to have single connection. Even my own answer to same question was accepted by OP. I used the Singleton approach to access SQLiteopenHelper class.
https://stackoverflow.com/a/35358702/3027124
But I am still confused after reading documentation of enableWriteAheadLogging which states
This method enables parallel execution of queries from multiple
threads on the same database. It does this by opening multiple
connections to the database and using a different database connection
for each query. The database journal mode is also changed to enable
writes to proceed concurrently with reads.
Now this is the confusing part. If I want to access database from simultaneous multiple threads, I have Singleton access to SQLiteOpenHelper which in my understanding means the serial execution of insertions while Simultaneous reads can be done with no-errors. But above documentation says, in order to have Simultaneous access, enableWriteAheadLogging should be called which in returns create multiple connections. What is this going on?? What does it mean if I do insertions by calling getWritableDatabase() using Singleton SQLiteOpenHelper from multiple threads? Will the calls be serial? Should enableWriteAheadLogging be called?
Please clarify.
I would use the singleton instance whether I'm using enableWriteAheadLogging or not when dealing with threading which is the case in most apps unless it's a very trivial app like a sample.
Using a singleton instance is ensuring Thread Safety: The singleton instance ensures that synchronization works across that instance, that means when you have a read and write methods calling the database at the same time from different threads, one of them should wait for the other as the database gets locked while writing to it.
It's really clear that this is the case as written in the documentation and quoted below
it is not
possible for reads and writes to occur on the database at the same
time. Before modifying the database, the writer implicitly acquires an
exclusive lock on the database which prevents readers from accessing
the database until the write is completed.
enableWriteAheadLogging is actually altering the above behavior as the above statement is only true When write-ahead logging is not enabled (the default).
So what happens when you enable write-ahead logging through enableWriteAheadLogging?
It's actually changing the default behavior enabling actual parralism as it changes the underlying database journal file to enable doing a write and a read at the same time, but to do that it needs more memory than usual. Read the documentation quote below to know more!
In contrast, when write-ahead logging is enabled (by calling this
method), write operations occur in a separate log file which allows
reads to proceed concurrently. While a write is in progress, readers
on other threads will perceive the state of the database as it was
before the write began. When the write completes, readers on other
threads will then perceive the new state of the database.
It is a good idea to enable write-ahead logging whenever a database
will be concurrently accessed and modified by multiple threads at the
same time. However, write-ahead logging uses significantly more memory
than ordinary journaling because there are multiple connections to the
same database. So if a database will only be used by a single thread,
or if optimizing concurrency is not very important, then write-ahead
logging should be disabled.
I have several lists that I want to store in my sqlite db in a multithreading fashion using activeAndroid. I run a thread for each list to persist.
The body of each thread look like this.
ActiveAndroid.beginTransaction();
try {
for (MyObjToPersist e : myListOfObjToPersist){
e.save();
}
ActiveAndroid.setTransactionSuccessful();
}
finally {
ActiveAndroid.endTransaction();
}
The transaction seems to add a lock to the db since each thread run one after the other.
Without using transaction things work as expected but the tasks are pretty slow (a hundred time slowest).
This subject are discuss here.
Anybody know how to avoid this behaviour?
Thanks
TL;DR: Only one thread can make modification, but all can read the data.
Can multiple applications or multiple instances of the same application access a single database file at the same time?
Multiple processes can have the same database open at the same time. Multiple processes can be doing a SELECT at the same time. But only one process can be making changes to the database at any moment in time, however.
SQLite uses reader/writer locks to control access to the database. (Under Win95/98/ME which lacks support for reader/writer locks, a probabilistic simulation is used instead.) But use caution: this locking mechanism might not work correctly if the database file is kept on an NFS filesystem. This is because fcntl() file locking is broken on many NFS implementations. You should avoid putting SQLite database files on NFS if multiple processes might try to access the file at the same time. On Windows, Microsoft's documentation says that locking may not work under FAT filesystems if you are not running the Share.exe daemon. People who have a lot of experience with Windows tell me that file locking of network files is very buggy and is not dependable. If what they say is true, sharing an SQLite database between two or more Windows machines might cause unexpected problems.
We are aware of no other embedded SQL database engine that supports as much concurrency as SQLite. SQLite allows multiple processes to have the database file open at once, and for multiple processes to read the database at once. When any process wants to write, it must lock the entire database file for the duration of its update. But that normally only takes a few milliseconds. Other processes just wait on the writer to finish then continue about their business. Other embedded SQL database engines typically only allow a single process to connect to the database at once.
However, client/server database engines (such as PostgreSQL, MySQL, or Oracle) usually support a higher level of concurrency and allow multiple processes to be writing to the same database at the same time. This is possible in a client/server database because there is always a single well-controlled server process available to coordinate access. If your application has a need for a lot of concurrency, then you should consider using a client/server database.
But experience suggests that most applications need much less
concurrency than their designers imagine.
When SQLite tries to access a file that is locked by another process, the default behavior is to return SQLITE_BUSY. You can adjust this behavior from C code using the sqlite3_busy_handler() or sqlite3_busy_timeout() API functions.
https://www.sqlite.org/faq.html#q5
I have an application that is doing a LOT of sqllite transactions, I currently have a bit of a hang because I am doing the sqllite actions on the UI thread... yes bad...
so I made each item have a thread and execute on it assuming sqllite api was smart enough to FIFO them.. nope ... now I get database is locked exceptions
this says it should work
without completely rewriting my code, and having a list of transactions queue up and execute them all on the same thread (many different classes, would be kind of a pain)
is there a way for me to check, and not execute a thread unless there isnt a lock? a lock check per se, or something similar that would get this to work, is efficient and isn't a huge rewrite?
Thanks
My answer that you quoted seems to be confusing. You don't have to do anything special when you are accessing the same Android database using the same database object with multiple threads. Under the covers, Sqlite has it's own locking to guarantee that the database will not be corrupted. To quote my answer;
Sqlite under Android is single threaded. Even if multiple threads were using the same database connection, my understanding is that they would be blocked from running concurrently. There is no way to get around this limitation
It has it's own locking which serializes the requests. This means that adding multiple threads will not increase the performance of the database unfortunately.
As my other answer mentions, you cannot use multiple database objects to the same database from multiple threads since there is no locking and you will corrupt your database.
I've read a lot of topics, but can't figure out answer for question: is it possible to read and write simultaneous?
I have background thread that updates some data and UI needs small piece of data stored in DB. So in UI thread SELECT operation is performed. But it blocks when update is in progress. As result, UI freezes for several seconds.
Does anyone has success in reading from DB when writing?
Its possible to read and write to DB on iPhone. Does the reason of such difference is in synchronious implementation of wrapper on native sqlite functions?
On Android 3.0 and higher SQLiteDatabases support WAL mode (write-ahead logging):
When write-ahead logging is not enabled (the default), it is not
possible for reads and writes to occur on the database at the same
time. Before modifying the database, the writer implicitly acquires an
exclusive lock on the database which prevents readers from accessing
the database until the write is completed.
In contrast, when write-ahead logging is enabled, write operations
occur in a separate log file which allows reads to proceed
concurrently. While a write is in progress, readers on other threads
will perceive the state of the database as it was before the write
began. When the write completes, readers on other threads will then
perceive the new state of the database.
http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html#enableWriteAheadLogging()
To start a transaction in WAL mode use beginTransactionNonExclusive() instead of beginTransaction().
While beginTransaction() starts a transaction in EXCLUSIVE mode, beginTransactionNonExclusive() starts one in IMMEDIATE mode
EXCLUSIVE mode uses exclusive locks (http://www.sqlite.org/lockingv3.html#excl_lock) meaning no other database connection except for read_uncommitted connections will be able to read the database and no other connection without exception will be able to write the database until the transaction is complete
IMMEDIATE mode uses reserved locks (http://www.sqlite.org/lockingv3.html#reserved_lock) meaning no other database connection will be able to write to the database or do a BEGIN IMMEDIATE or BEGIN EXCLUSIVE, other processes can continue to read from the database, however.
In simpler words: call beginTransactionNonExclusive() for IMMEDIATE mode and we can read while another thread is writing (the state before the write transaction started because we won't use read_uncommitted connections -> http://en.wikipedia.org/wiki/Isolation_%28database_systems%29#Dirty_reads).
Starting with API 11 Android has support for WAL mode. It keeps original data untouched during transaction, so other threads can read when transaction is running. You can check my article for more details about WAL mode:
http://www.skoumal.net/en/parallel-read-and-write-in-sqlite/
You should also avoid running database queries in UI thread. It could always become sluggish and block your UI.
You cannot read and write at the same time. SQLite is a serverless, file-based database.
From the SQLite FAQ:
"When any process wants to write, it must lock the entire database file for the duration of its update. But that normally only takes a few milliseconds. Other processes just wait on the writer to finish then continue about their business. Other embedded SQL database engines typically only allow a single process to connect to the database at once."
It is not possible to read & write simultaneously. However, if you have your SQLite database classes set up correctly (single instance of your DB & helper classes), different threads should be able to grab the DB connection synchronously so that there is not any noticeable lag.
It also sounds like you're trying to do backend work (write to the db) with your UI thread. You should not be doing this. Create an AsyncTask to handle this instead of having your UI thread handle it.
Refer to the SQLiteOpenHelper documentation. Here's a previous post that talks about this as well: What are the best practices for SQLite on Android?
I'm developing an Android application. It has multiple threads reading from and writing to the Android SQLite database. I am receiving the following error:
SQLiteException: error code 5: database is locked
I understand the SQLite locks the entire db on inserting/updating, but these errors only seem to happen when inserting/updating while I'm running a select query. The select query returns a cursor which is being left open quite a wile (a few seconds some times) while I iterate over it. If the select query is not running, I never get the locks. I'm surprised that the select could lock the db. Is this possible, or is something else going on?
What's the best way to avoid such locks?
You are probably opening and closing multiple database connections in your various threads. This is a bad idea. Just open a single database connection, and reuse it everywhere; SQLite will then ensure that concurrent accesses are serialized correctly.
As with jcwenger's answer, using a ContentProvider is another way of achieving this, but will require much more intrusive changes to your code.
By avoiding leaving cursors open for "quite a while". If you can afford to have all your data in memory all at once, then do so.
If you can't, then try increasing the busy timeout.
Migrate to a ContentProvider rather than directly accessing the DB. ContentResolver marshals away all the threading issues for you and also allows for other useful features like sharing data between apps or syncing with a server.
The api overhead of ContentResolver is minimal. You just need to define an AUTHORITY string (A unique string identifying the "kind" of your data -- use a "com.example.myapp.contacts" type of string) and use ContentResolver.bla rather than db.bla.
Its caused by beginTransaction() function.Look at your code, the problem is solved for my app to making a comment line this function(beginTransaction) line