Recently I started using room database in my android app. I am getting some problems when trying to access database from multiple threads. I am using same instance of database in all threads.
As far I know, if all threads have same instance of database then
database access is serialized. I read in a blog that in serialized
mode only one thread can read and write in database. However
according to sqlite docs, read enables shared lock hence multiple
thread can read simultaneously. So when using single instance of db,
default locking criteria of sqlite which allows multiple read and
one write operation is followed or not?
According to sqlite docs, while writing to database first reserved lock is enabled and if any other write operation tries to get
Reserved lock then write attempt fails and db returns SQLITE_BUSY.
But if I am trying to run two write operation consequently from
different threads I am never getting this error. Does it mean that
write operation are queued and It's guaranteed that we will never
get SQLITE_BUSY error, hence write operation will never fail?
I am doing a long insert operation(#Insert) in one thread(Thread1) and reading database in another thread(Thread2). If
I start Thread2 just after Thread1, Read operation does not return the new
inserted data in Thread1, so reading is happening before insertion.
Is it happening because initially write enables Reserved lock and
during this time new shared lock can be acquired #sqlite
docs.
Sorry for a long question.
Related
I am developing a multithreaded server that stores and reads information from a db. The db is implemented with RocksDB.
The problem i am having is that when i access the db from more than one thread at a time, i get that error.
Usually, it ment that the db wasn't deleted after usign it, but now it doesn't make sense, of course it isn't deleted, it's being used by another thread.
I know that the db itself is accessed in secuential order, so tecnically you will never get two threads reading it at the same time, but at least one of them should wait until the lock is released and access it then
anyone has any idea what might happen?
The problem was that i had 2 rocksdb::DB* objects in different threads with the same db name and was opening the db from both threads at the same time
I need to call multiple api's each executing in separate thread and insert the respective data from response to sqlite database without causing locks. Can anyone help me in this regard with a working example that I can refer to.
I need to call multiple api's each executing in separate thread and insert the respective data from response to sqlite database without causing locks.
You don't need to do anything special. Multiple threads can make use of the same SQLiteDatabase object without you doing locking at the application level. Sqlite does it's own locking under the covers. You should never get deadlocks but one thread will have to wait for the other thread to finish making its insert.
See these questions/answers:
Can two AsyncTasks share the same SQLiteDatabase object?
What are the best practices for SQLite on Android?
Sqlite under Android is single threaded. Even if multiple threads were using the same database connection, my understanding is that they would be blocked from running concurrently. There is no way to get around this limitation. If you open two connections to the same database, this would corrupt the database because database updates would not be coordinated.
I am trying to understand the possible ways to work with SQLite when there can be multiple threads work on DB.
Based on various responses in stackoverflow and other sites, it appears that there will be locking issue when same sqlitehelper instance is used from multiple threads. In a typical java application, I would expect instance to mean single object of type sqlite helper to be used by different threads of application.In such cases, the locks ,I guess, are a matter of correctly using the synchronized blocks. [Correct me here as I am not comfortable with this way of looking at sqliethelper instance here]
My concern is with sharing same data base : when one instantiate sqlite helper in different threads [ie each thread has its own object instance] but working on same Database [this I guess is more inline with having same db instance].
In such cases I'm getting frequent database lock errors. This occurs even when the threads are working on different tables of database.
In my application database can be updated by user interaction through application or by getting data through server [periodic synchronization]. And some time when synchronization process and user activity overlaps, I get the lock issues. As this pattern of data processing seems to be common in application synchronizing with server, would like to know how do lock issue due the concurrency is to be handled.
I would like to understand this since if this is bound to happen always then probably need to make only one handler over database and implement queue over that to avoid lock. But that will mean the complete application needs to be aware that the database may not get updated immediately and they need to implement listener to know when the data is actually updated in database.
thanks
pradeep
As far as I know sqlite is intended for single process usage. No matter what you will always need to access the database from one thread at a time. You can do selects from multiple clients but can only write from one at a time. And other readers and writers will ahve to lock in the mean time.
As a side note - database access can hardly ever be considered instantaneous.
I've read a lot of topics, but can't figure out answer for question: is it possible to read and write simultaneous?
I have background thread that updates some data and UI needs small piece of data stored in DB. So in UI thread SELECT operation is performed. But it blocks when update is in progress. As result, UI freezes for several seconds.
Does anyone has success in reading from DB when writing?
Its possible to read and write to DB on iPhone. Does the reason of such difference is in synchronious implementation of wrapper on native sqlite functions?
On Android 3.0 and higher SQLiteDatabases support WAL mode (write-ahead logging):
When write-ahead logging is not enabled (the default), it is not
possible for reads and writes to occur on the database at the same
time. Before modifying the database, the writer implicitly acquires an
exclusive lock on the database which prevents readers from accessing
the database until the write is completed.
In contrast, when write-ahead logging is enabled, write operations
occur in a separate log file which allows reads to proceed
concurrently. While a write is in progress, readers on other threads
will perceive the state of the database as it was before the write
began. When the write completes, readers on other threads will then
perceive the new state of the database.
http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html#enableWriteAheadLogging()
To start a transaction in WAL mode use beginTransactionNonExclusive() instead of beginTransaction().
While beginTransaction() starts a transaction in EXCLUSIVE mode, beginTransactionNonExclusive() starts one in IMMEDIATE mode
EXCLUSIVE mode uses exclusive locks (http://www.sqlite.org/lockingv3.html#excl_lock) meaning no other database connection except for read_uncommitted connections will be able to read the database and no other connection without exception will be able to write the database until the transaction is complete
IMMEDIATE mode uses reserved locks (http://www.sqlite.org/lockingv3.html#reserved_lock) meaning no other database connection will be able to write to the database or do a BEGIN IMMEDIATE or BEGIN EXCLUSIVE, other processes can continue to read from the database, however.
In simpler words: call beginTransactionNonExclusive() for IMMEDIATE mode and we can read while another thread is writing (the state before the write transaction started because we won't use read_uncommitted connections -> http://en.wikipedia.org/wiki/Isolation_%28database_systems%29#Dirty_reads).
Starting with API 11 Android has support for WAL mode. It keeps original data untouched during transaction, so other threads can read when transaction is running. You can check my article for more details about WAL mode:
http://www.skoumal.net/en/parallel-read-and-write-in-sqlite/
You should also avoid running database queries in UI thread. It could always become sluggish and block your UI.
You cannot read and write at the same time. SQLite is a serverless, file-based database.
From the SQLite FAQ:
"When any process wants to write, it must lock the entire database file for the duration of its update. But that normally only takes a few milliseconds. Other processes just wait on the writer to finish then continue about their business. Other embedded SQL database engines typically only allow a single process to connect to the database at once."
It is not possible to read & write simultaneously. However, if you have your SQLite database classes set up correctly (single instance of your DB & helper classes), different threads should be able to grab the DB connection synchronously so that there is not any noticeable lag.
It also sounds like you're trying to do backend work (write to the db) with your UI thread. You should not be doing this. Create an AsyncTask to handle this instead of having your UI thread handle it.
Refer to the SQLiteOpenHelper documentation. Here's a previous post that talks about this as well: What are the best practices for SQLite on Android?
I have an app (Android 2.2 Google API Level 8) that has multiple activities pulling data from a content provider (SELECT only database access).
It also has a service with a central blocking task queue accepting any database write tasks; activities can fire a service request (As intent) which places a task on a blocking queue for sequential retrieval by a single thread and execution. Database is around 4mb.
There is a single database helper which the service uses to call methods to interact with the database including writing to it; all SQL writes are carried out within the database helper.
All database writes are surrounded by a transaction.
All database reads have the cursor closed at the end of the method.
None of the Activities has a handle to the database object, they can only communicate via the content provider or the service.
Any AlarmManager fired tasks - like Activities - only use the service to pop an appropriate task onto the queue.
The service is the only class that has a handle to the database helper.
All database writes are only carried out via a task placed on a queue; I have exhaustedly checked that task execution is sequential being well aware of it being essential to avoid concurrent writes to an SQLite database.
During a run of task executions I consistently get one or two "database is locked" errors on attempting to write to the database triggered by a tasks execution of 'begin transaction'.
In attempting to track down the source of the lock I found that using dbhelper.inTransaction(), dbhelper.isLockedByThisThread(), dbhelper.isLockedByOtherThread() didn't help as they wouldn't indicate an unexpected database lock.
What I did find that worked in detecting a lock early was to create a method with beginTransaction() and setTransactionSuccessful without any actual SQL write code, within a try catch block that would log the issue - always triggered by beginTransaction().
I placed this database lock trap either side of each of the blocking queue task methods in the expectation/hope that I would find a singular culprit that was leaving the database in a locked state after finishing.
I could not find a consistent culprit. After drilling down through from the start of the task call through to the database write I found that a database lock could occur seemingly out of the blue without having been locked by the previously run task (All these tasks run in sequence under the same singular thread).
After looking at a number of other peoples experiences with database locking issues I've tried closing the database connection directly after the transaction has completed on all tasks but this didn't help, if anything seemed to get more database locking occurrences. Tried added a sleep between each task execution; not exhaustively tested but generally found that a delay of 3 seconds or above seemed to stop the database locks appearing. Tried disabling alarm manager fired tasks - didn't make any difference.
Impression I have is that some form of maintenance task external to my application is dropping in and locking the database periodically - perhaps delayed writing of logs. Obviously I'm less than keen on setting a task processing delay so I'm considering having a database lock retry task queue to reattempt database writing as necessary; much prefer to resolve but am running out of ideas.
Can anyone think of some principle or gotcha I've missed?
Is it in reality normal within Android and larger SQLite databases that you'll get occasional database locks?
Thanks
SQLite guarantees sequential access from multiple threads as long as you use a single database connection. How and where are you opening and closing the database connection?
I generally recommend opening the database once on startup, and never closing it. There's no benefit to closing, since the transactional nature of SQLite means that writes are flushed to persistent storage as soon as possible anyway.
I would check if some activity that calls the DB or calls other activity that calls the DB, has only one instance. Otherward it can lock itself, in some sense.
With regards to
Is it in reality normal within Android and larger SQLite databases that you'll get occasional database locks?
No, it is definitely not normal to get occasional database locks. From reading your story you say that you have both a service and a content provider pulling from the database, so it is possible that you are locking the database between the two accesses.
What I generally do is ensure that I handle all of my database access through the content provider. By having a single point of entry to the database you can ensure that every software component is using the same logic to access the DB. Would it be possible to have your service access the DB through the content provider?
It's also good to remember that by placing your DB behind a content provider, it can still be accessed by multiple threads at once. To ensure that you are accessing the DB only one thread at a time you could place synchronized constructs on the DB inside of your content provider. Obviously if you are doing lots of long writes/reads to the DB, locking in this fashion will absolutely destroy your app. Putting all of your DB code inside of the content provider will also give you a single point of debugging which would help you figure out if multiple threads are accessing the DB.