Develop Reference

RocksDB IO error: lock db/LOCK: No locks available

I am developing a multithreaded server that stores and reads information from a db. The db is implemented with RocksDB.
The problem i am having is that when i access the db from more than one thread at a time, i get that error.
Usually, it ment that the db wasn't deleted after usign it, but now it doesn't make sense, of course it isn't deleted, it's being used by another thread.
I know that the db itself is accessed in secuential order, so tecnically you will never get two threads reading it at the same time, but at least one of them should wait until the lock is released and access it then
anyone has any idea what might happen?

The problem was that i had 2 rocksdb::DB* objects in different threads with the same db name and was opening the db from both threads at the same time

Setting initial values for SQLite in onCreate() of SQLiteOpenHelper

I am creating an Android app for which I need to create a SQLite DB and pre-populate it with some values.
The Android documentation says this about what to do in "onCreate" of the SQLiteOpenHelper:
Called when the database is created for the first time. This is where the creation of tables and the initial population of the tables should happen.
Reference - http://developer.android.com/reference/android/database/sqlite/SQLiteOpenHelper.html#onCreate(android.database.sqlite.SQLiteDatabase)
I am doubtful about the following 2 things -
What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
Any help is appreciated.

1) The later. It is done on the first read or write to the DB.
Your fear might be correct, this is why you can ship your app with a database that's already populated. Or you can launch an AsyncTask with a simple SELECT 1 FROM anytable query. More about shipping with DB here. (60 rows is nothing to fear about tho, and you can safely just keep using AsyncTasks).
2) Yes it is fine. The onCreate logic will run when you first read/write the DB, so it if you always use AsyncTasks onCreate will run in an AsyncTask also.

What is meant by "when database is created for the first time"? Is this done on the first launch of the app or only when the first DB request (read/write etc) is done.
It happens when you first query from database in general term. After that only Upgrade method is called that too when you change the db version.
If it is the latter, I fear that it may take quite some time to create DB, pre-populate it with values (I have about 60 rows to be inserted into 1 table) and then read the DB to show it. Is this the best practice?
60 rows insertion is not a big task. More you can read about beginTransaction(),commitTransaction and endTransaction for insertion. It will make your insertion task lighting fast.
I have been doing all my DB operations in AsyncTasks. But I am doing the table creations in onCreate using "db.execSQL" statements. Is this fine (in terms of convention/ performance) or should I go for an AsyncTask here as well?
It good you are doing you Db operation in AsyncTask and its completely fine.

Speaking of DB operations:
Performing DB operations in AsyncTask is not a good approach, generally. As you might encounter a problem called "memory leak", and it might come as a silent assassin in the night.
There's lot written on this issue. Just google "asynctask leak context" and here you go.
So how to perform DB operations?
Using Loader API in conjunction with ContentProvider is considered good approach for querying database. Loader asynchronously queries your database and delivers the result to specified subscribers. Configuration changes or other sudden stuff does not bother it.
And it is really convenient to query your data using loader API once you know how to do it.
Single inserts/updates/deletes might be done directly from the main thread via ContentResolver. These calls will be blocking (synchronous), but I bet you user would never notice anything while the amount of data is not large.
If you're operating on a large dataset, and you fear you'll be significantly blocking UI thread, I'd suggest using IntentService or any custom Service capable of doing operations in background (note that by default Service operates on main UI thread and you have to specify background operation yourself or use IntentService)
Speaking of DB initialisation:
You might create a one-time IntentService, if you're initialising a large set of data. It will handle your request asynchronously and, for example, perform a broadcast that the application is set up and ready, so you might stop a "wait a sec, performing app initialisation" screen and show user your data.
There's also nothing wrong with shipping your database along with application, though it appears to be a bit hackish solution.
Either way, you choose what is more suitable for you.

is there a way to do multi threaded sqlite transactions and check for database lock state

I have an application that is doing a LOT of sqllite transactions, I currently have a bit of a hang because I am doing the sqllite actions on the UI thread... yes bad...
so I made each item have a thread and execute on it assuming sqllite api was smart enough to FIFO them.. nope ... now I get database is locked exceptions
this says it should work
without completely rewriting my code, and having a list of transactions queue up and execute them all on the same thread (many different classes, would be kind of a pain)
is there a way for me to check, and not execute a thread unless there isnt a lock? a lock check per se, or something similar that would get this to work, is efficient and isn't a huge rewrite?
Thanks

My answer that you quoted seems to be confusing. You don't have to do anything special when you are accessing the same Android database using the same database object with multiple threads. Under the covers, Sqlite has it's own locking to guarantee that the database will not be corrupted. To quote my answer;
Sqlite under Android is single threaded. Even if multiple threads were using the same database connection, my understanding is that they would be blocked from running concurrently. There is no way to get around this limitation
It has it's own locking which serializes the requests. This means that adding multiple threads will not increase the performance of the database unfortunately.
As my other answer mentions, you cannot use multiple database objects to the same database from multiple threads since there is no locking and you will corrupt your database.

How to Execute threads in parallel and insert corresponding data in sqlite database without database lock?

I need to call multiple api's each executing in separate thread and insert the respective data from response to sqlite database without causing locks. Can anyone help me in this regard with a working example that I can refer to.

I need to call multiple api's each executing in separate thread and insert the respective data from response to sqlite database without causing locks.
You don't need to do anything special. Multiple threads can make use of the same SQLiteDatabase object without you doing locking at the application level. Sqlite does it's own locking under the covers. You should never get deadlocks but one thread will have to wait for the other thread to finish making its insert.
See these questions/answers:
Can two AsyncTasks share the same SQLiteDatabase object?
What are the best practices for SQLite on Android?
Sqlite under Android is single threaded. Even if multiple threads were using the same database connection, my understanding is that they would be blocked from running concurrently. There is no way to get around this limitation. If you open two connections to the same database, this would corrupt the database because database updates would not be coordinated.

android sqlite "database is locked" errors despite use of content provider and sequential database access

I have an app (Android 2.2 Google API Level 8) that has multiple activities pulling data from a content provider (SELECT only database access).
It also has a service with a central blocking task queue accepting any database write tasks; activities can fire a service request (As intent) which places a task on a blocking queue for sequential retrieval by a single thread and execution. Database is around 4mb.
There is a single database helper which the service uses to call methods to interact with the database including writing to it; all SQL writes are carried out within the database helper.
All database writes are surrounded by a transaction.
All database reads have the cursor closed at the end of the method.
None of the Activities has a handle to the database object, they can only communicate via the content provider or the service.
Any AlarmManager fired tasks - like Activities - only use the service to pop an appropriate task onto the queue.
The service is the only class that has a handle to the database helper.
All database writes are only carried out via a task placed on a queue; I have exhaustedly checked that task execution is sequential being well aware of it being essential to avoid concurrent writes to an SQLite database.
During a run of task executions I consistently get one or two "database is locked" errors on attempting to write to the database triggered by a tasks execution of 'begin transaction'.
In attempting to track down the source of the lock I found that using dbhelper.inTransaction(), dbhelper.isLockedByThisThread(), dbhelper.isLockedByOtherThread() didn't help as they wouldn't indicate an unexpected database lock.
What I did find that worked in detecting a lock early was to create a method with beginTransaction() and setTransactionSuccessful without any actual SQL write code, within a try catch block that would log the issue - always triggered by beginTransaction().
I placed this database lock trap either side of each of the blocking queue task methods in the expectation/hope that I would find a singular culprit that was leaving the database in a locked state after finishing.
I could not find a consistent culprit. After drilling down through from the start of the task call through to the database write I found that a database lock could occur seemingly out of the blue without having been locked by the previously run task (All these tasks run in sequence under the same singular thread).
After looking at a number of other peoples experiences with database locking issues I've tried closing the database connection directly after the transaction has completed on all tasks but this didn't help, if anything seemed to get more database locking occurrences. Tried added a sleep between each task execution; not exhaustively tested but generally found that a delay of 3 seconds or above seemed to stop the database locks appearing. Tried disabling alarm manager fired tasks - didn't make any difference.
Impression I have is that some form of maintenance task external to my application is dropping in and locking the database periodically - perhaps delayed writing of logs. Obviously I'm less than keen on setting a task processing delay so I'm considering having a database lock retry task queue to reattempt database writing as necessary; much prefer to resolve but am running out of ideas.
Can anyone think of some principle or gotcha I've missed?
Is it in reality normal within Android and larger SQLite databases that you'll get occasional database locks?
Thanks

SQLite guarantees sequential access from multiple threads as long as you use a single database connection. How and where are you opening and closing the database connection?
I generally recommend opening the database once on startup, and never closing it. There's no benefit to closing, since the transactional nature of SQLite means that writes are flushed to persistent storage as soon as possible anyway.

I would check if some activity that calls the DB or calls other activity that calls the DB, has only one instance. Otherward it can lock itself, in some sense.

With regards to
Is it in reality normal within Android and larger SQLite databases that you'll get occasional database locks?
No, it is definitely not normal to get occasional database locks. From reading your story you say that you have both a service and a content provider pulling from the database, so it is possible that you are locking the database between the two accesses.
What I generally do is ensure that I handle all of my database access through the content provider. By having a single point of entry to the database you can ensure that every software component is using the same logic to access the DB. Would it be possible to have your service access the DB through the content provider?
It's also good to remember that by placing your DB behind a content provider, it can still be accessed by multiple threads at once. To ensure that you are accessing the DB only one thread at a time you could place synchronized constructs on the DB inside of your content provider. Obviously if you are doing lots of long writes/reads to the DB, locking in this fashion will absolutely destroy your app. Putting all of your DB code inside of the content provider will also give you a single point of debugging which would help you figure out if multiple threads are accessing the DB.