In my application, there exist more than one process, and in each process, I need access the same SQLite database (of course, it means more than 2 theads), so I worried about not only the thread-safety about the SQLite, but also the process-safety.
One solution for this case is using content-provider. But from android sdk, it warns that its methods may be called from multiple threads and therefore must be thread-safe. If content provider itself not necessarily means thread-safe, how can I assume it is process-safe?
The article also clarifies that SQLiteDatabase itself is synchronized by default, thus guaranteeing that no two threads will ever touch it at the same time. What if in the multi-process case? Can two processes modify the same table at the same time? Will it crash?
Multiple processes behave just like multiple threads, i.e., their transactions are safe from being interfered with by each other.
You can find answer here: https://www.sqlite.org/faq.html (point 5). Briefly:
Multiple processes can have the same database open at the same time. Multiple
processes can be doing a SELECT at the same time. But only one process can be
making changes to the database at any moment in time, however.
If you use Android Room, see https://issuetracker.google.com/issues/62334005
Related
I would like to add a SyncAdapter to my app to update server-side data with minimal battery-use. For this, I need to have my application's main activity write data to some shared storage location that the SyncAdapter can then read in the onPerformSync(...) method.
I'm trying to figure out where best to store this data and have the following questions:
Could the SyncAdapter ever be called in a background thread while my main activity is executing in the foreground thread? I.e. do I need to worry about thread-safety of my storage access between the SyncAdapter and my main activity?
Are there ever any situations in which two instances of my main activity could be in the "Created" state? I.e. do I need to worry about thread-safety between two instances of my main activity?
Assuming that I do have to worry about any sort of thread safety, I have the following questions about the different data storage options:
Files in internal storage:
Are there any atomic operations I can perform on files in internal storage?
How about file locks?
SharedPreferences:
If two editors in different threads simultaneously modify different!!! keys in the Shared-Preferences, could two simultaneous commits lead to the loss of one of the changes?
SQL database:
Is the Android SQL Lite framework thread-safe if I simultaneously open the same SQL Lite database file from different threads?
Are there other ways to share data among SyncAdapters and (multiple instances of (if that is even possible)) the main activity?
Aside: To maximize compatibility of my app, I would like to not use any APIs greater than level 5.
I think, it should be possible to fix this issue by using a ContentProvider.
ContentProviders don't inherently solve the multithreading issue, as stated in the documentation:
Data access methods (such as insert(Uri, ContentValues) and update(Uri, ContentValues, String, String[])) may be called from many threads at once, and must be thread-safe. Other methods (such as onCreate()) are only called from the application main thread, and must avoid performing lengthy operations. See the method descriptions for their expected thread behavior.
But, unless the ContentProvider is declared with android:multiprocess=true, there should only ever be a single instance of the ContentProvider (if I understand this correctly), or at least all instances will live in the same process, i.e. they should have access the same static fields in the class definition.
With this, it should be possible to use the standard Java synchronization features to manage access to the storage-backed resources.
Please comment if I'm missing something...
Aside: Why all the "should's"?
Because, unfortunately, with Android's documentation, I'm never 100% sure that it's reliable or complete. Example here: The quote above states that "[o]ther methods (such as onCreate()) are only called from the application main thread"... What happens when I set android:multiprocess=true? The docs there state that "if this flag is set to 'true', the system can create an instance in every process where there's a client that wants to interact with it". Would this lead to a call to onCreate() from a thread other than this application's main thread? Probably...
I have an application that is doing a LOT of sqllite transactions, I currently have a bit of a hang because I am doing the sqllite actions on the UI thread... yes bad...
so I made each item have a thread and execute on it assuming sqllite api was smart enough to FIFO them.. nope ... now I get database is locked exceptions
this says it should work
without completely rewriting my code, and having a list of transactions queue up and execute them all on the same thread (many different classes, would be kind of a pain)
is there a way for me to check, and not execute a thread unless there isnt a lock? a lock check per se, or something similar that would get this to work, is efficient and isn't a huge rewrite?
Thanks
My answer that you quoted seems to be confusing. You don't have to do anything special when you are accessing the same Android database using the same database object with multiple threads. Under the covers, Sqlite has it's own locking to guarantee that the database will not be corrupted. To quote my answer;
Sqlite under Android is single threaded. Even if multiple threads were using the same database connection, my understanding is that they would be blocked from running concurrently. There is no way to get around this limitation
It has it's own locking which serializes the requests. This means that adding multiple threads will not increase the performance of the database unfortunately.
As my other answer mentions, you cannot use multiple database objects to the same database from multiple threads since there is no locking and you will corrupt your database.
I am trying to understand the possible ways to work with SQLite when there can be multiple threads work on DB.
Based on various responses in stackoverflow and other sites, it appears that there will be locking issue when same sqlitehelper instance is used from multiple threads. In a typical java application, I would expect instance to mean single object of type sqlite helper to be used by different threads of application.In such cases, the locks ,I guess, are a matter of correctly using the synchronized blocks. [Correct me here as I am not comfortable with this way of looking at sqliethelper instance here]
My concern is with sharing same data base : when one instantiate sqlite helper in different threads [ie each thread has its own object instance] but working on same Database [this I guess is more inline with having same db instance].
In such cases I'm getting frequent database lock errors. This occurs even when the threads are working on different tables of database.
In my application database can be updated by user interaction through application or by getting data through server [periodic synchronization]. And some time when synchronization process and user activity overlaps, I get the lock issues. As this pattern of data processing seems to be common in application synchronizing with server, would like to know how do lock issue due the concurrency is to be handled.
I would like to understand this since if this is bound to happen always then probably need to make only one handler over database and implement queue over that to avoid lock. But that will mean the complete application needs to be aware that the database may not get updated immediately and they need to implement listener to know when the data is actually updated in database.
thanks
pradeep
As far as I know sqlite is intended for single process usage. No matter what you will always need to access the database from one thread at a time. You can do selects from multiple clients but can only write from one at a time. And other readers and writers will ahve to lock in the mean time.
As a side note - database access can hardly ever be considered instantaneous.
I have a certain update method in my Android App, which generates quite an amount of data - up to hundreds of database entries.
I also have a background service, aside from the UI thread. Both threads have to execute the update method, sometimes even at the same time - basically, this is about generating and caching data to display. Both the UI and background service need this data.
Currently, I have wrapped the method's execution in an ORMLite transaction, which maps to an ordinary SQLite transaction. However, I am afraid that this will bite me in the butt one day, when some race condition screws up the data cache.
The question: Do SQLite transactions protect me from concurrent execution, or should I rather implement some kind of worker thread which is spawned when the generator-method shall start, or blocking if the generator-method is already running?
UPDATE:
I have decided to not rely on SQLite logic for the protection of my high-level Java method. The solution was for me as follows:
Wrap the generating part of the method with synchronized
Introduce a variable which tracks the last time of executing the method (set at the end of the method, so it is the marker of execution END)
First thing in the synchronized section, check if the last execution is in a specific threshold (e.g. <= 100ms in the past)
If yes, skip generation
If no, perform generation
In this way, duplicate generation should not take place, since when the method is accessed from two threads at the same time, the first will generate, but the second will not. The most important part for me here is that it is still blocking, since both threads rely on the generation having taken place after they have called the method.
EDIT:
It seems I'm wrong in my below statement: The SQLite implementation is, according to many, thread safe. I have, however, bitterly experienced threading issues, especially when testing database access, but that must have been caused by other factors in my code then, I assume.
Sorry for the misleading answer.
ORIGIN:
Good question!
You should be very careful here because the standard Android database access objects (such as SQLiteDatabase, Cursor etc) are not thread-safe by default. Not even ContentProvider's seem to give you a complete protection unless you explicitly write them with multithreading in mind.
According to Android documentation on ContentProvider's and threading (almost at the end of the page):
"Because these methods [update() is one of the functions] might be called from any number of threads at the same time, they too must be implemented to be thread-safe."
I don't know if there is any explicit locking mechanism to SQLiteDatabases (as in locking the actual database file). I would assume that a transaction itself would lock, at least the very handle you access your database with. I don't know what is true for the case where you have multiple handles to your database.
Maybe you could try to implement some singleton object (A ContentProvider maybe?) to access your database with, but even then you'd have to manage some sort of "request queue" I suppose.
You should also consider not to make any calls to the file system (the database is on the file system) from the UI-thread, what-so-ever. There is no guarantee that the database will answer in time and you're likely to end up with an ANR (especially as you write "...which generates quite an amount of data").
I have a an application that has 2 parts.
A service which creates content.
An application that uses the content
Each of these run as different processes. The problem is that both of them share a database. And I frequently get database locked error, both when the service tries to write something and the UI is reading data. Also vice versa.
How do go about this?
The class used to access DB is a singleton class. But since both UI & the service are 2 different processes, there are 2 singletons I presume. So that doesn't help.
Even synchronise won't help I suppose, since again because of 2 different processes.
Content Providers maybe an option, but since I use complex queries to dig info, it would be really hard to use that too.
How do I get the two processes share the database.
Any cues would be greatly appreciated.
Using a content provider is one option. Another is to take a look at Berkeley DB. The BDB SQL API is SQLite compatible and the BDB lock manager allows multiple threads and/or processes to read/write to the database concurrently.
close the connection after each operation
catch the database locked error and try to reconnect after 50ms
or let the service handle the database and the activity ask the service for data
may be there is isDatabaseInUseMethod ?
You should use a content provider to funnel your database queries through one source. Inside of the content provider you can use any locking mechanisms you would like to ensure you're not having concurrent access. You may also think about using content observers to coordinate service actions with changes to the database.
The following is a great article on how locking works with SQLite on Android and what things to be aware of: http://kagii.squarespace.com/journal/2010/9/10/android-sqlite-locking.html
I would think you'll find some answers there :)