In one of my Android applications,
- I have the SyncAdapter running in a different process, which actually queries the data from sqlite db and pushes the data to server.
- SqliteDatabase is singleton and there is only instance of it across the application.
But, right after I install the application, two SqliteDatabase instances are getting created. One for my application and one for the background process(sync). This way two instances are created and they are acting on the same db.
In these scenarios, if the two instances at a time try to insert the db, one of the request will throw error.
Is this the correct way to handle (CRUD db) when having multiple processes ?
Code where the single instance of db is created
public class SqliteStorage extends SqliteOpenHelper {
public SqliteStorage(Context context) {
super(context, DB_NAME, null, DB_VERSION);
db = getWritableDatabase();
db.setVersion(DB_VERSION);
}
public static synchronized SqliteStorage getInstance() {
if(mDBStorage == null) {
mDBStorage = new SqliteStorage(MyApplication.getContext());
}
return mDBStorage;
}
}
Related
My app uses an SQLite DB, wrapped with a SQLiteOpenHelper and a ContentProvider. I added a sign-in feature to the app, and now I want every user to only be able to see his own data. The way I thought to achieve this is for the app to create a separate DB for every user that signs in to the app, and use the user's ID in the filename of the database.
I have this ContentProvider:
public class MyProvider extends ContentProvider {
//...
#Override
public boolean onCreate() {
dbHelper = new MyDBHelper(getContext());
return true;
}
I have this SQLiteOpenHelper:
public class MyDBHelper extends SQLiteOpenHelper {
Which has this constructor:
public MyDBHelper(Context context) {
super(context, DB_NAME, null, DB_VERSION);
}
Up until now the app couldn't have multiple users, so it just had one database. so DB_NAME was always the same String. I now tried to set it like that:
private static String UID = FirebaseAuth.getInstance().getCurrentUser().getUid();
public static final String DB_NAME = String.format("data%s.db", UID);
(As you can see, I'm using Firebase Authentication)
but this resulted in a crash, because apparently the content provider is created on app start, before the user has authenticated. (so user is null. Yeah, I should check that user is not null before I try to call getUid(). but this won't make this thing work)
So this doesn't seem like the right approach. How can I use a different DB according to the signed user? Can I make the content provider to first be created after a user has authenticated?
I could also just keep everything in one database and add a UID column. But will this be protect the different users' data good enough from each other? Also, this would mean a lot more code changes.
How can I use a different DB according to the signed user?
The simple solution is to get rid of the ContentProvider. The only reason to use a ContentProvider is if you are going to be serving this data to other apps.
Also, I would be wary of just taking getUid() and putting it in a filename. You are not in control over what getUid() returns, and it might someday contain characters that are invalid in filenames.
Can I make the content provider to first be created after a user has authenticated?
No, sorry.
Seems that the right solution here is to not use ContentProviders. So I accepted the other answer.
But to answer my actual question, for people that are determined to make different DBs work with one ContentProvider, here is how it can be done:
I changed the custom SQLiteOpenDBHelper's constructor to also take a uid:
public MyDBHelper(Context context, String uid) {
super(context, String.format(DB_NAME, uid), null, DB_VERSION);
UID = uid;
}
and I changed the onCreate of my ContentProvider not to create the DBHelper. I created this function that initializes the DBHelper instead:
public void initDB(Uri uri) {
String uid = uri.getPathSegments().get(0);
if (dbHelper == null){
dbHelper = new MyDBHelper(getContext(), uid);
} else if (!uid.equals(dbHelper.UID)){
dbHelper.close();
dbHelper = new MyDBHelper(getContext(), uid);
}
}
and I call this method at the start of the query, insert, update and delete methods.
So the DBHelper which holds the open connection to the DB, is initialized whenever the content provider is preforming some action on the DB but there is either not yet an existing connection with the DB, or the connection is with a DB of a different user.
This is not the right way to solve this problem and this probably has consequences in some cases. But I didn't want to leave the question I asked unanswered.
I am asking this question in context of a problem in my app, about which I find it difficult to create an exact question. But I do have a lead.
I do have parallel threads running and my problem revolves around the case where running queries on database returns NullpointerException on the initialized database instance.
So what I want to know is that if you initialize an instance of a database by db.getWritableDatabase() in 2 parallel threads, does closing the database in one thread by db.close(), closes it in the other thread ? infact across the application level ?
You should create singleton of SQLiteOpenHelper/db (you did not specify what class db is) which would return you only one instance and then you could check if db is closed or not.
I had similar problem and in the end 2 parallel threads and 1 database ? You are asking for problems.
You cannot safely have 2 actions operating with database at the same time.
/**
* Returns a writable database instance in order not to open and close many
* SQLiteDatabase objects simultaneously
*
* #return a writable instance to SQLiteDatabase
*/
public SQLiteDatabase getMyWritableDatabase() {
if ((db == null) || (!db.isOpen())) {
db = this.getWritableDatabase();
}
return db;
}
#Override
public void close() {
super.close();
if (db != null) {
db.close();
db = null;
}
}
There are a few Android APIs (after donut and before honeycomb) if Im not mistaken, where Google have enabled the AsyncTasks to run paralelly aiming for faster execution. Then lots of devs made mistakes when reaching out to the same database using multiple AsyncTasks, and since Android 3.0 AsyncTasks are running serially by default.
I am suffering this problem now when testing my app on an Android 2.3.4 device with my SQLite
First, Im getting categories from the server, I open DB, insert them close DB.
Second I get the subcategories from the server, open DB, insert them into DB, close DB
Third I get user items from the server, open DB, insert items, then close DB
Im taking good care to ensure that one starts after another, but in every 8-10 iterations something somewhere slows down and overlaps with another procedure right in the moment where a task is opening the db, another task closes it right after, and the first task starts trying to write to a closed db....
What do I do? I want clean, reliable separation, sequential execution and I dont want to start the asynctasks from the previous asynctask's onPostExecute, because these three will not always run in a row
I read an article yesterday that you CANT do it on android 2.x
Shall I try to open the DB and DBHelper before ALL of the operations and close the DB afterwards?
EDIT: Usually I get the error here (at Begin transaction):
(The error says that the DB is closed)
#Override
protected Void doInBackground(Void... arg0) {
// dbTools.close();
try {
if (database == null) {
database = dbTools.getWritableDatabase();
}
} catch (Exception e) {
e.printStackTrace();
}
ContentValues values = new ContentValues();
database.beginTransaction();
try {
// Iterating all UserItem objects from the LinkedHashSet and getting their info
for (UserItem userItem : userItems) {
// Inserting values for the database to insert in a new record
values.put(C.DBColumns.ITEM_ID, userItem.getItemId());
values.put(C.DBColumns.ITEM_NAME, userItem.getItemName());
// database.insertWithOnConflict(C.DBTables.ITEMS, null, values, SQLiteDatabase.CONFLICT_REPLACE);
database.insert(C.DBTables.ITEMS, null, values);
} // End of For loop
database.setTransactionSuccessful();
} finally {
database.endTransaction();
}
// Closing all cursors, databases and database helpers properly because not closing them can spring lots of trouble.
if (database != null && database.isOpen()) {
try {
database.close();
} catch (Exception e) {
e.printStackTrace();
}
}
return null;
} // End of doInBackground
And this is my DBTOOLS CLASS:
public class DBTools extends SQLiteOpenHelper {
// Its a good practice for DBTools to be a singleton. Do not instantiate it with "new DBTools(context)" but with
// DBTools.getInstance(context) instead
private static DBTools sInstance;
public static DBTools getInstance(Context context) {
if (sInstance == null) {
sInstance = new DBTools(context);
}
return sInstance;
}
public DBTools(Context context) {
super(context, C.Preferences.LOCAL_SQLITE_DATABASE_NAME, null, 1);
}
public void onCreate(SQLiteDatabase database) {
database.execSQL(SQLQueries.tableCategoriesCreate);
database.execSQL(SQLQueries.tableSubcategoriesCreate);
database.execSQL(SQLQueries.tableItemsCreate);
}
public void onOpen(SQLiteDatabase database) {
database.execSQL(SQLQueries.tableCategoriesCreate);
database.execSQL(SQLQueries.tableSubcategoriesCreate);
database.execSQL(SQLQueries.tableItemsCreate);
}
public void onUpgrade(SQLiteDatabase database, int version_old, int current_version) {
database.execSQL(SQLQueries.tableCategoriesDrop);
database.execSQL(SQLQueries.tableSubcategoriesDrop);
database.execSQL(SQLQueries.tableItemsDrop);
onCreate(database);
}
} // End of Class
Since you can't call from onPostExecute, I would say you have two options, one would be to move your open close calls to the beginning and end of your activity or service.
Option two would be to setup a reference counter in your DB and DBHelper where you track the number of times open has been called, and then decrement that count when close is called. That way you can perform close only when the count is 0. One thing to remember when taking this approach is that you should probably have a method that will force the db to close that you call when you are sure your other connections are done. This shouldn't be necessary but will be a failsafe to ensure the db gets closed if something goes wrong.
Edit: You would have to make DBTools a singleton for it to work, but it's not equivalent. Here's a quick example.
public class DBTools {
private static DBTools instance;
private static int openCount;
public DBTools getInstance() {
if (instance == null) {
instance = new DBTools();
}
return instance;
}
private DBTools() {
openCount = 0;
}
public void open() {
openCount++;
//Do open
}
public close() {
openCount--;
if (openCount == 0) {
//Do close
}
public void forceDBClose() {
//Do close
}
}
I am also a newbee in android. I was having a problem like this too.
To overcome this, i used Singleton class.
I created one instance of the DBHelper class and used it in all my asynctasks.
So, until the DB is closed, all the asynctasks access the initialised DB object.
If there is no object in the memory, the async tasks, instantiates it and use it then.
I don't know how to handle this correctly without getting database locked errors.
My app basically downloads many items in batches of 100 rows (~ 60.000 items) and inserts them in the database. Each batch of 100 rows is processed into a transaction.
The main activity allows the user to navigate between screens (fragments) while records are being downloaded and inserted. Most of the other screens contains read data from the database. I get a lot of database lock errors during reading. All readings are done in the main activity (not fragments) in different async tasks
So far I just used the "classic approach"
public class DBAdapter {
public DBAdapter(Context ctx) {
this.context = ctx;
DBHelper = new DatabaseHelper(context);
}
private static class DatabaseHelper extends SQLiteOpenHelper {
DatabaseHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
}
#Override
public void onCreate(SQLiteDatabase db) {
db.execSQL(DB_CREATE_TABLES);
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
Utils.log("Upgrading database from version " + oldVersion + " to " + newVersion + ", which will destroy all old data");
onCreate(db);
}
public DBAdapter open() throws SQLException {
database = DBHelper.getWritableDatabase();
return this;
}
public void close() {
DBHelper.close();
}
Then on my activity's onCreate() I call db = new DBAdapter(context); and each time I am doing an database operation (read/write) I call db.open() and after the insert/read is done I call db.close()
My questions are:
what would be the best approach to this situation ?
Considering I do a lot of write/read would it be better to call db.open on onCreate and db.close() on onDestroy() ? Would this be better than calling open/close each time I need to access the database ?
What do I need to do to avoid database locking on reading ?
I had a exactly similar situation like yours. In addition to what you described, in my app the user also can update the database through input on the screen.
The way I resolved it ( I don't know if it's the best way, but I hardly see any locking issue now)
Make a singleton class derived from SQLiteOpenHelper to make sure only one instance is running at any given time.
Implement ContentProvider class for insert/update/delete/query operations. Make all those functions 'synchronized'
Only close the db in ContentProvider's shutdown function. I do a very frequent db operations, so I don't want to open/close everytime. But I am not sure if it's the correct way of handling it.
Do access DB only through ContentProvider interface from anywhere
A very simple approach, or maybe a workaround is using synchronized methods for opening and closing the database object. I don't really know if it's the best practice, but at least it's simple and easy. Add this methods to your DBAdapter Class, and use them instead of db.open and db.close. The use_count attribute simple holds how many times open has been called. Initialize it with a value of 0. Also, in order to make it work on your solution be sure to pass the same DBAdapter object between the fragments. Don't create a new one everytime :
private int use_count = 0;
public synchronized void doOpen()
{
use_count++;
this.open();
}
public synchronized void doClose()
{
use_count--;
if (use_count == 0)
{
this.close();
}
}
Consider wrapping the SQLite database in a ContentProvider and using CursorLoader to do the queries from the various activities & fragments. This isolates the management of the database from the Activity/Fragment life cycle and can result in many fewer open/close cycles.
You may still run into contention between the reads and writes, but having all the database interaction in the same module should make it easier for you to address these issues.
Some interesting links: http://www.vogella.com/articles/AndroidSQLite/article.html#todo
When to use a Content Provider
We are using AsyncTasks to access database tables and cursors.
Unfortunately we are seeing occasional exceptions regarding the database being locked.
E/SQLiteOpenHelper(15963): Couldn't open iviewnews.db for writing (will try read-only):
E/SQLiteOpenHelper(15963): android.database.sqlite.SQLiteException: database is locked
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.native_setLocale(Native Method)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.setLocale(SQLiteDatabase.java:1637)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.<init>(SQLiteDatabase.java:1587)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:638)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:659)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:652)
E/SQLiteOpenHelper(15963): at android.app.ApplicationContext.openOrCreateDatabase(ApplicationContext.java:482)
E/SQLiteOpenHelper(15963): at android.content.ContextWrapper.openOrCreateDatabase(ContextWrapper.java:193)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:98)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteOpenHelper.getReadableDatabase(SQLiteOpenHelper.java:158)
E/SQLiteOpenHelper(15963): at com.iview.android.widget.IViewNewsTopStoryWidget.initData(IViewNewsTopStoryWidget.java:73)
E/SQLiteOpenHelper(15963): at com.iview.android.widget.IViewNewsTopStoryWidget.updateNewsWidgets(IViewNewsTopStoryWidget.java:121)
E/SQLiteOpenHelper(15963): at com.iview.android.async.GetNewsTask.doInBackground(GetNewsTask.java:338)
E/SQLiteOpenHelper(15963): at com.iview.android.async.GetNewsTask.doInBackground(GetNewsTask.java:1)
E/SQLiteOpenHelper(15963): at android.os.AsyncTask$2.call(AsyncTask.java:185)
E/SQLiteOpenHelper(15963): at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:256)
E/SQLiteOpenHelper(15963): at java.util.concurrent.FutureTask.run(FutureTask.java:122)
E/SQLiteOpenHelper(15963): at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:648)
E/SQLiteOpenHelper(15963): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:673)
E/SQLiteOpenHelper(15963): at java.lang.Thread.run(Thread.java:1060)
Does anybody have a general example for code which writes to a database from a different thread than the one reading and how can we ensure thread safety.
One suggestion I've had is to use a ContentProvider, as this would handle the access of the database from multiple threads. I am going to look at this, but is this the recommended method of handling such a problem? It seems rather heavyweight considering we're talking about in front or behind.
We used a ContentProvider in the end. This appeared to clear up the problems.
I solved this same exception just by making sure all my database opens have closes, and (more importantly) to assure this, making the scope of each database instance local ONLY to the method that needs it. ContentProvider is a good, safe class to use when accessing a db from multiple threads, but also make sure you're using good db practices:
Keep db instances local (no SQLiteDatabase class members!)
call close() on the db in the same method in which it's opened
call close() on the cursors you get from the db
listen to LogCat for any complaints that SQLiteDatabse might have
Before some code, let's resume some of the approachs:
Semaphores: by far the best solution presented. It goes in the heart of the problem: resource sharing! It will treat the locking of the database access, avoiding conflicts (database is locked).
Java synchronization: A kind of semaphore implementation, but less sofisticated. Using synchronized you will not easily solve some cases involving transactions.
ContentProvider: implement ContentProvider solve the problem only for some cases (or sweep the problem under the carpet). You'll yet face the same issues. The difference is that ContentProvider pattern will guide you to not make some commom mistakes when accessing Sqlite database. The ContentProvider docs says: "You don't need a provider to use an SQLite database if the use is entirely within your own application."
Almost mandatory: keep db instances local, call close() on the db in the same method in which it's opened using finally statements, close() on the cursors using finally statements, etc are almost mandatory to avoid problems using Sqlite.
Let's show an example of the semaphore solution presented by Moss, which I took from CL. and improoved to cover transactions.
class DataAccess {
private final ReentrantReadWriteLock rwl = new ReentrantReadWriteLock();
private final Lock r = rwl.readLock();
private final Lock w = rwl.writeLock();
public Data readSomething(int id) {
Cursor c = null;
r.lock();
try {
c = getReadableDatabase().query(...);
return c.getString(0);
} finally {
if (c != null) c.close();
r.unlock();
}
}
public void changeSomething(int id, int value) {
w.lock();
try {
getWritableDatabase().update(...);
} finally {
w.unlock();
}
}
private void beginTransactionWithSemaphores() {
getWritableDatabase().beginTransactionWithListener(new SQLiteTransactionListener() {
#Override
public void onBegin() {
w.lock();
}
#Override
public void onRollback() {
w.unlock();
}
#Override
public void onCommit() {
w.unlock();
}
});
}
}
Take into account that SQLite databases are file based and are not intended to be able to be accessed in a multi-process way. The best procedure on mixing SQLite with multi-processing is using semaphores (aquire(), release()) in each database related access.
If you create a Db wrapper that aquires/releases a global semaphore your DB access will be thread safe. Indeed this means that you could get a bootleneck because you are queueing the access to the DB. So in addition you could only wrap the access using semaphores if it's an operation that alters the database, so while you are alterin the db no one will be able to access it and wait until the write process has been completed.
We could not share Db connection with multiple thread to perform read and write operation in database simultaniously.We will have to make single object of DB using syncronization concept and we will perform one task at a time .We will use singleton pattern to make the DB object and it will be share within multiple threads.At a time will perform single task . then we will start other task or any operation on DB .
Content provider is not the solution of DB locking issue .
import java.util.concurrent.atomic.AtomicInteger;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteOpenHelper;
import android.util.Log;
public class DatabaseManager {
private AtomicInteger mOpenCounter = new AtomicInteger();
private static DatabaseManager instance;
private static SQLiteOpenHelper mDatabaseHelper;
private SQLiteDatabase mDatabase;
//private static String DB_PATH = "";
// private static String DB_NAME = "xyz.db";// Database name
private static String dbPathh;
public static synchronized void initializeInstance(SQLiteOpenHelper helper,
String dbPath) {
if (instance == null) {
instance = new DatabaseManager();
mDatabaseHelper = helper;
dbPathh=dbPath;
}
}
public static synchronized DatabaseManager getInstance() {
if (instance == null) {
throw new IllegalStateException(DatabaseManager.class.getSimpleName() +
" is not initialized, call initializeInstance(..) method first.");
}
return instance;
}
public synchronized SQLiteDatabase openDatabase(String thread) {
if(mOpenCounter.get() == 0) {
// Opening new database
// mDatabase = mDatabaseHelper.getWritableDatabase();
MyLog.e("Path Of DataBase", dbPathh);
// mDatabase=mDatabaseHelper.getWritableDatabase();
mOpenCounter.incrementAndGet();
mDatabase=SQLiteDatabase.openDatabase(dbPathh, null,
SQLiteDatabase. CREATE_IF_NECESSARY|SQLiteDatabase.OPEN_READWRITE);
MyLog.e("Open Data Base", " New Connection created" +thread);
}
else{
MyLog.e("Open Data Base", " Old Connection given " +thread);
}
// Toast.makeText(NNacres.getConfig(), "open conn: present connection =
" +mOpenCounter.get(), Toast.LENGTH_LONG).show();
return mDatabase;
}
public synchronized void closeDatabase() {
MyLog.e("Close db connection", ""+mOpenCounter.get());
if(mOpenCounter.get() == 1) {
// Closing database
mDatabase.close();
mOpenCounter.decrementAndGet();
Log.e("DB CLOSED", "DONE");
}
//Toast.makeText(NNacres.getConfig(), "close conn: after close =
" +mOpenCounter.get(), Toast.LENGTH_LONG).show();
}
}
and write this method in your YourSQLiteDataABse helper class which extends SQLiteOpenHelper Class
public SQLiteDatabase getWritableDatabase() {
DatabaseManager.initializeInstance(this,"data/data/your packgae name/databases/xyz");
return DatabaseManager.getInstance().openDatabase(getClass().getSimpleName());
}
public static String getMyDbPath(String DB_NAME, Context context) {
String myDbPath = context.getDatabasePath(DB_NAME).getPath();
MyLog.e("DB Path: "+myDbPath);
return myDbPath;
}
You must be calling getWritableDatabase() from a function rather then the constructor of the db helper class. If the db helper class object is created with SQLiteDatabase.openOrCreateDatabase(DB_PATH, null); or similar and then getWritableDatabase() is called from a function, it will try to make a synchronous call to DB causing a DB lock exception.
Are you talking of a single user action that, inside your program, causes multiple threads to be run, more than one of which may be accessing the database in update mode ?
That's bad design, period. There is no way for you to know in which order the threads will be scheduled by your OS (/VM), and therefore there is no way for you to know in which order the database accesses will happen, and that is very likely to imply that there is no way for you to know that database accesses will always happen in the order that you are expecting.
All database accesses generated by/coming from some user action should all be done in one single thread.