Develop Reference

SQLite database, multithreading, Locks and account sync on android

I'm trying to get a pattern that doesn't fail for a multithreaded access to my sqlite database. Also, what is driving me nuts is that I can't reproduce the issue.
I have an app which uses a DB, but also Android Accounts and Android sync to sync my app's data. My guess is that when the two happen a the same time, it crashes. I'm getting a lot of errors like:
* android.database.sqlite.SQLiteDatabaseLockedException: database is locked
* android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5)
* android.database.sqlite.SQLiteDatabaseLockedException: error code 5: database is locked
* android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5): , while compiling: PRAGMA journal_mode
* android.database.sqlite.SQLiteDiskIOException: disk I/O error (code 778)
* android.database.sqlite.SQLiteException: Failed to change locale for db '/data/data/net.bicou.redmine/databases/redmine.db' to 'en_US'. \n Caused by: android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5)
Maybe not all of them are related to the same root cause, however I'm kind of lost.
What I have is:
an abstract base class, DbAdapter, that is extended by subclasses which want to manage a single table
a class that manages the SQLite database, called DbManager, which contains a Lock
Right now the users have a version of the DbManager that is not a singleton. I'm planning to make DbManager a singleton, so that all threads share the same object. This shouldn't be a problem, because as far as I have understood/seen, the background sync and app share the same process.
Here are the classes (only the relevant parts):
public abstract class DbAdapter {
Context mContext;
protected DbManager mDbManager;
SQLiteDatabase mDb;
public static final String KEY_ROWID = "_id";
public DbAdapter(final Context ctx) {
mContext = ctx;
}
public DbAdapter(final DbAdapter other) {
mContext = other.mContext;
mDb = other.mDb;
mDbManager = other.mDbManager; // removed with singleton version
}
public synchronized DbAdapter open() throws SQLException {
if (mDb != null) {
return this;
}
mDbManager = new DbManager(mContext); // currently in production
mDbManager = DbManager.instance(mContext); // currently investigating this singleton solution
try {
mDb = mDbManager.getWritableDatabase();
} catch (final SQLException e) {
L.e("Unable to open DB, trying again in 1 second", e);
try {
Thread.sleep(1000);
} catch (final InterruptedException e1) {
L.e("Could not wait 1 second " + e1);
}
mDb = mDbManager.getWritableDatabase();// This may crash
}
return this;
}
public synchronized void close() {
mDbManager.close();
mDbManager = null;
mDb = null;
}
}
A class that needs to handle a database table will extend DbAdapter, and implement methods such as select, insert, delete, etc.
Here's the DB manager:
public class DbManager extends SQLiteOpenHelper {
private static final String DB_FILE = "db";
private static final int DB_VERSION = 15;
Context mContext;
Lock mLock = new ReentrantLock();
// Currently in prod
public DbManager(final Context context) {
super(context, DB_FILE, null, DB_VERSION);
mContext = context;
}
// singleton version will make this constructor private and add:
private static DbManager mInstance;
public static synchronized DbManager instance(Context context) {
if (instance == null) {
instance = new DbManager(context);
}
return instance;
}
#Override
public SQLiteDatabase getWritableDatabase() {
mLock.lock();
return super.getWritableDatabase();
}
#Override
public void close() {
super.close();
mLock.unlock();
}
#Override
public void onCreate(final SQLiteDatabase db) {
// ...
}
#Override
public void onUpgrade(final SQLiteDatabase db, final int oldVersion, final int newVersion) {
// ...
}
private void createTables(final SQLiteDatabase db, final String[] statements) {
for (final String sql : statements) {
try {
db.execSQL(sql);
} catch (final Exception e) {
L.e("Unable to create table: " + sql, e);
}
}
}
}
OK, now, the questions.
Is my lock properly implemented? I'm really new to this, I don't know if the ReentrantLock is a good choice, and if I'm locking/unlocking at the right moment
Are my synchronized method properly implemented? I mean, I have placed the synchronized keyword around methods that I don't want interrupted by concurrent threads. Is this right? Can you advice on my synchronized use?
How can I reproduce the issue? I have created a test that uses 3 threads that make concurrent read/write access to the DB, and use some Thread.sleep to ensure that the db open/close from each thread overlap, but it doesn't crash. This is really bugging me, I don't think there is a lot of people that have the issue, so I don't know how to reproduce.
Is my DbAdapter + DbManager technical choice a good idea? Is there a better pattern?
Is it a good idea to make DbManager a singleton?

For multiple threads accessing, it is advisable to use the singleton pattern.
Such a way, successive calls to the same database will be seamlessly serialised.
However, it's not impossible to have some NullPointerExceptions on inserts. So, to expand your "Thread.sleep" logic, you could use this code:
#Override
public SQLiteDatabase getWritableDatabase() {
while (true) {
try {
return super.getWritableDatabase();
} catch (SQLiteDatabaseLockedException e) {
System.err.println(e);
}
try {
Thread.sleep(500);
} catch (InterruptedException e) {
System.err.println(e);
}
}
}

Android: Database INSERT triggers exception "getWritableDatabase called recursively"

Whenever the asynchonous task tries to insert data into the SQLite database I run into the following exception.
java.lang.IllegalStateException: getWritableDatabase called recursively
The data is produced by a service that runs separate asynchonous tasks to download, convert and finally to insert the data. I am not sure if the context objects I am passing are correct. Please note the comments I added to the source code. In the following I added the relevant classes and functions. Please leave a comment if you need futher information.
public class CustomServiceHelper {
// This method gets called by the activities.
public static void loadData(Context context) {
Intent intent = new Intent(context, CustomService.class);
context.startService(intent);
}
}
...
public class CustomService extends Service {
private void startStoringTask(Users users) {
// Passing the context of the service.
mStoringTask = new StoringTask(this);
Object[] params = { users };
mStoringTask.execute(params);
}
}
...
public class UsersProvider extends ContentProvider {
#Override
public boolean onCreate() {
// Not sure if getContext() is correct.
mDatabase = new CustomDatabase(getContext());
return true;
}
#Override
public Uri insert(Uri uri, ContentValues values) {
switch (URI_MATCHER.match(uri)) {
case URI_CODE_USERS:
long id = mDatabase.insertUsers(values);
// TODO: Not sure about the return value.
return ContentUris.withAppendedId(uri, id);
}
return null;
}
}
...
public class CustomDatabase {
public class CustomSQLiteOpenHelper extends SQLiteOpenHelper {
CustomSQLiteOpenHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
}
#Override
public void onCreate(SQLiteDatabase database) {
database.execSQL(TABLE_USERS_CREATE);
}
#Override
public void onUpgrade(SQLiteDatabase database, int oldVersion, int newVersion) {
throw new UnsupportedOperationException();
}
}
private CustomSQLiteOpenHelper mDatabaseHelper = null;
public CustomDatabase(Context context) {
// The context is passed in by the UsersProvider.
mDatabaseHelper = new CustomSQLiteOpenHelper(context);
}
public long insertUsers(ContentValues contentValues) {
return mDatabaseHelper.getWritableDatabase().insert(
CustomSQLiteOpenHelper.TABLE_USERS, null, contentValues);
}
}
...
public class StoringTask extends AsyncTask<Object, Void, Boolean> {
private Context mContext = null;
public StoringTask(Context context) {
mContext = context;
}
#Override
protected Boolean doInBackground(Object... params) {
Users users = (Users)params[0];
return storeUser(users);
}
private boolean storeUsers(Users users) {
if (users == null) return false;
// Not sure if calling getContentResolver on this context is correct.
ContentResolver contentResolver = mContext.getContentResolver();
Iterator<User> iterator = users.iterator();
do {
User user = iterator.next();
storeUser(contentResolver, user);
}
while (iterator.hasNext());
return true;
}
private void storeUser(ContentResolver contentResolver, User user) {
ContentValues cv = new ContentValues(1);
cv.put(CustomDatabase.Contract.COLUMN_NAME, user.name);
contentResolver.insert(UsersProvider.Contract.URI_USERS, cv);
}
}
When I run the application in Debug mode I end up in the framework class ThreadPoolExecutor in the finally block of the runWorker() method.
Edit:
Here is the full exception stack trace.
java.lang.RuntimeException: An error occured while executing doInBackground()
android.os.AsyncTask$3.done(AsyncTask.java:200)
java.util.concurrent.FutureTask$Sync.innerSetException(FutureTask.java:273)
java.util.concurrent.FutureTask.setException(FutureTask.java:124)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:307)
java.util.concurrent.FutureTask.run(FutureTask.java:137)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1068)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:561)
java.lang.Thread.run(Thread.java:1096)
Caused by: java.lang.IllegalStateException: getWritableDatabase called recursively
android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:81)
com.test.users.database.CustomDatabase.insertUsers(CustomDatabase.java:125)
com.test.users.database.CustomDatabase$CustomSQLiteOpenHelper.insertTestData(CustomDatabase.java:85)
com.test.users.database.CustomDatabase$CustomSQLiteOpenHelper.onCreate(CustomDatabase.java:60)
android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:106)
com.test.users.database.CustomDatabase.insertUsers(CustomDatabase.java:125)
com.test.users.contentprovider.UsersProvider.insert(UsersProvider.java:51)
android.content.ContentProvider$Transport.insert(ContentProvider.java:174)
android.content.ContentResolver.insert(ContentResolver.java:587)
com.test.users.tasks.StoringTask.storeFeature(StoringTask.java:82)
com.test.users.tasks.StoringTask.storeResponse(StoringTask.java:58)
com.test.users.tasks.StoringTask.doInBackground(StoringTask.java:32)
com.test.users.tasks.StoringTask.doInBackground(StoringTask.java:1)
android.os.AsyncTask$2.call(AsyncTask.java:185)
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
I did not mention the method CustomSQLiteOpenHelper.insertTestData() which invokes getWritableDatabase(). I call the method insertTestData() in CustomSQLiteOpenHelper.onCreate(). The crash happens everytime whenever the application has not created a database yet. To summarize, hawaii.five-0 was totally right!

java.lang.IllegalStateException: getWritableDatabase called recursively
So this kind of Exception is usually thown when you use getWritableDatabase() or getReadableDatabase() in onCreate or onUpgrade methods of SQLiteOpenHelper.

SQLiteOpenHelper getWritableDatabse() fails with no Exception

I have a very strange problem. It only shows from time to time, on several devices. Can't seem to reproduce it when I want, but had it so many times, that I think I know where I get it.
So I have a Loader which connects to sqlite through a singleton SQLiteOpenHelper:
try {
Log.i(TAG, "Get details offline / db helper: "+DatabaseHelper.getInstance(getContext()));
SQLiteDatabase db=DatabaseHelper.getInstance(this.getContext()).getWritableDatabase();
Log.i(TAG, "Get details offline / db: "+db);
//doing some work on the db...
} catch(SQLiteException e){
e.printStackTrace();
return null;
} catch(Exception e){
e.printStackTrace();
return null;
//trying everything to grab some exception or whatever
}
My SQLIteOpenHelper looks something like this:
public class DatabaseHelper extends SQLiteOpenHelper {
private static DatabaseHelper mInstance = null;
private static Context mCxt;
public static DatabaseHelper getInstance(Context cxt) {
//using app context as suggested by CommonsWare
Log.i("DBHELPER1", "cxt"+mCxt+" / instance: "+mInstance);
if (mInstance == null) {
mInstance = new DatabaseHelper(cxt.getApplicationContext());
}
Log.i("DBHELPER2", "cxt"+mCxt+" / instance: "+mInstance);
mCxt = cxt;
return mInstance;
}
//private constructor
private DatabaseHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
this.mCxt = context;
}
#Override
public void onCreate(SQLiteDatabase db) {
//some tables created here
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
//upgrade code here
}
}
It really works great in most cases. But from time to time I get a log similar to this:
06-10 23:49:59.621: I/DBHELPER1(26499): cxtcom.myapp#407152c8 / instance: com.myapp.helpers.DatabaseHelper#40827560
06-10 23:49:59.631: I/DBHELPER2(26499): cxtcom.myapp#407152c8 / instance: com.myapp.helpers.DatabaseHelper#40827560
06-10 23:49:59.631: I/DetailsLoader(26499): Get event details offline / db helper: com.myapp.helpers.DatabaseHelper#40827560
06-10 23:49:59.631: I/DBHELPER1(26499): cxtcom.myapp#407152c8 / instance: com.myapp.helpers.DatabaseHelper#40827560
06-10 23:49:59.651: I/DBHELPER2(26499): cxtcom.myapp#407152c8 / instance: com.myapp.helpers.DatabaseHelper#40827560
This line Log.i(TAG, "Get details offline / db: "+db); never gets called! No Exceptions, silence. Plus, the thread with the Loader is not running anymore.
So nothing past the line:
SQLiteDatabase db=DatabaseHelper.getInstance(this.getContext()).getWritableDatabase();
gets executed.
What can possibly go wrong on this line?

SQLite database never close in my Android app

I have a problem with my Android application. I'm using code bellow to opening SQLite database in AsyncTask. Everything works fine but when I try to close database in onStop() or onDestroy method, it's never closed.
Code for creating and opening database:
public class SQLiteDB extends SQLiteOpenHelper{
private final Context context;
private SQLiteDatabase sqliteDatabase = null;
public SQLiteDB(Context context, String DBName) {
super(context, DBConstant.DB_NAME, null, context.getResources().getInteger(ppredota.android.navigation.view.activities.R.string.database_version));
this.context = context;
}
public void createDB() throws IOException{
if(existDB()){
this.getReadableDatabase();
this.close();
}
else {
this.getWritableDatabase();
try {
copyDB();
this.close();
}
catch (Exception e) {
throw new Error("Chyba pri kopirovani databaze");
}
}
}
private boolean existDB() {
SQLiteDatabase checkDatabase = null;
try{
String fullPath = DBConstant.DB_PATH + DBConstant.DB_NAME;
checkDatabase = SQLiteDatabase.openDatabase(fullPath, null, SQLiteDatabase.OPEN_READWRITE);
}
catch (SQLiteException sqle) {
Log.i("existDB()", "Databaze nelze otevrit, neexistuje");
}
if(checkDatabase == null){
Log.i("existDB", "Databaze jeste neexistuje...");
return false;
}
else{
Log.i("existDB", "Databaze uz existuje...");
checkDatabase.close();
return true;
}
}
private void copyDB() throws IOException {
InputStream inDBStream = context.getAssets().open(DBConstant.DB_NAME);
String newDBPath = DBConstant.DB_PATH + DBConstant.DB_NAME;
OutputStream outDBStream = new FileOutputStream(newDBPath);
Log.i("copyDB", "Otevren outputstream s cestou k nove databazi");
byte[] buffer = new byte[1024];
int length;
while ((length = inDBStream.read(buffer))>0){
outDBStream.write(buffer, 0, length);
}
outDBStream.flush();
outDBStream.close();
inDBStream.close();
}
public void openDB() throws SQLException {
String fullPath = DBConstant.DB_PATH + DBConstant.DB_NAME;
if(sqliteDatabase!=null){
if(sqliteDatabase.isOpen()){
Log.i("openDB()", "Databaze je jiz otevrena");
}
else{
sqliteDatabase = SQLiteDatabase.openDatabase(fullPath, null, SQLiteDatabase.OPEN_READONLY);
Log.i("openDB()", "Databaze" + sqliteDatabase.getPath() + "otevrena");
}
}
else{
sqliteDatabase = SQLiteDatabase.openDatabase(fullPath, null, SQLiteDatabase.OPEN_READONLY);
if(sqliteDatabase.isOpen()){
Log.i("openDB()", "Databaze otevrena");
}
}
}
#Override
public void close() {
if(sqliteDatabase!=null){
sqliteDatabase.close();
Log.i("close()", "Databaze zavrena");
}
super.close();
}
public SQLiteDatabase getSQLiteDatabase() {
if(sqliteDatabase==null){
Log.i("getSQLiteDatabase()","Problem, vraci sqliteDatabase = null");
}
else{
Log.i("getSQLiteDatabase()","instance sqliteDatabase vracena bez problemu");
}
return sqliteDatabase;
}
AssyncTask class:
public class OpenDatabaseTask extends AsyncTask {
private Context context;
private SQLiteDB sqliteDB;
public OpenDatabaseTask(Context context,SQLiteDB sqliteDB) {
this.context = context;
this.sqliteDB = sqliteDB;
}
#Override
protected Void doInBackground(Void... params) {
publishProgress();
try {
sqliteDB.createDB();
} catch (IOException e) {
e.printStackTrace();
}
sqliteDB.openDB();
return null;
}
#Override
protected void onProgressUpdate(Void...unused){
Log.i(OpenDatabaseTask.class.toString(), "Spusteno vlakno");
}
}
and Activity (only important part):
private SQLiteDB sqliteDB;
private SQLiteData sqliteData;
private OpenDatabaseTask openDatabaseTask;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.navigatemenu);
sqliteDB = new SQLiteDB(getApplicationContext(), sourceDatabaseName);
openDatabaseTask = new OpenDatabaseTask(getApplicationContext(), sqliteDB);
openDatabaseTask.execute();
protected void onDestroy(){
super.onDestroy();
Log.i("onDestroy()", "Pokus o zavreni databaze");
//here is the problem, database never closed
sqliteDB.close();
}
protected void onStop(){
super.onStop();
Log.i("onStop()", "Pokus o zavreni databaze");
//here is the problem, database never closed
sqliteDB.close();
}
protected void onPause(){
super.onPause();
}
protected void onResume(){
super.onResume();
//Log.i("onResume()", "Pokus o otevreni databaze");
}
}
So when I try use a close() method to closing database, sqliteDatabase is always null and database is never closed. So database is still opened and after calling onDestroy exception occurs.
Thank you for your time and sorry for my english, I'm czech :)

Just a guess (assuming you want to close on destroy)
close the database before calling the super
protected void onDestroy(){
// close befor super is called
sqliteDB.close();
super.onDestroy();
Log.i("onDestroy()", "Pokus o zavreni databaze");
// sqliteDB.close(); // super.onDestroy may already has destroyed the DB
}
Note:
Assuming that the database only exists and is open while the activity is visible you should open the database in onResume() and close it in onPause().
If the database should be open when the code is loaded into memory create it in onCreate and close it in onDestroy
In your example you open it in onCreate and close it in onStop. Problem: when the activity becomes visible for the 2nd time the db is closed.
For details see android activity documentation and look into the Application-Lifecycle at the button.

I have an app with heavy db interaction. I didnt user helper. My db is opened lots of times, in UI thread and in background thread too, but never ever closed. Had no problems so far, dont know if it is the right way to do...

SQLite database leak found

I'm creating an application. I'm getting this error:
11-08 13:46:24.665: ERROR/Database(443):
java.lang.IllegalStateException:
/data/data/com.testproj/databases/Testdb SQLiteDatabase created and
never closed
I can't seem to find the reason for this, as it somethimes shows me the error, sometimes not. Here is my code:
public class SQLiteAssistant extends SQLiteOpenHelper {
public SQLiteAssistant(Context context){
super(context, DB_NAME, null, DB_VERSION_NUMBER);
this.myContext = context;
}
public void openDataBase() throws SQLException{
String myPath = DB_PATH + DB_NAME;
myDataBase = SQLiteDatabase.openDatabase(myPath, null, SQLiteDatabase.OPEN_READWRITE);
}
public void closeDataBase() {
if(this.myDataBase != null) {
if(this.myDataBase.isOpen())
this.myDataBase.close();
}
}
}
}
In another class, I have these queries:
public class Db{
private static SQLiteAssistant sqlite;
public static String getSomeString(Context ctx) {
sqlite = new SQLiteAssistant(ctx);
sqlite.openDataBase();
Cursor cursor = sqlite.myDataBase.rawQuery("SELECT someColumn from SomeTable",null);
if (cursor != null) {
if (cursor.getCount()==1) {
if(cursor.moveToFirst()) {
String testString = cursor.getString(cursor.getColumnIndex("someColumn"));
cursor.close();
sqlite.closeDataBase();
sqlite.close();
return testString
}
}
}
sqlite.closeDataBase();
sqlite.close();
return null;
}
}
My problem is when I start a new activity in which I get an AsyncTask. This task gets data from a web service and accesses the database for the String. Here is the AsyncTask:
protected class BackTask extends AsyncTask<Context, String, String> {
#Override
protected String doInBackground(Context... params) {
try{
//get requeste data from the database
//access the web service
return result;
} catch (Exception e) {
return null;
}
return null;
}
}
If I let the activity take its course, everything goes fine. If I don't and quickly press the back button, I get the error. Any suggestion on how to solve this problem?

Am not sure you're using SQLiteOpenHelper properly... you don't need that myDataBase field, the idea is that it manages your database connection for you. Don't subclass in that way... unless you're doing things in onCreate() etc that aren't posted here it looks like you can just use SQLiteOpenHelper directly, i.e.:
SQLiteOpenHelper sqlite = new SQLiteOpenHelper(ctx, DB_PATH+DB_NAME, null,
DB_VERSION_NUMBER);
Assuming that ending the activity should also stop your background task, I'd recommend calling AsyncTask.cancel(true) from your Activity.onPause(). Ensure the database is cleaned up from onCancelled().
And if your background task is the only thing reading the database then make it own the SQLiteOpenHelper instance. It's easy to get into trouble with static data, so it's best avoided IMHO. I'd do something like this:
protected class BackTask extends AsyncTask<String, Integer, String>
{
private SQLiteOpenHelper sqlite;
public void BackTask(Context ctx) {
sqlite = new SQLiteOpenHelper(ctx, DB_PATH+DB_NAME, null,
DB_VERSION_NUMBER);
}
#Override
protected String doInBackground(String... params)
{
try {
//get requeste data from the database
//access the web service
return result;
} catch (Exception e) {
}
return null;
}
#Override
protected void onCancelled() {
sqlite.close();
}
#Override
protected void onPostExecute(String result)
sqlite.close();
// Update UI here
}
}

I think this part :
cursor.close();
sqlite.closeDataBase();
sqlite.close();
must be in a finally close like
Try{
//Do something
}
catch(){
//Catch exception
}
finally{
//Close cursor or/and eventually close database if you don't need it in the future
}
Also don't forget to close database in onDestroy method .
onCreate(Bundle b){
//create database instance
}
onDestroy{
//close db
}