Develop Reference

Android sqlite insert / multiple reads from many threads

I don't know how to handle this correctly without getting database locked errors.
My app basically downloads many items in batches of 100 rows (~ 60.000 items) and inserts them in the database. Each batch of 100 rows is processed into a transaction.
The main activity allows the user to navigate between screens (fragments) while records are being downloaded and inserted. Most of the other screens contains read data from the database. I get a lot of database lock errors during reading. All readings are done in the main activity (not fragments) in different async tasks
So far I just used the "classic approach"
public class DBAdapter {
public DBAdapter(Context ctx) {
this.context = ctx;
DBHelper = new DatabaseHelper(context);
}
private static class DatabaseHelper extends SQLiteOpenHelper {
DatabaseHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
}
#Override
public void onCreate(SQLiteDatabase db) {
db.execSQL(DB_CREATE_TABLES);
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
Utils.log("Upgrading database from version " + oldVersion + " to " + newVersion + ", which will destroy all old data");
onCreate(db);
}
public DBAdapter open() throws SQLException {
database = DBHelper.getWritableDatabase();
return this;
}
public void close() {
DBHelper.close();
}
Then on my activity's onCreate() I call db = new DBAdapter(context); and each time I am doing an database operation (read/write) I call db.open() and after the insert/read is done I call db.close()
My questions are:
what would be the best approach to this situation ?
Considering I do a lot of write/read would it be better to call db.open on onCreate and db.close() on onDestroy() ? Would this be better than calling open/close each time I need to access the database ?
What do I need to do to avoid database locking on reading ?

I had a exactly similar situation like yours. In addition to what you described, in my app the user also can update the database through input on the screen.
The way I resolved it ( I don't know if it's the best way, but I hardly see any locking issue now)
Make a singleton class derived from SQLiteOpenHelper to make sure only one instance is running at any given time.
Implement ContentProvider class for insert/update/delete/query operations. Make all those functions 'synchronized'
Only close the db in ContentProvider's shutdown function. I do a very frequent db operations, so I don't want to open/close everytime. But I am not sure if it's the correct way of handling it.
Do access DB only through ContentProvider interface from anywhere

A very simple approach, or maybe a workaround is using synchronized methods for opening and closing the database object. I don't really know if it's the best practice, but at least it's simple and easy. Add this methods to your DBAdapter Class, and use them instead of db.open and db.close. The use_count attribute simple holds how many times open has been called. Initialize it with a value of 0. Also, in order to make it work on your solution be sure to pass the same DBAdapter object between the fragments. Don't create a new one everytime :
private int use_count = 0;
public synchronized void doOpen()
{
use_count++;
this.open();
}
public synchronized void doClose()
{
use_count--;
if (use_count == 0)
{
this.close();
}
}

Consider wrapping the SQLite database in a ContentProvider and using CursorLoader to do the queries from the various activities & fragments. This isolates the management of the database from the Activity/Fragment life cycle and can result in many fewer open/close cycles.
You may still run into contention between the reads and writes, but having all the database interaction in the same module should make it easier for you to address these issues.
Some interesting links: http://www.vogella.com/articles/AndroidSQLite/article.html#todo
When to use a Content Provider

Android, How to manage opening/closing database connection when we have multiple tables?

I am looking for best approach to handling database connection. I have designed my application base on following method and works fine. However, sometimes application crashes especially on old model devices. Please look at my following method.
I have a DatabaseHelper class which all of tables will be created here. This class is like this:
public class DatabaseHelper extends SQLiteOpenHelper {
public DatabaseHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
Log.i(TAG, "Object created.");
}
#Override
public void onCreate(SQLiteDatabase db) {
db.execSQL(CRT_TBL_1);
db.execSQL(CRT_TBL_2);
db.execSQL(CRT_TBL_3);
db.execSQL(CRT_TBL_4);
db.execSQL(CRT_TBL_5);
db.execSQL(CRT_TBL_6);
db.execSQL(CRT_TBL_7);
db.execSQL(CRT_TBL_8);
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
Log.w(DatabaseHelper.class.getName(), "Upgrading database from version " + oldVersion + " to " + newVersion + ", which will destroy all old data.");
onCreate(db);
}
}
As you have seen, I have 8 tables which provides data for 8 different activities. Implementation of handler class for one activity is like following code.
public class DatabaseHandler_1 {
private final String TAG = "DatabaseHandler_1 ";
private DatabaseHelper dbHelper;
private SQLiteDatabase database;
public DatabaseHandler_1 (Context context) {
dbHelper = new DatabaseHelper(context);
Log.i(TAG, "Object created.");
}
public void open() throws SQLException {
database = dbHelper.getWritableDatabase();
}
public void close() {
dbHelper.close();
}
// ... Other methods insert, update, query, ...
}
Finally, based on required information one or some of them will be used in each activity.
public class Activity_1 {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
Log.i(TAG, "Try to create activity...");
// Creating Database/Table
dbHandler_1 = new DatabaseHandler_1 (this);
dbHandler_1 .open();
dbHandler_2 = new DatabaseHandler_2 (this);
dbHandler_2 .open();
dbHandler_3 = new DatabaseHandler_3 (this);
dbHandler_3 .open();
}
#Override
protected void onDestroy() {
super.onDestroy();
// Closing database
dbRecipesHandler.close();
dbFavoritsHandler.close();
dbShoppingHandler.close();
}
// Other methods for displaying data...
}
When crashes happens logcat shows the connection is close while my application tries to query in database. I think scenario is like this:
User opens 1'st activity then he goes to second activity and then he goes to third activity. Now, if user clicks on back button third activity will be closed and onDestroy() method of this activity closes database connection. Because, second activity is still in memory, activity assumes that the connection is still open and when it wants to query into database, crash happens.
Logcat shows a message like this: Database connection is close. You wanted to query...
One solution is instead of letting Android OS handles database connection, we manually open connection and use it the manually close it.
Second solution is in Handler class manually open and close connection for each method.
What is you suggestion? did you have any experience about it?
Any suggestion would be appreciated.

In general I would suggest you consider wrapping your data with Content Providers then using Loaders in your activities to mingle data with the UI. It might seem like a lot of work, but among the many benefits of content providers you will not have to manage database connections or cursors within your activities. (Note, your DatabaseHelper class would remain the same)
Having done data both your way as well as with content providers in the different apps I've made, I would say content providers are almost always the way to go. It's always ended up being more maintainable and extendable in my experience, plus you get a lot of baked in features that would be really expensive to build yourself.
That said, if you really prefer the more direct approach you're currently using, I would at the least not tie the opening and closing of your connections to the activity's life cycle. Only open those connections at the instant you actually need data, get that data, then immediately close any cursors along with the connection itself.

Finally I solved the problem. I thought that I should open database one time in activity. I tried to open database again in onResume() method of each activity and closing it in onPause() of each activity instead of closing in onDestroy() method.
So far problem solved. Hope to don't see it again. If i face it again then will update/remove answer.

How to Open/Close SQLite db in Android Properly

I have an app that functions properly and does not force close or crash. But when I look at LogCat, it occasionally gives me this:
05-20 15:24:55.338: E/SQLiteDatabase(12707): close() was never explicitly called on database '/data/data/com.---.--/databases/debt.db'
05-20 15:24:55.338: E/SQLiteDatabase(12707): android.database.sqlite.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here
a little ways down...
05-20 15:24:55.338: E/System(12707): Uncaught exception thrown by finalizer
05-20 15:24:55.338: E/System(12707): java.lang.IllegalStateException: Don't have database lock!
I am not sure when I should be opening and closing my Database?
I have a Main activity that is simply a splash screen. It then goes into an activity that calls a ListView using info from the DB; so it is at this activity where the DB is first opened.
There is also one other Activity where the DB is required that branches off the one with the ListVeew. When am I supposed to be opening and closing this? Word seems to be that I simply need to open once, and then close when the app is "paused", "stopped" or "destroyed".
If this is the case, where do I put the db.close() method... in the Splash Screen Main Activity where onStop, etc is located? or the same Activity as the one that opens the DB? or.. is there another place?
UPDATE:
This is the line in code that the error keeps pointing to:
public void open() throws SQLException {
database = dbHelper.getWritableDatabase();
}

If you're using an instance of a DatabaseHelper class, and after you initialize the DBHelper object, every time you do work in the database you should call the open method before you do work, then create a new cursor, query the database, do work with the information you just stored in the cursor, when you're done close the cursor, then close the database. For example if you wanted to grab every item in a database you would do something like :
...
DataBaseHelper db = new DataBaseHelper(this);
...
db.open();
Cursor cursor = db.getAllItems();
maxCount = cursor.getCount();
Random gen = new Random();
row = gen.nextInt(maxCount); // Generate random between 0 and max
if (cursor.moveToPosition(row)) {
String myString = cursor.getString(1); //here I want the second column
displayString(myString); //private method
}
cursor.close();
db.close();
getAllItems is a public method in my DatabaseHelper, it looks like this in case you were wondering
public Cursor getAllItems() {
return db.query(DATABASE_TABLE,
new String[] {
KEY_ROWID,
KEY_NAME
},
null,
null,
null,
null,
null);
}
This is how I access my database and I haven't gotten any of the errors you've got, and it works perfectly.

I used to do the way #Shikima mentioned above but in complex applications which has many background services, multi-threading,etc it can get real tiresome when you have to manage many database instances and on top of that, opening and closing them.
To overcome this, I used the following method and it seems to be working fine.
1.
Declare and initialize an instance of YourDBHelperClass in your Application base class like this :
public class App extends Application {
public static YourDBHelperClass db;
#Override
public void onCreate() {
super.onCreate();
db = new YourDBHelperClass(getApplicationContext());
db.open();
}
}
2.
In you activity, or any other place you want to use the DB, initialize the YourDBHelperClass object like this :
YourDBHelperClass db = App.db;
And then you can use the database anyway you want without having to worry about opening and closing it manually each time. The SQLiteOpenHelper takes care of the closing when the Application is destroyed

You are probably not handling your database correctly; you are opening more database instances than you are closing.
There are a number of design patterns you can follow to correct this behavior. You might want to consult this answer for more information.

Android threading and database locking

We are using AsyncTasks to access database tables and cursors.
Unfortunately we are seeing occasional exceptions regarding the database being locked.
E/SQLiteOpenHelper(15963): Couldn't open iviewnews.db for writing (will try read-only):
E/SQLiteOpenHelper(15963): android.database.sqlite.SQLiteException: database is locked
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.native_setLocale(Native Method)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.setLocale(SQLiteDatabase.java:1637)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.<init>(SQLiteDatabase.java:1587)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:638)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:659)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:652)
E/SQLiteOpenHelper(15963): at android.app.ApplicationContext.openOrCreateDatabase(ApplicationContext.java:482)
E/SQLiteOpenHelper(15963): at android.content.ContextWrapper.openOrCreateDatabase(ContextWrapper.java:193)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:98)
E/SQLiteOpenHelper(15963): at android.database.sqlite.SQLiteOpenHelper.getReadableDatabase(SQLiteOpenHelper.java:158)
E/SQLiteOpenHelper(15963): at com.iview.android.widget.IViewNewsTopStoryWidget.initData(IViewNewsTopStoryWidget.java:73)
E/SQLiteOpenHelper(15963): at com.iview.android.widget.IViewNewsTopStoryWidget.updateNewsWidgets(IViewNewsTopStoryWidget.java:121)
E/SQLiteOpenHelper(15963): at com.iview.android.async.GetNewsTask.doInBackground(GetNewsTask.java:338)
E/SQLiteOpenHelper(15963): at com.iview.android.async.GetNewsTask.doInBackground(GetNewsTask.java:1)
E/SQLiteOpenHelper(15963): at android.os.AsyncTask$2.call(AsyncTask.java:185)
E/SQLiteOpenHelper(15963): at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:256)
E/SQLiteOpenHelper(15963): at java.util.concurrent.FutureTask.run(FutureTask.java:122)
E/SQLiteOpenHelper(15963): at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:648)
E/SQLiteOpenHelper(15963): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:673)
E/SQLiteOpenHelper(15963): at java.lang.Thread.run(Thread.java:1060)
Does anybody have a general example for code which writes to a database from a different thread than the one reading and how can we ensure thread safety.
One suggestion I've had is to use a ContentProvider, as this would handle the access of the database from multiple threads. I am going to look at this, but is this the recommended method of handling such a problem? It seems rather heavyweight considering we're talking about in front or behind.

We used a ContentProvider in the end. This appeared to clear up the problems.

I solved this same exception just by making sure all my database opens have closes, and (more importantly) to assure this, making the scope of each database instance local ONLY to the method that needs it. ContentProvider is a good, safe class to use when accessing a db from multiple threads, but also make sure you're using good db practices:
Keep db instances local (no SQLiteDatabase class members!)
call close() on the db in the same method in which it's opened
call close() on the cursors you get from the db
listen to LogCat for any complaints that SQLiteDatabse might have

Before some code, let's resume some of the approachs:
Semaphores: by far the best solution presented. It goes in the heart of the problem: resource sharing! It will treat the locking of the database access, avoiding conflicts (database is locked).
Java synchronization: A kind of semaphore implementation, but less sofisticated. Using synchronized you will not easily solve some cases involving transactions.
ContentProvider: implement ContentProvider solve the problem only for some cases (or sweep the problem under the carpet). You'll yet face the same issues. The difference is that ContentProvider pattern will guide you to not make some commom mistakes when accessing Sqlite database. The ContentProvider docs says: "You don't need a provider to use an SQLite database if the use is entirely within your own application."
Almost mandatory: keep db instances local, call close() on the db in the same method in which it's opened using finally statements, close() on the cursors using finally statements, etc are almost mandatory to avoid problems using Sqlite.
Let's show an example of the semaphore solution presented by Moss, which I took from CL. and improoved to cover transactions.
class DataAccess {
private final ReentrantReadWriteLock rwl = new ReentrantReadWriteLock();
private final Lock r = rwl.readLock();
private final Lock w = rwl.writeLock();
public Data readSomething(int id) {
Cursor c = null;
r.lock();
try {
c = getReadableDatabase().query(...);
return c.getString(0);
} finally {
if (c != null) c.close();
r.unlock();
}
}
public void changeSomething(int id, int value) {
w.lock();
try {
getWritableDatabase().update(...);
} finally {
w.unlock();
}
}
private void beginTransactionWithSemaphores() {
getWritableDatabase().beginTransactionWithListener(new SQLiteTransactionListener() {
#Override
public void onBegin() {
w.lock();
}
#Override
public void onRollback() {
w.unlock();
}
#Override
public void onCommit() {
w.unlock();
}
});
}
}

Take into account that SQLite databases are file based and are not intended to be able to be accessed in a multi-process way. The best procedure on mixing SQLite with multi-processing is using semaphores (aquire(), release()) in each database related access.
If you create a Db wrapper that aquires/releases a global semaphore your DB access will be thread safe. Indeed this means that you could get a bootleneck because you are queueing the access to the DB. So in addition you could only wrap the access using semaphores if it's an operation that alters the database, so while you are alterin the db no one will be able to access it and wait until the write process has been completed.

We could not share Db connection with multiple thread to perform read and write operation in database simultaniously.We will have to make single object of DB using syncronization concept and we will perform one task at a time .We will use singleton pattern to make the DB object and it will be share within multiple threads.At a time will perform single task . then we will start other task or any operation on DB .
Content provider is not the solution of DB locking issue .
import java.util.concurrent.atomic.AtomicInteger;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteOpenHelper;
import android.util.Log;
public class DatabaseManager {
private AtomicInteger mOpenCounter = new AtomicInteger();
private static DatabaseManager instance;
private static SQLiteOpenHelper mDatabaseHelper;
private SQLiteDatabase mDatabase;
//private static String DB_PATH = "";
// private static String DB_NAME = "xyz.db";// Database name
private static String dbPathh;
public static synchronized void initializeInstance(SQLiteOpenHelper helper,
String dbPath) {
if (instance == null) {
instance = new DatabaseManager();
mDatabaseHelper = helper;
dbPathh=dbPath;
}
}
public static synchronized DatabaseManager getInstance() {
if (instance == null) {
throw new IllegalStateException(DatabaseManager.class.getSimpleName() +
" is not initialized, call initializeInstance(..) method first.");
}
return instance;
}
public synchronized SQLiteDatabase openDatabase(String thread) {
if(mOpenCounter.get() == 0) {
// Opening new database
// mDatabase = mDatabaseHelper.getWritableDatabase();
MyLog.e("Path Of DataBase", dbPathh);
// mDatabase=mDatabaseHelper.getWritableDatabase();
mOpenCounter.incrementAndGet();
mDatabase=SQLiteDatabase.openDatabase(dbPathh, null,
SQLiteDatabase. CREATE_IF_NECESSARY|SQLiteDatabase.OPEN_READWRITE);
MyLog.e("Open Data Base", " New Connection created" +thread);
}
else{
MyLog.e("Open Data Base", " Old Connection given " +thread);
}
// Toast.makeText(NNacres.getConfig(), "open conn: present connection =
" +mOpenCounter.get(), Toast.LENGTH_LONG).show();
return mDatabase;
}
public synchronized void closeDatabase() {
MyLog.e("Close db connection", ""+mOpenCounter.get());
if(mOpenCounter.get() == 1) {
// Closing database
mDatabase.close();
mOpenCounter.decrementAndGet();
Log.e("DB CLOSED", "DONE");
}
//Toast.makeText(NNacres.getConfig(), "close conn: after close =
" +mOpenCounter.get(), Toast.LENGTH_LONG).show();
}
}
and write this method in your YourSQLiteDataABse helper class which extends SQLiteOpenHelper Class
public SQLiteDatabase getWritableDatabase() {
DatabaseManager.initializeInstance(this,"data/data/your packgae name/databases/xyz");
return DatabaseManager.getInstance().openDatabase(getClass().getSimpleName());
}
public static String getMyDbPath(String DB_NAME, Context context) {
String myDbPath = context.getDatabasePath(DB_NAME).getPath();
MyLog.e("DB Path: "+myDbPath);
return myDbPath;
}

You must be calling getWritableDatabase() from a function rather then the constructor of the db helper class. If the db helper class object is created with SQLiteDatabase.openOrCreateDatabase(DB_PATH, null); or similar and then getWritableDatabase() is called from a function, it will try to make a synchronous call to DB causing a DB lock exception.

Are you talking of a single user action that, inside your program, causes multiple threads to be run, more than one of which may be accessing the database in update mode ?
That's bad design, period. There is no way for you to know in which order the threads will be scheduled by your OS (/VM), and therefore there is no way for you to know in which order the database accesses will happen, and that is very likely to imply that there is no way for you to know that database accesses will always happen in the order that you are expecting.
All database accesses generated by/coming from some user action should all be done in one single thread.

Problem in creating a database in SQLite in Android

Hi I am new to android and I have a problem in creating a database.
public class database extends ListActivity {
/** Called when the activity is first created. */
private final String MY_DATABASE_NAME = "myCoolUserDB.db";
private final String MY_DATABASE_TABLE = "t_Users";
Context c;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
ArrayList<String> results = new ArrayList<String>();
setContentView(R.layout.main);
SQLiteDatabase mydb=null;
try
{
mydb.openOrCreateDatabase(MY_DATABASE_NAME, null);
} catch(Exception e){}
}
}
When I run this code it throws a run time exception. Please help me.

If you are going to call a static method like openOrCreateDatabase, do it on the class (SQLiteDatabase.openOrCreateDatabase(...)), not an instance. It's a lot clearer - the way you've done it looks like you're calling an instance method, so looks like a sure NullPointerException, which of course is misleading.
As someone else has stated, the stack trace would be the most useful thing when asking for help with an exception.
(Almost) never catch an exception without at the very least logging it. Don't just do nothing with it. There are of course exceptions to every rule, but let's not go there for the moment. Anyway, if you don't at least log it, you're just throwing away information that would tell you what went wrong when everything goes to crap later.
You shouldn't be using that method directly, and should instead be extending SQLiteOpenHelper . See the android developers page on data storage to get started (I'd post a link but apparently I'm only allowed one link in my post ?!), and since you've probably had to download the SDK to get going, look in the samples that come with it for the Notepad sample application. That contains a NotePadProvider class, which is a good example of both a content provider and database access, which often go hand-in-hand on android. I'd suggest compiling that application and making some simple changes to it before you jump into making your own one.

For working with sqlite database you need to create class extended from SQLiteOpenHelper:
private class DBHelper extends SQLiteOpenHelper {
public DBHelper(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
}
#Override
public void onCreate(SQLiteDatabase db) {
db.execSQL(CREATE_TABLES);
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
db.execSQL(UPGRADE_TABLES);
}
}
Then you can get access to db using DbHelper object:
DBHelper dbHelper = new DBHelper(Activity.this);
SQLiteDatabase db = dbHelper.getReadableDatabase();

I run into the same problem. It figures out that two bugs happens during development
dir "databases" was not existent
accendently ".db" was created as directory.
They following code cover both
File dbFile = getDatabasePath ("abc.db");
if (dbFile.isDirectory ()) {
dbFile.delete();
}
if (! dbFile.exists()) {
String path = dbFile.getParent ();
new File (path).mkdirs ();
}
database = SQLiteDatabase.openDatabase (dbFile.getAbsolutePath (), this, SQLiteDatabase.OPEN_READWRITE | SQLiteDatabase.CREATE_IF_NECESSARY);
Hope this helps
I think SQLiteOpenHelper is only useful for "single table" databases. For multiple table applications I consider directly using SQLiteDatabase fit better to a good architecture.

This is a simple post which tells you how to insert data in to a SQLite database in Android and further more this links shows you how to retrieve data from a SQLite database in Android .