I'm having SQLite trouble in a multithreaded application. I have an Activity which uses a subclass of AsyncTaskLoader to perform some data import from a file (specified by an Uri as it comes from Android Storage Access Framework), and when the loader is started and does its work (it writes to the database) and the device is rotated, I get a 'android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5)' exception. I know what the problem is (multiple SQLiteOpenHelpers accessing a database), but I am not sure how I'm supposed to fix this. Here is my code:
Activity:
private void importTests(Uri uri) {
Bundle loaderArgs = new Bundle();
loaderArgs.putParcelable(URI_IMPORTER_LOADER_ARG, uri);
getLoaderManager().initLoader(0, loaderArgs, this).forceLoad();
}
#Override
public Loader<Exception> onCreateLoader(int id, Bundle args) {
Uri uri = args.getParcelable(URI_IMPORTER_LOADER_ARG);
return new ImporterLoader(this, uri, dbHelper);
}
#Override
public void onLoadFinished(Loader<Exception> loader, Exception exception) {
getLoaderManager().destroyLoader(loader.getId());
if (exception != null) {
// import failed, show toast
} else {
// init Ui
}
}
#Override
public void onLoaderReset(Loader<Exception> loader) {
// nothing
}
(Note: I don't create the Loader in onCreate; rather, I do it on demand when the import functionality is invoked, and destroy it as soon as it is ready. I'm not sure if this is a correct way of using loaders.)
Now more detailed information about the problem:
when the activity is started, it creates a DbHelper (which is a SQLiteOpenHelper subclass), stores it in a member field, and reads the database (using getReadableDatabase(), but most likely it will be writable anyways) to initialize the Ui (show a list of items etc.)
if an import is triggered, a Loader is created and it gets the DbHelper from the activity; this import reads a Uri using its InputStreams, does a bit of parsing and writes rows to the database
when, during import, the device is rotated, the following happens: the loader is still going on (I don't want to destroy it, I want it to finish its task and trigger updating the Ui when it's done, possibly in the new, rotated, activity - that's the very reason I use a Loader), and it still uses its instance of DbHelper and its open connection; however, the activity is destroyed and then created again, creating another instance of DbHelper, which again tries to read the database to initialize the Ui
as a result, there are 2 DbHelpers with one open connection each, and the second one from the new activity instance throws the SQLiteDatabaseLockedException from getReadableDatabase()
Until now, I've been using a new DbHelper in every activity in the app as it wasn't possible to use the db from multiple threads, but now that I implemented the first background use case, all hell breaks loose, of course. So actually, it seems to be possible to have multiple connections opened at the same time, as long as they are not accessed simultaneously, as I had many activities stack on top of each other, each having its own helper, and (wrongly?) didn't close them in onPause and open in onResume.
So, the question is if I'm doing anything fundamentally wrong here? Based on my research, there seems to be only 2 solutions: create a ContentProvider which will manage the database (I would like not to be forced to do this as I really don't need no CP), or somehow keep only one DbHelper/connection for the whole app. How I do it is unimportant, but as of now my favorite would probably be a singleton (yuck) initialized in a custom Application subclass. In the future I would like to try Dagger so I would probably make it a #Singleton, but not yet.
Edit: unfortunately, my ui still blocks - the importer transaction blocks any db reads from the ui initialization methods. I guess I need to figure out a way to start reader transactions which are not blocked by the single write tx.
Edit 2: I was able to successfully unblock readers while a writer is working. To do this:
I call
setWriteAheadLoggingEnabled(true);
in the DbHelper constructor
use
db.beginTransactionNonExclusive();
instead of
db.beginTransaction();
in the ImporterLoader (the thing which performs the writes)
Related
I am looking for how to share functions and data across multiple activities within a single application. I researched the daylights out of it and find some ideology war between overriding the extend for the application and doing a singleton, neither of which I can find examples sufficient to make me understand. Basically I want to share data and share functions. All activities need the same functions and data so this is not one activity sharing data with another activity. It is all activities needing to have access to the same functions and data.
What I want to know is what is the way to go and how do I do it. I need to see what I need to do in my 34 activities, what the class that is going to be common looks like, and what the Manifest entry needs to be. I also need to be sure the common data area will not be closed by the OS.
This is my first Android - Java program and now find my 15,000 line, 34 activity application needs some structure. I know, should have done things differently but the app works really well with two exceptions. One is that it is structurally a mess. Two is that the fact it is a mess is making it hard to fix one behavior I would like to fix.
This is a GPS based application for racing sailboats. It is timing critical and every activity basically runs a once a second loop inside the location manager onLocationChanged function. That part is fine and I do not want to put the GPS code in one place. The problem is that most activities need to filter the data so a lot of code is copied and pasted to the activities. The filter needs history so it needs to remember a state. There are other functions that are used by several activities so these have been copied as well. Think of a function that averages the last three GPS speed readings. It needs to save some history, do its thing, and give a result. All activities need to do the exact same thing. All this works but the problem is that the averaging starts over every time I switch activities because every activity has its own filter. That gives a glitch in the data that I need to get rid of. I need common place to save the data and hopefully a common place to run the filtering and other functions that are common. If every activity can call the filter function that is using common state data, there will be no glitch across activity changes.
I would appreciate some guidance.
Why you don't just make a Class with only static functions, passing needed Parameters? An example if you want to show an ErrorDialog
public class SharedHelper{
public static Dialog showErrorDialog(Context ctx, String message, String title, DialogInterface.OnClickListener okListener, DialogInterface.OnClickListener cancelListener){
AlertDialog.Builder builder = new AlertDialog.Builder(ctx);
builder.setMessage(message).setTitle(tilte);
if (okListener != null){
builder.setPositiveButton(R.string.button_positive, okListener);
}
if (cancelListener != null){
builder.setNegativeButton(R.string.button_negative, cancelListener);
}
return builder.show();
}
}
Singletons are (from my point of view) one of the uglyest design pattern and will bite you sooner or later. Putting anything in Application requires you to cast it everytime to the Special Application class you designed. A class with only statics however is very flexible in its usage and doesn't need an instance to work.
For the storage-issue:
lookup "SharedPreferences" & "SQLite" and decide afterwards which storage-type suits your needs more.
For the methods-issue:
This question is a bit more complex and there are different ways to do it. For example you could write a parent-class that implements all your globally needed questions and you let all your activity-classes inherit from it.
public class MyParentActivity extends Activity {
public void myMethod() {
}
}
and:
public class Activity1of34 extends MyParentActivity {
myMethod();
}
I think what this comes down to is not an Android problem but an Object-Oriented Programming problem. If I understand the situation correctly, I'm betting the best solution would be to take your shared filter and create a new Filter class that is instantiated within each Activity (this is likely more manageable than a singleton, but not having seen your use case, it's hard to say for sure). If you need to centrally track the averaging, you can simply create a static variable within the Filter class that maintains the same value during the life of the application. If you really want to maintain that average (even past the application's current lifecycle), you can persist it in a database or other local data options. However, I don't see any reason to put everything in a singleton just to maintain that average. Singletons (and all static data structures) can be potentially troublesome if used incorrectly.
I, for one, do not mind the singleton pattern. Of course as everything else it should not be abused.
This is the construction I use for my shared objects. My app is divided into modules this way but can just as well be used in your case.
public class SharedDataObject {
private Context context;
private static SharedDataObject instance;
public static SharedDataObject getInstance() {
if (instance == null) throw new RuntimeException("Reference to SharedDataObject was null");
return instance;
}
public static SharedDataObject createInstance(Context context) {
if (instance != null) {
return instance;
}
return instance = new SharedDataObject(context.getApplicationContext());
}
// notice the constructor is private
private SharedDataObject(Context context) {
this.context = context;
}
...
public void myMethod() {
// do stuff
}
}
Notice that it uses the application context, that means among other things, means that the context owned by SharedDataObject cannot be used for GUI operations. But, the context will live for the entire lifetime of the application, which is nice.
Furthermore I hate having to pass a context everytime I wish to call methods on my SharedDataObject, thus I have a splashscreen calling SharedDataObject.createInstance() on all my modules.
Once an instance is create, I can call:
SharedDataObject.getInstance().myMethod();
Anywhere in my code, regardless of a context being present or not (from the place calling this code that is).
I'm getting the following error when using the encrypted SQLCipher database in my Android app, but only off and on:
net.sqlcipher.database.SQLiteException: not an error
at net.sqlcipher.database.SQLiteDatabase.dbopen(Native Method)
at net.sqlcipher.database.SQLiteDatabase.<init>(SQLiteDatabase.java:1950)
at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:900)
at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:947)
at net.sqlcipher.database.SQLiteOpenHelper.getReadableDatabase(SQLiteOpenHelper.java:195)
at com.android.storage.DatabaseHelper.getReadable(DatabaseHelper.java:99)
...
I've got the proper files in the assets/ and libs/ folders because the database works fine most of the time. However, every once in awhile I'll see this error. I've seen this twice now on my phone and it's always been after resuming the app after hours of inactivity (I check for user's oauth token in db if it gets cleared from memory).
I call "SQLiteDatabase.loadLibs(this)" only from the Application::onCreate() method so my hunch is that this isn't getting called on a resume and is throwing the error. Does this sound possible? If so, where should I call loadLibs? A user could enter the app in any activity and I access the db if the token isn't in memory. I see my options as either calling loadLibs on each Activity::onCreate or calling it each time I attempt to open the db. Would it cause any harm or performance issues if I called it multiple times like this?
You might consider moving the SQLiteDatabase.loadLibs(this); to your application subclass of net.sqlcipher.database.SQLiteOpenHelper. You can then pass the static instance of your Application subclass as its argument. Something like the following might be an example:
public class SchemaManager extends net.sqlcipher.database.SQLiteOpenHelper {
private static SchemaManager instance;
public static synchronized SchemaManager getInstance() {
if(instance == null) {
SQLiteDatabase.loadLibs(YourApplication.getInstance());
instance = new SchemaManager(…)
}
return instance;
}
}
With regard to the exception that was provided, the Java routine calls into a JNI layer that calls sqlite3_open_v2, setting the soft heap limit and setting the busy timeout. I would suggest adding logging locally to verify you are passing a valid path and a non null passphrase when attempting to acquire the SQLiteDatabase instance when you get a crash. Calling SQLiteDatabase.loadLibs(this); multiple times shouldn't cause a noticeable performance impact, much of what occurs are calls to System.loadLibrary(…) which get mapped into Runtime.getRuntime().loadLibrary(…), once a dynamic library has been loaded, subsequent calls are ignored.
I'm using the SQLite Asset Helper library to handle the dirty work of setting up and upgrading my app's database. It works really well but unfortunately I have yet to figure out a way to notify the user when the library:
A) loads the database for the first time (by unzipping it from the \assets\databases\ folder)
or
B) upgrades the database (using information in an upgraded database file in \assets\datates)
I tried putting this code in my app's main Activity.onCreate(), thinking I could load the database (if it didn't exist) on the main thread while distracting the user with a un-dismiss-able AlertDialog:
File dbFile=this.getDatabasePath("gShoJMDict");
Boolean dbExists = dbFile.exists();
Log.i("ActivityStartScreen", String.valueOf(dbExists));
if(!dbExists)
{
DialogFirstRun dialogFirstRun = new DialogFirstRun();
dialogFirstRun.show(getSupportFragmentManager(), "dialogFirstRun");
dialogFirstRun.setCancelable(false);
DictHelper helper = new DictHelper(this);
helper.getReadableDatabase();
helper.close();
dialogFirstRun.dismiss();
}
Unfortunately it appears (based on LogCat entries) that SQLite Asset Helper checks to see if the database exists well before onCreate(), so by the time the above chunk of code runs, the database already exists so the dialog never shows up.
I'm using a ContentProvider, and I've verified that I'm only calling getReadableDatabase() from within query() or update(). My ContentProvider's onCreate() looks like this...
#Override
public boolean onCreate()
{
// Load our database
gdb = new JMDictHelper(getContext());
return true;
}
...but despite moving gdb = new JMDictHelper(getContext()); into query() or update(), SQLite Asset Helper library still loads the database well before I can notify the user.
What can I do in this situation to intercept the initial setup or upgrade of the database and notify the user that the app is busy performing these tasks? Right now the app just sits there doing nothing until the library finishes - that's fine for testing since I know to expect it, but I can't leave it like that when I'm ready for the app to go live.
Just in case you're still wondering about this or anyone else has this question, here's a solution to a similar problem that I just implemented. SQLiteAssetHelper doesn't copy the database until you call getReadableDatabase() or getWritableDatabase(). If one of those methods is being called before MainActivity.onCreate(), it might be because you're calling it in a ContentProvider's onCreate() method. Here's the method description from the documentation:
Implement this to initialize your content
provider on startup. This method is called for all registered content
providers on the application main thread at application launch time.
It must not perform lengthy operations, or application startup will be
delayed.
You should defer nontrivial initialization (such as opening,
upgrading, and scanning databases) until the content provider is used
(via query(Uri, String[], String, String[], String), insert(Uri,
ContentValues), etc). Deferred initialization keeps application
startup fast, avoids unnecessary work if the provider turns out not to
be needed, and stops database errors (such as a full disk) from
halting application launch.
If you use SQLite, SQLiteOpenHelper is a helpful utility class that
makes it easy to manage databases, and will automatically defer
opening until first use. If you do use SQLiteOpenHelper, make sure to
avoid calling getReadableDatabase() or getWritableDatabase() from this
method. (Instead, override onOpen(SQLiteDatabase) to initialize the
database when it is first opened.)
You can make your calls to getReadableDatabase() or getWritableDatabase() when your ContentProvider is actually accessed, as described here.
Now that your ContentProvider isn't holding things up, you can check whether the database exists in MainActivity.onCreate() and, if necessary, copy it in a background thread while displaying a ProgressDialog:
// Create an AsyncTask to copy the database in a background thread while
// displaying a ProgressDialog.
private class LoadDatabaseTask extends AsyncTask<Context, Void, Void> {
Context mContext;
ProgressDialog mDialog;
// Provide a constructor so we can get a Context to use to create
// the ProgressDialog.
LoadDatabasesTask(Context context) {
super();
mContext = context;
}
#Override
protected void onPreExecute() {
super.onPreExecute();
mDialog = new ProgressDialog(mContext);
mDialog.setMessage(mContext.getString("Loading database..."));
mDialog.show();
}
#Override
protected Void doInBackground(Context... contexts) {
// Copy database.
new MyAssetHelper(contexts[0]).getReadableDatabase();
return null;
}
#Override
protected void onPostExecute(Void result) {
super.onPostExecute(result);
mDialog.dismiss();
}
}
#Override
protected void onCreate(Bundle savedInstanceState) {
// ...
// Install databases if necessary.
File database = getDatabasePath(DB_NAME);
if (!database.exists()) {
new LoadDatabaseTask(this).execute(this, null, null);
}
// ...
}
Source code tells, that AssetHelper correctly inherits from SqliteOpenHelper. Without any of your code or logcat to look into, the only sane place where your database can be created before onCreate of main Activity is some class static initialization (if you are not using Application object). Check for that.
As for notification: use EmptyView placeholder when data is not ready and use Loaders, because:
they do not block user interface: user can see in EmptyView that the data is processed and even can leave activity for a while.
they do not die on the destruction of an Activity and can be reused.
they are Standard.
Dear Fellow Android Developers!
EDIT:
Thank you all for your answers. I see from many of them that it seems to be common (and accepted) practice to write your own close() method in your database adapter. Fair enough.
But how does that work with a ContentProvider? Usually when querying my database through my ContentProvider I simply issue something like:
Cursor managedCursor = managedQuery(...);
I don't see how I, with this methodology, can access the custom close() method in my custom ContentProvider implementation. Should I instead, from my Activity, do something like:
MyCustomProvider myProvider = (MyCustomProvider) getContentResolver();
and then:
myProvider.query(...);
myProvider.close();
And above all; is this at all necessary (as of point 2 below)?
END EDIT
To a certain degree I must say that I get the concept of the SQLiteOpenHelper, what it is, how it's used and so. I even use it on a regular basis when I write my own ContentProvider's.
The thing is that I'm not sure what to do with the SQLiteDatabase object, returned by the myOpenHelper.getWritableDatabase() (or the myOpenHelper.getReadableDatabase() function for what matters) when I'm done with it.
According to Android ContentProvider.onCreate() documentation:
You should defer nontrivial initialization (such as opening, upgrading, and scanning databases) until the content provider is used (via query(Uri, String[], String, String[], String), insert(Uri, ContentValues), etc).
[...]
If you do use SQLiteOpenHelper, make sure to avoid calling getReadableDatabase() or getWritableDatabase() from this method. (Instead, override onOpen(SQLiteDatabase) to initialize the database when it is first opened.)
The above gives me a hint where to initialize the database (the query(...), insert(...), etc functions), but it doesn't tell me anything on how to treat the created SQLiteDatbase object when I've finished using it.
Should I save it as a member variable of my ContentProvider implementation (and treat it much like a "private singleton" for future use)?
Should I just leave it when exiting the query(...), insert(...), etc. functions and trust that the SQLiteOpenHelper will manage it for me in future calls?
[Insert your alternative point-of-view here]
Being the confiding (or lazy) developer I've implemented my code according to the second alternative above. But I can't get rid of the creepy feeling that I'm neglecting something important.
It depends on what you're doing with your database. If you just do an insert, delete or select where you get an business object back, then you can close the database right after using it. As far as I know it is designed that you simply close it and request a new one when ever you need it.
But be careful when you're working with a cursor then you have to keep the database open as long as the cursor is in use. Otherwise the application will crash when the cursor has to reload data.
I guess you should close it, for example in onDestroy() of an activity that is using it.
So in my DBAdapter class I have:
/**
* Close the database
*/
public void close() {
mDb.close(); //mDb was obtained using mDbHelper.getWritableDatabase();
}
And in my activity:
public void onCreate(Bundle bundle){
...
mDBAdapter = new DBAdapter(this);
// Open or create the database
mDBAdapter.open();
}
#Override
public void onDestroy() {
// Close the database
mDBAdapter.close();
super.onDestroy();
}
Not sure if this is suitable for your provider concept.
If you check the example of use for that object in the API of Android, you can see the object is just used, but no close is necesary.
They implement the method close() though, but I havent seen they use it.
I am creating an app which allows for many different Activities to be started from a TabActivity(up to ~25). Most of the activities require data from the sqlite database, so when onCreate is run, an AsyncTask creates an SQLiteOpenHelper object(which will open a readable/writable database), runs a query, data is retrieved, and everything is then closed.
i was just testing messing around to see if i could break something, so i added every Activityto the TabActivity's TabHost. I then started mashing each tab as quickly as possible.
i noticed that very quickly i began to see in the LogCat: Caused by: android.database.sqlite.SQLiteException: database is locked: BEGIN EXCLUSIVE; and the app proceeded to die.
Typically there will only be about 4-6 tabs(i can just limit the user anyway) for the TabHost. I haven't been able to break anything with a small amount of tabs to mash, but i am still worried that maybe i am accessing the database in a poor way.
How can i prevent my SQLiteDatabase objects to cause a lock?
If i create a ContentProvider will that eliminate the possibility of database locking?
Do you have any suggestions for changes I could make for accessing data from an SQLiteDatabase?
I ended up taking the approach of using the Application class and storing 1 SQLiteOpenHelper and trying my best to keep it synchronized. This seems to be working great - i put all my 25 activities in the TabHost and mashed away on them with no errors.
I am calling ((SQLiteDbApplication)getApplication()).setDbHelper(new DBHelper(this, Constants.DB_NAME, null, Constants.DB_VERSION_CODE)); method(shown below) in every onCreate() in my activities
Any further suggestions to this approach or to the changes i made using this Application class?
import android.app.Application;
import android.database.sqlite.SQLiteDatabase;
public class SQLiteDbApplication extends Application {
private DBHelper dbHelper;
private SQLiteDatabase db;
public synchronized DBHelper getDbHelper() {
db = dbHelper.getDatabase();//returns the already opened database object
while(db.isDbLockedByCurrentThread() || db.isDbLockedByOtherThreads());
return dbHelper;
}
public synchronized void closeDb() {
if(null != dbHelper)
dbHelper.close();
if(null != db)
db.close();
}
#Override
protected void finalize() throws Throwable {
if(null != dbHelper)
dbHelper.close();
if(null != db)
db.close();
super.finalize();
}
public synchronized void setDbHelper(DBHelper dbHelper) {
if(null == this.dbHelper) {
this.dbHelper = dbHelper;
this.dbHelper.setDb(this.dbHelper.getWritableDatabase());//creates and sets the database object via getWritableDatabase()
}
}
}
If you are to worried about all the database connections try to limit yourself to one SqliteOpenHelper and be sure to wrap a synchronization layer around it.
You can extend the application class and then call getApplication and cast the object you get into your application. Now you can store a SqliteOpenHelper in this application class and build your own thread safe access method to the database connection.
If you are using AsyncTask in all of your onCreate methods and you are experiencing problems with a lot of tabs these problems can also occur with a slower device, a faster user or a database that is grown big over the time of usage.
Depending on the use case of you app you can go the save way and go through all the effort and pain of threading and locking, or you can just publish the app with a number of tabs that never produced the error and be sure to catch the database exception and send yourself a notification (for example through google analytics) to test if the threading problem does occur in real life usage of the app.
All activity callbacks happen on the main thread, so in the scenario you describe there is no multi-threading going on, no matter how many activities or tabs you have.
ContentProvider doesn't provide any locking. In fact, it can introduce multithreading where you wouldn't already have it because it allows other processes to make calls in to your own process, and when that happens the call is dispatched from a separate thread in your process (not on the main UI thread).
Of course if you create your own threads, then you will also have multi-threading going on.