Develop Reference

SQLite trouble with multiple concurrently open connections

I'm having SQLite trouble in a multithreaded application. I have an Activity which uses a subclass of AsyncTaskLoader to perform some data import from a file (specified by an Uri as it comes from Android Storage Access Framework), and when the loader is started and does its work (it writes to the database) and the device is rotated, I get a 'android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5)' exception. I know what the problem is (multiple SQLiteOpenHelpers accessing a database), but I am not sure how I'm supposed to fix this. Here is my code:
Activity:
private void importTests(Uri uri) {
Bundle loaderArgs = new Bundle();
loaderArgs.putParcelable(URI_IMPORTER_LOADER_ARG, uri);
getLoaderManager().initLoader(0, loaderArgs, this).forceLoad();
}
#Override
public Loader<Exception> onCreateLoader(int id, Bundle args) {
Uri uri = args.getParcelable(URI_IMPORTER_LOADER_ARG);
return new ImporterLoader(this, uri, dbHelper);
}
#Override
public void onLoadFinished(Loader<Exception> loader, Exception exception) {
getLoaderManager().destroyLoader(loader.getId());
if (exception != null) {
// import failed, show toast
} else {
// init Ui
}
}
#Override
public void onLoaderReset(Loader<Exception> loader) {
// nothing
}
(Note: I don't create the Loader in onCreate; rather, I do it on demand when the import functionality is invoked, and destroy it as soon as it is ready. I'm not sure if this is a correct way of using loaders.)
Now more detailed information about the problem:
when the activity is started, it creates a DbHelper (which is a SQLiteOpenHelper subclass), stores it in a member field, and reads the database (using getReadableDatabase(), but most likely it will be writable anyways) to initialize the Ui (show a list of items etc.)
if an import is triggered, a Loader is created and it gets the DbHelper from the activity; this import reads a Uri using its InputStreams, does a bit of parsing and writes rows to the database
when, during import, the device is rotated, the following happens: the loader is still going on (I don't want to destroy it, I want it to finish its task and trigger updating the Ui when it's done, possibly in the new, rotated, activity - that's the very reason I use a Loader), and it still uses its instance of DbHelper and its open connection; however, the activity is destroyed and then created again, creating another instance of DbHelper, which again tries to read the database to initialize the Ui
as a result, there are 2 DbHelpers with one open connection each, and the second one from the new activity instance throws the SQLiteDatabaseLockedException from getReadableDatabase()
Until now, I've been using a new DbHelper in every activity in the app as it wasn't possible to use the db from multiple threads, but now that I implemented the first background use case, all hell breaks loose, of course. So actually, it seems to be possible to have multiple connections opened at the same time, as long as they are not accessed simultaneously, as I had many activities stack on top of each other, each having its own helper, and (wrongly?) didn't close them in onPause and open in onResume.
So, the question is if I'm doing anything fundamentally wrong here? Based on my research, there seems to be only 2 solutions: create a ContentProvider which will manage the database (I would like not to be forced to do this as I really don't need no CP), or somehow keep only one DbHelper/connection for the whole app. How I do it is unimportant, but as of now my favorite would probably be a singleton (yuck) initialized in a custom Application subclass. In the future I would like to try Dagger so I would probably make it a #Singleton, but not yet.
Edit: unfortunately, my ui still blocks - the importer transaction blocks any db reads from the ui initialization methods. I guess I need to figure out a way to start reader transactions which are not blocked by the single write tx.
Edit 2: I was able to successfully unblock readers while a writer is working. To do this:
I call
setWriteAheadLoggingEnabled(true);
in the DbHelper constructor
use
db.beginTransactionNonExclusive();
instead of
db.beginTransaction();
in the ImporterLoader (the thing which performs the writes)

Call API in content provider for global search

We are attempting to hook up our AndroidTV app to append results into the global search. I'm running into an issue where I cannot make an api call to get the results because the system calls my content provider on the main thread.
#Override
public Cursor query(Uri uri, String[] projection, String search, String[] selectionArgs, String searchOrder) {
... Logic here that calls the API using RxJava / Retrofit
return cursor;
}
<searchable xmlns:android="http://schemas.android.com/apk/res/android"
android:label="#string/foo"
android:searchSettingsDescription="#string/foo_results"
android:includeInGlobalSearch="true"
android:searchSuggestAuthority="com.foo.search.provider"
android:searchSuggestIntentAction="android.intent.action.VIEW" />
<provider
android:authorities="com.foo.search.provider"
android:name=".search.GlobalSearchProvider"
android:exported="true"/>
When i do a global search i can see that the ContentProvider#query is called. If i attempt to do an api call on the current thread i get an networkonmainthreadexception.
I have attempted to notifty the cursor that data has changed via but had no success either.
getContext().getContentResolver().notifyChange(Uri.parse("content://com.foo.test"), null);
...
cursor.setNotificationUri(getContext().getContentResolver(), Uri.parse("content://com.foo.test"));
Is there anyway i can force the O.S to call the content provider on a seperate thread or at least notify the search that the cursor has new content?
Thank You

One of the solutions can be to set the content provider process
android:process:":androidtv"
and set the ThreadPolicy to LAX just before making network call
ThreadPolicy tp = ThreadPolicy.LAX;
StrictMode.setThreadPolicy(tp);
By running contentprovider in a different process, even if the query runs on main thread, it will not affect your UI operations

I've also struggled with this as I didn't find the currently accepted answer of blocking the UI acceptable.
However, according to Marc Bächinger from the Google TV team, this is only a problem with the emulator. In more recent builds (such as those in the currently available hardware) the search providers are called in a background thread, which avoid the problem altogether.
I've been able to test it on the Nexus Player and can confirm it works properly.
Source: https://plus.google.com/+DanielCachapa/posts/dbNMoyoRGEi

EDITED ANSWER
I've experienced this issue myself and I had to rely on the accepted answer's proposed solution. However, I've noticed that there is a noticeable lag when typing in the 'Global search' box. This lag is:
Caused by the application, since its removal makes the lag disappear
Most likely due to a synchronous wait on the queried applications - since our app does two network requests, the query() method takes time to complete, resulting in this lag
I found out that the separate process (:androidtv) is not necessary. By setting the ThreadPolicy.LAX configuration, the network request will still execute without throwing a NetworkOnMainThreadException.
I still don't understand why the lag is there.
ORIGINAL ANSWER
I don't believe that the accepted answer, while it certainly works, is the right way do it.
Once the query() method is called, you should spawn a new thread/task/job to perform a network call (therefore, avoiding the NetworkOnMainThreadException), which will update the adapter once it obtains the desired data.
There are different ways of doing this. You can either use a callback or an event bus (e.g., Otto). This is the method that I call to update the adapter:
public void updateSearchResult(ArrayList<Data> result) {
mListRowAdapter.clear();
mListRowAdapter.addAll(0, result);
HeaderItem header = new HeaderItem(0, "Search results", null);
mRowsAdapter.add(new ListRow(header, mListRowAdapter));
}

In order to solve for displaying the result in the global search using API in query method what I basically did is introduce a delay between fetching of api result and querying the db for results to return a cursor.
You can do this via
private Cursor getSuggestions(final String query) {
Cursor cursor;
cursor = getCursor(query);
if (cursor==null || cursor.getCount() == 0) {
//apiCall
try {
Thread.sleep(X millis);
} catch (InterruptedException e) {
e.printStackTrace();
}
cursor = getCursor(query);
}
return cursor;
}
Will keep on looking to see if, we can get some kind of hook to reattach without using a delay.

Because I wasn't really familiar with the Android thread. For anyone who has the same problem as me. The main point is that the query() method in content provider is not running on the UI thread.
Instead of using the asynchronous function to do HTTP requests and then update the cursor, please simply use the synchronous function to do HTTP requests and then return the cursor with the data you want.

Loaders and onLoaderReset Android

I implemented a Loader in my application for querying data from the database. I listen the changes that happen' by implementing LoaderCallbacks<Cursor> listener. The problem that I have is when using the onLoaderReset(Loader<Cursor> loader) method when my data change and I want to invalidate and free any data associated with the loader. In all the examples, in this method there is the following call:
mAdapter.swapCursor(null);
But the thing is I don't use the data from the cursor in adapter, I use it in some other way in my application.
(directly from the returned cursor in onLoadFinished(Loader<Cursor> loader, Cursor data), for example)
#Override
public void onLoadFinished(Loader<Cursor> loader, Cursor data) {
if (data.moveToFirst()) {
TOTAL_CARDS = data.getCount();
mView.createCards(TOTAL_CARDS);
} else {
TOTAL_CARDS = 0;
mView.createCards(TOTAL_CARDS);
}
}
What would be the corresponding thing to do here, that is similar with mAdapter.swapCursor.
I don't have much experience with loaders, in fact I just started working with them, so if someone has a solution to this, I would appreciate it. Thx!
EDIT:
For now, I am passing null to the loader and it works, like this:
#Override
public void onLoaderReset(Loader<Cursor> loader) {
loader = null;
}
};
But is this the right solution?

Doing
#Override
public void onLoaderReset(Loader<Cursor> loader) {
loader = null;
}
is just as good as doing nothing. In your example code, you are merely nulling your method's local reference to its argument. However, this reference will always be removed after the return of the method call. (You might want to read Is Java "pass-by-reference" or "pass-by-value"? for a further discussion of the topic.)
The onLoaderReset(Loader) method gets called when your loader's callback (usually an Activity or Fragment instance) is asked to release all references to the Cursor which it has gained via onLoadFinished(Loader, Cursor) before. Basically this method asks you to clean up since the Loader will soon close the Cursor it provided to you before. After the cursor was closed, you will not longer be able to retrieve data by it. If the cursor would however still be in use (typically by a CursorAdapter as you mentioned) after it was closed, this would cause an exception to be thrown.
Similarly, onLoadFinished(Loader, Cursor) has an implicit contract asking that after the method returns any formerly provided Cursor objects must not longer be in use. Instead, you have to replace these references by the new cursor which is provided as a method argument. In contrast, onLoaderReset(Loader) asks you to fulfill the same contract, but without providing a replacement, i.e. you should remove all references to a formerly retrieved Cursor.
In your example, you do not let your Cursor escape the method scope but instead you are reading the data right away. Therefore, it is not necessary to remove any references to a Cursor object which was provided via onLoadFinished(Loader, Cursor) since there are none. An empty implementation of onLoaderReset(Loader) to fulfill the interface contract will therefore do the job for you.

Updating Cursor in background thread

I have a similar problem to the one described in this discussion: I need to refresh a ListView when the underlying database changes, but the query is expensive so I'm doing it in an AsyncTask.
Here's what I do when the updated Cursor is ready. (This is also how the list is initially populated on startup.)
#Override
protected void onPostExecute(Cursor result) {
if (activity != null) {
if (currentCursor != null) {
// existing cursor is closed by adapter.changeCursor() so
// we don't need to explicitly close it here
stopManagingCursor(currentCursor);
}
currentCursor = result;
startManagingCursor(currentCursor);
if (adapter == null) {
adapter = getAdapter(result);
setListAdapter(adapter);
} else {
adapter.changeCursor(result);
}
activity.onGotList(result, dbAdapter);
}
}
Here's the error I get. It doesn't happen every time, which is even more frustrating.
Releasing statement in a finalizer. Please ensure that you explicitly call close() on your cursor: SELECT DISTINCT t._id AS _id, t.amount, t.date, t.memo, t.synced, t.flag, (children.pa
android.database.sqlite.DatabaseObjectNotClosedException: Application did not close the cursor or database object that was opened here
at android.database.sqlite.SQLiteCompiledSql.<init>(SQLiteCompiledSql.java:62)
at android.database.sqlite.SQLiteProgram.<init>(SQLiteProgram.java:100)
at android.database.sqlite.SQLiteQuery.<init>(SQLiteQuery.java:46)
at android.database.sqlite.SQLiteDirectCursorDriver.query(SQLiteDirectCursorDriver.java:53)
at android.database.sqlite.SQLiteDatabase.rawQueryWithFactory(SQLiteDatabase.java:1412)
at android.database.sqlite.SQLiteDatabase.rawQuery(SQLiteDatabase.java:1382)
So, I am obviously not closing the Cursor correctly. If I call currentCursor.close() instead of relying on the outgoing Cursor being closed by adapter.changeCursor(), then I get warnings about closing the Cursor twice or closing a null Cursor.
What is the correct way to do this?
In the discussion I linked to, Dianne Hackborn suggests using a Loader instead. That is not an option for me since my code has to run on Android 2.1.

Try to .close() the Cursor when the Activity pause or terminates.
In the onPause() or onDestroy() section of the activity.

Basically, it's possible, but very bad practice to access the same database from two different helpers, so if you have an activity performing database queries, you shouldn't also have a thread accessing it, otherwise android will throw up a quiet error in logcat, and then forget about the query...
The best solution I have found is to implement a thread pool of runnables, each one is a database query and they all use the same database helper. Consequently, only one thread is accessing the database at any one time and the database is just opened and closed when the thread pool starts/stops.
An implementation of the thread pool pattern can be found here: http://mindtherobot.com/blog/159/android-guts-intro-to-loopers-and-handlers/

If you are not changing anything other then redrawing from the list is it necessary to change the cursor at all. Could you get away with just requiring the current adapter.
something along the lines of
adapter.getCursor().requery();
although if you are in a thread other then the main ui thread you may want to call it with
//Did not realize this was deprecated Thanks to Graham Borland for the heads up
runOnUiThread(new Runnable(){
public void run(){
adapter.getCursor().requery();
}
});
Depending on your setup.
New solution still need testing and make sure it is not going to cause issues apparently startManaginCursor and stopManaginCursor are deprecated too so this solution is not worth good either.
stopManagingCursor(adapter.getCursor());
if (!adapter.getCursor().isClosed())
adapter.getCursor().close();
//cursor creation stuff here if needed
startManagingCursor(newCursor);
adapter.changeCursor(newCursor);

What to do with a database, retrieved by a SQLiteOpenHelper implementation, when finished using it?

Dear Fellow Android Developers!
EDIT:
Thank you all for your answers. I see from many of them that it seems to be common (and accepted) practice to write your own close() method in your database adapter. Fair enough.
But how does that work with a ContentProvider? Usually when querying my database through my ContentProvider I simply issue something like:
Cursor managedCursor = managedQuery(...);
I don't see how I, with this methodology, can access the custom close() method in my custom ContentProvider implementation. Should I instead, from my Activity, do something like:
MyCustomProvider myProvider = (MyCustomProvider) getContentResolver();
and then:
myProvider.query(...);
myProvider.close();
And above all; is this at all necessary (as of point 2 below)?
END EDIT
To a certain degree I must say that I get the concept of the SQLiteOpenHelper, what it is, how it's used and so. I even use it on a regular basis when I write my own ContentProvider's.
The thing is that I'm not sure what to do with the SQLiteDatabase object, returned by the myOpenHelper.getWritableDatabase() (or the myOpenHelper.getReadableDatabase() function for what matters) when I'm done with it.
According to Android ContentProvider.onCreate() documentation:
You should defer nontrivial initialization (such as opening, upgrading, and scanning databases) until the content provider is used (via query(Uri, String[], String, String[], String), insert(Uri, ContentValues), etc).
[...]
If you do use SQLiteOpenHelper, make sure to avoid calling getReadableDatabase() or getWritableDatabase() from this method. (Instead, override onOpen(SQLiteDatabase) to initialize the database when it is first opened.)
The above gives me a hint where to initialize the database (the query(...), insert(...), etc functions), but it doesn't tell me anything on how to treat the created SQLiteDatbase object when I've finished using it.
Should I save it as a member variable of my ContentProvider implementation (and treat it much like a "private singleton" for future use)?
Should I just leave it when exiting the query(...), insert(...), etc. functions and trust that the SQLiteOpenHelper will manage it for me in future calls?
[Insert your alternative point-of-view here]
Being the confiding (or lazy) developer I've implemented my code according to the second alternative above. But I can't get rid of the creepy feeling that I'm neglecting something important.

It depends on what you're doing with your database. If you just do an insert, delete or select where you get an business object back, then you can close the database right after using it. As far as I know it is designed that you simply close it and request a new one when ever you need it.
But be careful when you're working with a cursor then you have to keep the database open as long as the cursor is in use. Otherwise the application will crash when the cursor has to reload data.

I guess you should close it, for example in onDestroy() of an activity that is using it.
So in my DBAdapter class I have:
/**
* Close the database
*/
public void close() {
mDb.close(); //mDb was obtained using mDbHelper.getWritableDatabase();
}
And in my activity:
public void onCreate(Bundle bundle){
...
mDBAdapter = new DBAdapter(this);
// Open or create the database
mDBAdapter.open();
}
#Override
public void onDestroy() {
// Close the database
mDBAdapter.close();
super.onDestroy();
}
Not sure if this is suitable for your provider concept.

If you check the example of use for that object in the API of Android, you can see the object is just used, but no close is necesary.
They implement the method close() though, but I havent seen they use it.