Develop Reference

Race conditions in Cursors for Android?

So I have this app that loads data via a provider into a cursor.
The cursor is then used to fetch the data for the app to use.
The loading process is done with the LoaderManager API.
Now the problem arises after I call swapCursor() from onLoadFinished() in some circumstances; the cursor reports having the correct amount of data, but when I try to access it I get a CursorIndexOutOfBoundsException.
The peculiar thing is, I know exactly which line this occurs on, so I put a breakpoint there that suspends all threads and tried to inspect the data to see what was up, suddenly the error disappears.
Now by trial and error I figured that the problem goes away even if I do nothing. Like literally nothing after it suspends everything. If I just wait for a few seconds before resuming execution the data is perfectly accessible.
So my question is: Does anybody know if the Android operating system does something in the background that creates a race condition? I don't really see how this can be a bug on my side since when I tested it I suspended all threads, so my app can't do anything while I wait for the issue to resolve.
As a sidenote, if I just replace the swapCursor() call with changeCursor() (doing this means I have to use restartLoader() instead of initLoader() due to some other feature of the system) the bug goes away, but I've read somewhere that this is not the proper way to do it, since it doesn't handle the closing of the cursor properly (I think this was the reason I had to use restartLoader(), but I can't really recall at the moment).
Some code excerpts:
#Override
public void onCreate( Bundle savedInstanceState ) {
super.onCreate( savedInstanceState );
...
getLoaderManager().initLoader( ACTIVE_TEAM_LOADER, null, this );
}
...
#Override
public void onLoadFinished( Loader<Cursor> loader, Cursor data ) {
switch ( loader.getId() ) {
case ACTIVE_TEAM_LOADER:
activeTeamAdapter.swapCursor(
new MergeCursor( new Cursor[]{ data, createNewCursor } ) );
...
}
And the offending line, called deep in the stack from a CursorAdapter.bindView():
long value = data.getLong( data.getColumnIndex( getDBName() ) );
To me it looks like the data in the cursor is not properly loaded yet.

The problem was caused by the fact that swapCursor() does not close the previous MergeCursor cursor. Closing the previous cursor solved the problem, but introduced a new problem because you are not supposed to close cursors from onLoadFinished(), and when closing a MergeCursor the MergeCursor closes all sub-cursors as well.
This was solved by switching back to using changeCursor() whenever I use adapters to only display MergeCursors (some of the adapters display MergeCursors and MatrixCursors when there is no data from onLoadFinished() and for some reason MatrixCursors seems like they need to be closed between usages so this does not apply to them, I don't know why) and wrapping any cursors in a class that disregards closing.
onLoadFinished() excerpt:
Cursor[] cursors = new Cursor[] { new UnclosableCursor( data ), createNewCursor };
activeTeamAdapter.changeCursor( new MergeCursor( cursors ) );
The wrapper:
public class UnclosableCursor extends CursorWrapper {
public UnclosableCursor( Cursor cursor ) {
super( cursor );
}
#Override
public void close() {}
}
A previous version of this answer alluded to the fact that this had to do with casting, but after more research this seems to only have been a symptom of the problem, not the actual problem.

SQLite trouble with multiple concurrently open connections

I'm having SQLite trouble in a multithreaded application. I have an Activity which uses a subclass of AsyncTaskLoader to perform some data import from a file (specified by an Uri as it comes from Android Storage Access Framework), and when the loader is started and does its work (it writes to the database) and the device is rotated, I get a 'android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5)' exception. I know what the problem is (multiple SQLiteOpenHelpers accessing a database), but I am not sure how I'm supposed to fix this. Here is my code:
Activity:
private void importTests(Uri uri) {
Bundle loaderArgs = new Bundle();
loaderArgs.putParcelable(URI_IMPORTER_LOADER_ARG, uri);
getLoaderManager().initLoader(0, loaderArgs, this).forceLoad();
}
#Override
public Loader<Exception> onCreateLoader(int id, Bundle args) {
Uri uri = args.getParcelable(URI_IMPORTER_LOADER_ARG);
return new ImporterLoader(this, uri, dbHelper);
}
#Override
public void onLoadFinished(Loader<Exception> loader, Exception exception) {
getLoaderManager().destroyLoader(loader.getId());
if (exception != null) {
// import failed, show toast
} else {
// init Ui
}
}
#Override
public void onLoaderReset(Loader<Exception> loader) {
// nothing
}
(Note: I don't create the Loader in onCreate; rather, I do it on demand when the import functionality is invoked, and destroy it as soon as it is ready. I'm not sure if this is a correct way of using loaders.)
Now more detailed information about the problem:
when the activity is started, it creates a DbHelper (which is a SQLiteOpenHelper subclass), stores it in a member field, and reads the database (using getReadableDatabase(), but most likely it will be writable anyways) to initialize the Ui (show a list of items etc.)
if an import is triggered, a Loader is created and it gets the DbHelper from the activity; this import reads a Uri using its InputStreams, does a bit of parsing and writes rows to the database
when, during import, the device is rotated, the following happens: the loader is still going on (I don't want to destroy it, I want it to finish its task and trigger updating the Ui when it's done, possibly in the new, rotated, activity - that's the very reason I use a Loader), and it still uses its instance of DbHelper and its open connection; however, the activity is destroyed and then created again, creating another instance of DbHelper, which again tries to read the database to initialize the Ui
as a result, there are 2 DbHelpers with one open connection each, and the second one from the new activity instance throws the SQLiteDatabaseLockedException from getReadableDatabase()
Until now, I've been using a new DbHelper in every activity in the app as it wasn't possible to use the db from multiple threads, but now that I implemented the first background use case, all hell breaks loose, of course. So actually, it seems to be possible to have multiple connections opened at the same time, as long as they are not accessed simultaneously, as I had many activities stack on top of each other, each having its own helper, and (wrongly?) didn't close them in onPause and open in onResume.
So, the question is if I'm doing anything fundamentally wrong here? Based on my research, there seems to be only 2 solutions: create a ContentProvider which will manage the database (I would like not to be forced to do this as I really don't need no CP), or somehow keep only one DbHelper/connection for the whole app. How I do it is unimportant, but as of now my favorite would probably be a singleton (yuck) initialized in a custom Application subclass. In the future I would like to try Dagger so I would probably make it a #Singleton, but not yet.
Edit: unfortunately, my ui still blocks - the importer transaction blocks any db reads from the ui initialization methods. I guess I need to figure out a way to start reader transactions which are not blocked by the single write tx.
Edit 2: I was able to successfully unblock readers while a writer is working. To do this:
I call
setWriteAheadLoggingEnabled(true);
in the DbHelper constructor
use
db.beginTransactionNonExclusive();
instead of
db.beginTransaction();
in the ImporterLoader (the thing which performs the writes)

Call API in content provider for global search

We are attempting to hook up our AndroidTV app to append results into the global search. I'm running into an issue where I cannot make an api call to get the results because the system calls my content provider on the main thread.
#Override
public Cursor query(Uri uri, String[] projection, String search, String[] selectionArgs, String searchOrder) {
... Logic here that calls the API using RxJava / Retrofit
return cursor;
}
<searchable xmlns:android="http://schemas.android.com/apk/res/android"
android:label="#string/foo"
android:searchSettingsDescription="#string/foo_results"
android:includeInGlobalSearch="true"
android:searchSuggestAuthority="com.foo.search.provider"
android:searchSuggestIntentAction="android.intent.action.VIEW" />
<provider
android:authorities="com.foo.search.provider"
android:name=".search.GlobalSearchProvider"
android:exported="true"/>
When i do a global search i can see that the ContentProvider#query is called. If i attempt to do an api call on the current thread i get an networkonmainthreadexception.
I have attempted to notifty the cursor that data has changed via but had no success either.
getContext().getContentResolver().notifyChange(Uri.parse("content://com.foo.test"), null);
...
cursor.setNotificationUri(getContext().getContentResolver(), Uri.parse("content://com.foo.test"));
Is there anyway i can force the O.S to call the content provider on a seperate thread or at least notify the search that the cursor has new content?
Thank You

One of the solutions can be to set the content provider process
android:process:":androidtv"
and set the ThreadPolicy to LAX just before making network call
ThreadPolicy tp = ThreadPolicy.LAX;
StrictMode.setThreadPolicy(tp);
By running contentprovider in a different process, even if the query runs on main thread, it will not affect your UI operations

I've also struggled with this as I didn't find the currently accepted answer of blocking the UI acceptable.
However, according to Marc Bächinger from the Google TV team, this is only a problem with the emulator. In more recent builds (such as those in the currently available hardware) the search providers are called in a background thread, which avoid the problem altogether.
I've been able to test it on the Nexus Player and can confirm it works properly.
Source: https://plus.google.com/+DanielCachapa/posts/dbNMoyoRGEi

EDITED ANSWER
I've experienced this issue myself and I had to rely on the accepted answer's proposed solution. However, I've noticed that there is a noticeable lag when typing in the 'Global search' box. This lag is:
Caused by the application, since its removal makes the lag disappear
Most likely due to a synchronous wait on the queried applications - since our app does two network requests, the query() method takes time to complete, resulting in this lag
I found out that the separate process (:androidtv) is not necessary. By setting the ThreadPolicy.LAX configuration, the network request will still execute without throwing a NetworkOnMainThreadException.
I still don't understand why the lag is there.
ORIGINAL ANSWER
I don't believe that the accepted answer, while it certainly works, is the right way do it.
Once the query() method is called, you should spawn a new thread/task/job to perform a network call (therefore, avoiding the NetworkOnMainThreadException), which will update the adapter once it obtains the desired data.
There are different ways of doing this. You can either use a callback or an event bus (e.g., Otto). This is the method that I call to update the adapter:
public void updateSearchResult(ArrayList<Data> result) {
mListRowAdapter.clear();
mListRowAdapter.addAll(0, result);
HeaderItem header = new HeaderItem(0, "Search results", null);
mRowsAdapter.add(new ListRow(header, mListRowAdapter));
}

In order to solve for displaying the result in the global search using API in query method what I basically did is introduce a delay between fetching of api result and querying the db for results to return a cursor.
You can do this via
private Cursor getSuggestions(final String query) {
Cursor cursor;
cursor = getCursor(query);
if (cursor==null || cursor.getCount() == 0) {
//apiCall
try {
Thread.sleep(X millis);
} catch (InterruptedException e) {
e.printStackTrace();
}
cursor = getCursor(query);
}
return cursor;
}
Will keep on looking to see if, we can get some kind of hook to reattach without using a delay.

Because I wasn't really familiar with the Android thread. For anyone who has the same problem as me. The main point is that the query() method in content provider is not running on the UI thread.
Instead of using the asynchronous function to do HTTP requests and then update the cursor, please simply use the synchronous function to do HTTP requests and then return the cursor with the data you want.

Multiple threading questions when using sqlite transaction in Android

I use sqlite transaction in Android:
SQLiteDatabase database = sqlite_helper.getWritableDatabase();
database.beginTransaction();
...
database.setTransactionSuccessful();
database.endTransaction();
My questions are :
1. Should I place endTransaction() in finally code block like this：
try {
database.beginTransaction();
...
database.setTransactionSuccessful();
}
finally {
database.endTransaction();
}
If there are exepctions during database operations, will the database be rolled back automatically without using "finally"?
When the transaction is not ended, can other threads read or write the same database? I hear sqlite in Android is threading safe, but I are not sure with it. I guess there will be some problems during transaction. Is there an error raised if another thread writes the same database with the same connection?
I ever found this error in my app, but I don't know whether it's related to the threading safe problem:
android.database.sqlite.SQLiteMisuseException: library routine called out of sequence:
, while compiling
Does anyone help me to answer these questions? Thanks a lot!

1.you should always place endTransaction in finally block
2.transaction in SQLite is thread safe,see the doc http://www.sqlite.org/atomiccommit.html

You should always put endTransaction() into a finally block (also see the docs).
Otherwise, the database would not be able to notice than an exception has happened.
The only other way to end a transaction would be to close the connection, in which case SQLite automatically rolls back any active transaction.
As long as one connection writes to the database (which means that a transaction is active), no other connections can read or write. Therefore, you should take care not to forget to end transactions.
You should never write from multiple threads; what would happen if one threads ends the transaction while the other one is still writing?
Your SQLiteMisuseException might be related, or not; that's impossible to say without seeing the code.

Yes, you should use the finally block. Here is a simple, THREAD SAFE method I use:
/**
* Call for multiple DB insertion transactions. It is thread safe and fast!
*/
private synchronized void writeTransaction(Runnable task) {
try {
db.beginTransaction();
task.run();
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
}
the synchronized keyword locks the method with its containing object, thus making it thread safe...

What to do with a database, retrieved by a SQLiteOpenHelper implementation, when finished using it?

Dear Fellow Android Developers!
EDIT:
Thank you all for your answers. I see from many of them that it seems to be common (and accepted) practice to write your own close() method in your database adapter. Fair enough.
But how does that work with a ContentProvider? Usually when querying my database through my ContentProvider I simply issue something like:
Cursor managedCursor = managedQuery(...);
I don't see how I, with this methodology, can access the custom close() method in my custom ContentProvider implementation. Should I instead, from my Activity, do something like:
MyCustomProvider myProvider = (MyCustomProvider) getContentResolver();
and then:
myProvider.query(...);
myProvider.close();
And above all; is this at all necessary (as of point 2 below)?
END EDIT
To a certain degree I must say that I get the concept of the SQLiteOpenHelper, what it is, how it's used and so. I even use it on a regular basis when I write my own ContentProvider's.
The thing is that I'm not sure what to do with the SQLiteDatabase object, returned by the myOpenHelper.getWritableDatabase() (or the myOpenHelper.getReadableDatabase() function for what matters) when I'm done with it.
According to Android ContentProvider.onCreate() documentation:
You should defer nontrivial initialization (such as opening, upgrading, and scanning databases) until the content provider is used (via query(Uri, String[], String, String[], String), insert(Uri, ContentValues), etc).
[...]
If you do use SQLiteOpenHelper, make sure to avoid calling getReadableDatabase() or getWritableDatabase() from this method. (Instead, override onOpen(SQLiteDatabase) to initialize the database when it is first opened.)
The above gives me a hint where to initialize the database (the query(...), insert(...), etc functions), but it doesn't tell me anything on how to treat the created SQLiteDatbase object when I've finished using it.
Should I save it as a member variable of my ContentProvider implementation (and treat it much like a "private singleton" for future use)?
Should I just leave it when exiting the query(...), insert(...), etc. functions and trust that the SQLiteOpenHelper will manage it for me in future calls?
[Insert your alternative point-of-view here]
Being the confiding (or lazy) developer I've implemented my code according to the second alternative above. But I can't get rid of the creepy feeling that I'm neglecting something important.

It depends on what you're doing with your database. If you just do an insert, delete or select where you get an business object back, then you can close the database right after using it. As far as I know it is designed that you simply close it and request a new one when ever you need it.
But be careful when you're working with a cursor then you have to keep the database open as long as the cursor is in use. Otherwise the application will crash when the cursor has to reload data.

I guess you should close it, for example in onDestroy() of an activity that is using it.
So in my DBAdapter class I have:
/**
* Close the database
*/
public void close() {
mDb.close(); //mDb was obtained using mDbHelper.getWritableDatabase();
}
And in my activity:
public void onCreate(Bundle bundle){
...
mDBAdapter = new DBAdapter(this);
// Open or create the database
mDBAdapter.open();
}
#Override
public void onDestroy() {
// Close the database
mDBAdapter.close();
super.onDestroy();
}
Not sure if this is suitable for your provider concept.

If you check the example of use for that object in the API of Android, you can see the object is just used, but no close is necesary.
They implement the method close() though, but I havent seen they use it.