Develop Reference

Firebase Database CLI remove command do not work

By accident, I had created an endless loop in my android studio app which has added over 100k+ records to my Realtime Database. After closing everything I wanted to delete this massive table using Firebase CLI but every time I try it i get an error:
> firebase database:remove /matches
? You are about to remove all data at https://XXX.firebaseio.com/matches. Are you sure? Yes
Error: Task index 0 failed: retries exhausted after 4 attempts
How am I supposed to delete it then?

The most likely cause is that other clients are updating the data while your code is trying to remove it. If that is the case, the only way to work around it, is to remove the data in smaller chunks, to reduce the contention during the operation.
To do this, you'll:
Read the data from /matches either through the API, or from an automatic backup.
Determine a reasonable subset of the nodes to remove in one go.
Make an API call to remove that subset of the data.
Repeat this process until all data is gone.

Mystery error: "Failed to open database phenotype.db, database is locked." Can't find any info online

I was trying to test an app and it wouldn't launch, instead I kept getting this error:
Failed to open database '/data/data/com.google.android.gms/databases/phenotype.db'.
android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5): , while compiling: PRAGMA journal_mode
I can't find any information about this error online, and no information on what phenotype.db does. I was not doing anything related to databases at the time when I tested this; the only changes I made were layout-related.
Weirdly enough this happened on another app I was testing a few weeks ago. It was a different app, being tested on a different device, from a different computer, and the exact same error came up. In that case I switched to a different device and the error went away. However now the device I switched to is having that error.
Is this some kind of glitch with Google? What can I do to resolve it so I can test my app?

You have forgotten to close the database, or another thread is writing to the database when you are trying to write to it. SQLite locks the database when it is writing to it to avoid corruption if another entity tries to write to the same database at the same time. Android, will only show a error in logcat, and the query you supplied will be just forgotten.
Recommendations:
You only access the database from one SQLOpenHelper
You make sure you close all instances of database helpers once you have finished with them
You make sure you always end transactions with endTransaction() also if you do not set them successful (i.e. if you want to roll 'em back), in case you use transactions

Android Lollipop 5.0.1 SQLiteLog POSIX Error 11 SQLite Error: 3850

I am having an issue while upgrading an app to support Android Lollipop.
The app implements a SyncAdapter that writes on a db through a content provider.
At the same time it can happen that the user is browsing the front-end of the app where loaders read the same data from the database.
Loaders also listen to data changes.
Now, if I run the program on a pre-Lollipop device everything works without any error output.
On Lollipop instead I receive the following logcat message:
11:20:59.344 22341-22376/com.example.com E/SQLiteLog﹕ (10) POSIX Error : 11 SQLite Error : 3850
11:20:59.364 22341-22376/com.example.com E/SQLiteLog﹕ (10) POSIX Error : 11 SQLite Error : 3850
11:20:59.364 22341-22376/com.example.com E/SQLiteLog﹕ (10) POSIX Error : 11 SQLite Error : 3850
11:20:59.364 22341-22376/com.example.com E/SQLiteLog﹕ (10) POSIX Error : 11 SQLite Error : 3850
Now, from SQLite docs:
(3850) SQLITE_IOERR_LOCK
The SQLITE_IOERR_LOCK error code is an extended error code for
SQLITE_IOERR indicating an I/O error in the advisory file locking
logic. Usually an SQLITE_IOERR_LOCK error indicates a problem
obtaining a PENDING lock. However it can also indicate miscellaneous
locking errors on some of the specialized VFSes used on Macs.
Everything seems to work properly on a high level (that is both reads
and writes are performed)
and:
A PENDING lock means that the process holding the lock wants to write
to the database as soon as possible and is just waiting on all current
SHARED locks to clear so that it can get an EXCLUSIVE lock. No new
SHARED locks are permitted against the database if a PENDING lock is
active, though existing SHARED locks are allowed to continue.
I know that the SQLite version has been updated by few major releases in Lollipop, so I am prone to think that the error is due to some new behaviour of SQLite that I cannot isolate.
However, everything seems to work fine from a higher level point of view (App doesn't crash, both reads and writes are performed, framerate doesn't drop - at least to human eyes) but I wouldn't want to ignore the issue to release the app until I am sure it won't cause data corruption or troubles.
Perhaps I am missing on some important changes to lollipop regarding locks and multiprocess database access, but I feel it's an issue that lies on a lower level with respect to the Art/Dalvik domain and so has to be fixed in an NDK context.
Is there a way to fix this possibly without distributing an app specific version of SQLite? Is there any manifest/SQLite option to avoid the error?
Thanks in advance

Writer locks the database for both reading and writing.
That means it has to wait for all readers to finish and release locks in order to obtain lock.
After the writer requested a lock, new reader locks must wait for writer to first obtain lock and then release it.
This could be a solution for you: WAL mode
Activating And Configuring WAL Mode:
An SQLite database connection defaults to journal_mode=DELETE. To convert to WAL mode, use the following pragma:
PRAGMA journal_mode=WAL;
WAL will not block readers while writing, which means also that writer does not need to wait for current read locks to be released.
Minimum SQLite version required for WAL is 3.7.0 (2010-07-21).
Lollipop 5.0 uses SQLite 3.8.4.3 so WAL should be available for you.
But WAL does not exist in Android version less than 3.0 although there are some exceptions from this. Take a look at Version of SQLite used in Android?. If you don't need your app to work below Android 3.0 you can use WAL.

ContentProvider throws SQLiteCantOpenDatabaseException: unable to open database file

In my content provider I create and maintain 3 SQLiteDatabase objects. They are created like this:
private ContentProviderHelper helper;
#Override
public boolean onCreate() { // that's the ContentProvider onCreate()
SQLiteDatabase dbLog = new DbLog(getContext()).getWritableDatabase();
SQLiteDatabase dbSession = new DbSession(getContext()).getWritableDatabase();
SQLiteDatabase dbLocation = new DbLocation(getContext()).getWritableDatabase();
helper = new ContentProviderHelper(UriManager.getAuthority(getContext()));
helper.addDb(dbLog, DbLog.TABLE_NAME, UriManager.LOG, SQLiteDatabase.CONFLICT_REPLACE);
helper.addDb(dbSession, DbSession.TABLE_NAME, UriManager.SESSION, SQLiteDatabase.CONFLICT_REPLACE);
helper.addDb(dbLocation, DbLocation.TABLE_NAME, UriManager.LOCATION, SQLiteDatabase.CONFLICT_REPLACE);
the ContentProviderHelper stores those SQLiteDatabase in an ArrayList indexed with the UriMatcher.
The <provider> is properly registered in the manifest and my app have SD-card permission.
It runs fine for most of our 500.000 user base, but every once in a while I get a SQLiteCantOpenDatabaseException: unable to open database file from the Google Play
the relevant stack track is:
Caused by: android.database.sqlite.SQLiteCantOpenDatabaseException: unable to open database file
at android.database.sqlite.SQLiteDatabase.dbopen(Native Method)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:1013)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:986)
at android.database.sqlite.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:1051)
at android.app.ContextImpl.openOrCreateDatabase(ContextImpl.java:787)
at android.content.ContextWrapper.openOrCreateDatabase(ContextWrapper.java:221)
at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:157)
most often than not those error reports come from generic brandless poor quality devices.
Any help on how to properly avoid those errors will be much appreciated.
edit:
a different SQL exceptions that I'm getting on the same code.
Caused by: android.database.sqlite.SQLiteDiskIOException: disk I/O error: COMMIT;
at android.database.sqlite.SQLiteDatabase.native_execSQL(Native Method)
at android.database.sqlite.SQLiteDatabase.execSQL(SQLiteDatabase.java:1763)
at android.database.sqlite.SQLiteDatabase.endTransaction(SQLiteDatabase.java:583)
at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:137)
at ***.***.***.data.ContentManager.onCreate(ContentManager.java:26)
so is there any other way of handling this besides try{} catch(){} the hell out of it ?
edit:
Extra info regarding cursors usage:
In general cursors are kept opened for the minimum amount of time as possible. All 3 SQLiteDatabase have a Wrapper that do the actual Cursor calls, read the data from them and close them. There's only 1 instance where the cursor is passed back to the calling object, but it is an IntentService, again, single thread, single instance, single process (just like ContentProvider) that loops through the cursor and close it.

This doesn't look like a programming fault on your (Java) side, neither of the Android SQLite wrapping classes.
The provided stacktraces and code doesn't provide much information to proof my guess, but I think this should be caused by the unreliable nature of those SDCards (you already mentioned that the observed traces usually come from low quality phones).
The exception itself is generated in the native sqlite code, and while I havn't looked up the C/C++ part of the JNI interface, this should come directly from the underlying sqlite3_open call. For non obvious reasons the actuall errorcode is not included in the thrown exception, so you are basically out of luck here to find the root cause.
As this comes directly from the native layer it's some kind of filesystem/hardware problem. The card may be broken, the cardsocket may be broken/dirty/whatever or anything in between could be messed up (most likely physically).
To give you a possible solution for a problem you can't fix: Don't use the (sometimes unreliable) SDCard. If your databases are small enough (less than a few MBs) you should be fine storing them on the internal flash. This has to be reliable storage as if thouse writes or queries fail the user has more serious problems anyway.
If this is not an option, you could write "most recent data" (I don't know which kind of data you store) internally and migrate this data reguallary on the SDCard. This way you could at least try to store it without flodding your code with try/catch (if you try this in a seperate thread which catches any exception at the top level), but it doesn't solve the read part of the problem.
Another, while a little ugly, thing I can imagine: Just let the exceptions fly right through and catch them in the Application class (you can overload it), push it to disk (internal, obviously) and rethrow it to crash your app. On the next start you could then lookup if you've been crashed by an SQLite Exception and present a msg to the user which states something like "Sorry, your SDCard seems to be broken as we couldn't use it. Consider buying a new one of order a better device". ACRA may help you with this, it does what I tried to describe pretty well the last time I used it (1 year ago).

What I suppose is that you have some problem with concurrency and you are trying to get DB object more than once.
Moreover - having 3 different DB (probably) makes the application run 3 DB engines at a time (memory problems).
There is no single and simple solution while the problem is located in other place than exception is thrown, but some steps that you can do:
Put 3 databases into one file (if possible).
Make a simple singleton to keep DB object in memory and avoid concurrent attempts to the stored file. Remember that SQLite is claimed as "thread safe" not as "multithread" so the IO error can be just a natural behavior to protect DB corruption.
Make sure that cursors are closed as quick as possible (i.e. after query the object read the cursor and map data to some POJO objects.
Workaround - use try/catch while opening db, in case of exception try to do it after some time (1 second).

Closing an SQLite database on one thread breaks the database opened on another thread

We all learn that resources, such as database connections, should be acquired late and released early.
Yet applying this principle to SQLite database connections on Android have caused me some headache.
I have an app that download updates from a backend server in a service working in the background, writing updates to the database at a regular basis. The problem I experience arise when the following sequence occurs:
Service opens a writable database connection
Some activity opens a readable database connection
Service closes its database connection concurrently with the activity reading data
Activity fails due to its database connection was closed
Both the service and the activity uses the same SQLiteOpenHelper class, though different instances, to open their connections. My initial assumption was that this should work just fine, but somehow it seems that the underlying connection is shared between the two database instances.
To work around the problem I ended up not closing the database objects, only closing any opened cursors. This seems to work, though I'm not sure that I'm not leaking memory here.
Is there something obvious I am missing here?

Is there something obvious I am missing here?
I'd say no. Looking at the source code to SQLiteOpenHelper, I can't see how two instances could be sharing a SQLiteDatabase object.
Some diagnostic suggestions:
Dump the toString() value of each SQLiteDatabase, which should give you a Java instance ID. If they are the same, that is where your problem lies, and you will need to work your way upstream to figure out how the heck this is happening (e.g., you really are using the same instance of the SQLiteOpenHelper).
With your database in a stable state (i.e., no need to create or upgrade), flip one of your two database spots to use SQLiteDatabase directly, rather than via SQLiteOpenHelper, and see if that changes matters.