Develop Reference

repopulating room database by deleting all the content

I now finished the android codelab about room db with MVVM arch. But there is one part that i didn't exactly understand. This is a sentence from the codelab:
To delete all content and repopulate the database
whenever the app is created, you'll create a RoomDatabase.Callback and override onCreate().
and this is the code they provide:
private class WordDatabaseCallback(
private val scope: CoroutineScope
) : RoomDatabase.Callback() {
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
INSTANCE?.let { database ->
scope.launch {
populateDatabase(database.wordDao())
}
}
}
suspend fun populateDatabase(wordDao: WordDao) {
// Delete all content here.
wordDao.deleteAll()
}
}
the part that I don't understand is "deleting all content". why do I need to delete all content when the app is created? and what do they mean by "whenever the app is created"? is it for the first time the app installed or everytime app is opened?
when I don't use this code, the app works fine too. can someone explain the purpose of deleting everything?

why do I need to delete all content when the app is created?
You don't need to, this is a specific scenario where for whatever reason you want the App to delete all content stored in the database. However there should not be any anyway as onCreate is only called when the database doesn't actually exist (unless manually invoked), so there will be no content to delete.
If you used a pre-packaged database using the .createFromAsset then onCreate isn't called.
I believe that they have just included a simple, unlikely to fail, introduction to using a Callback.
and what do they mean by "whenever the app is created"? is it for the first time the app installed or everytime app is opened?
The former. That is the whole purpose of a Database is to store data long term. As such the database is stored in the App's data space. When an App is installed then the data space will not have the database. So the App and thus Room has to know to create the database before it can be used.
So when you attempt to use the database, the processing of preparing to use the database checks to see if the database exists.
If the database does exist then it carries on without calling onCreate.
If the database does not exist then Room will try to create the database and the tables therein via the onCreate method. The Callback allows intervention at this stage by overriding the onCreate method.
If the App is stopped and rerun the database will still exist, and onCreate is not called.
If the App is uninstalled and then re-installed the the database will be deleted, so onCreate is called.
If a new version of the App is installed, then the database will still exist and onCreate is not called.
If a new version of the App is installed and it includes a new version for the database, onCreate is not called. Instead whatever migration path is specified will be taken.

How to abandon a manual Room migration script, and fall back to destructive migration?

In the app I'm working on, we had a complex manual migration that required data parsing, manual SQL commands, etc. This was to convert a List<X> column into a new linked table of X. I've previously written about the approach, but the specific commands are not especially relevant for this question.
The issue I'm encountering is ~1% of users are experiencing a crash as part of this migration. This cannot be reproduced in testing, and due to our table's size, Crashlytics cannot show any useful error:
Losing customer data isn't catastrophic in this context, but being stuck in the current "try migrate, crash, reopen app and repeat" loop is. As such, I want to just give up on the migration and fall back to a destructive migration if we encounter an exception.
Any ideas how this can be done? My current solution is rerunning the DB changes (but not the presumably failing data migration) inside the catch, but this feels very hacky.
Our database is defined as:
Room.databaseBuilder(
context.applicationContext,
CreationDatabase::class.java,
"creation_database"
)
.addMigrations(MIGRATION_11_12, MIGRATION_14_15)
.fallbackToDestructiveMigration()
.build()
where MIGRATION_14_15 is:
private val MIGRATION_14_15 = object : Migration(14, 15) {
override fun migrate(database: SupportSQLiteDatabase) {
try {
// database.execSQL create table etc
} catch (e: Exception) {
e.printStackTrace()
// Here is where I want to give up, and start the DB from scratch
}
}
}

The problem you have is that you cannot (at least easily) invoke the fall-back as that is only invoked when there is no migration.
What you could do is to mimic what fall back does (well close to what it does). That is the fall-back will delete (I think) the database file and create the database from scratch and then invoke the databases _Impl (generated java) createAllTables method.
However, you would likely have issues if you deleted the file as the database connection has been passed to the migration.
So instead you could DROP all the app's tables using the code copied from the dropAllTables method from the generated java. You could then follow this with the code from the createAllTables method.
These methods are in the generated java as the class that is the same as the class that is annotated with #Database suffixed with _Impl.
The gotcha, is that the exception
(Expected .... Found ....) that you have shown is NOT within the migration but after the migration when Room is trying to build the database, so you have no control/place to do the above fall-back mimic unless this was done for all 14-15 migrations.
Perhaps what you could do is to trap the exception, present a dialog requesting the user to uninstall the app and to then re-install. This would then bypass the migration as it would be a fresh install.

RealmError: Realm Out of memory size

I am using Realm 3.0.0 as the DB of my Android app. It's like a questionnaire application, in which the user navigates inside the app a lot. When I use the app (go back and forth) continuously, I get the following error:
Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 1073741824 offset: 0 in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109
at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
at io.realm.internal.SharedRealm.(SharedRealm.java:187)
at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:229)
at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:204)
at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:124)
at io.realm.Realm.getDefaultInstance(Realm.java:210)
Now I know the main cause of this is not closing Realm instances. But I've already checked for that multiple times. And I am positive that I close every instance I open.
The app has many activities and fragments that all get a Realm instance on their onCreate and close it on their onDestroy. There are also other background network jobs that run to upload data that get Realm instances. These jobs close their Realm instances when they've finished running or when they cancel.
All of the above get their Realm instance thru injection via Dagger 2:
#Provides
#Nullable
static Realm realm(#Nullable RealmConfiguration configuration) {
if (configuration != null) {
Realm.setDefaultConfiguration(configuration);
return Realm.getDefaultInstance();
}
return null;
}
Configuration is also provided in the same Dagger Module.
To be more specific, a Questionnaire consists of many Question Fragments displayed in a ViewPager. Each Fragment gets injected with a realm. Many interactions in a given Question Fragment write data to the DB (some async, some blocking). These Fragments also query the database on onResume to get their updated Data. Some of this data is also copied out of Realm via realm.copyFromRealm(). Now at any given time of these happening, an upload job is most likely running and reading data from the DB and uploading it to a server. When an upload job finishes, it then writes to the DB.
I think I can have up to 7-12 fragment/activities holding a realm reference on the UI thread at a given moment. And 0-6 other references on 0-3 other threads (Background Jobs).
Moreover, I compact my realm DB via Realm.compactRealm(realmConfiguration) on every app launch (perhaps as a separate problem, this doesn't seem to do it's job consistently).
Above I've tried to describe my Realm usage descriptively without going into details. Now my problem is, when a user excessively uses the app (going back and forth between activities/fragments (realm injection + DB read query), uploading data (realm injection + DB read&write query)), I get the above posted Out of Memory Error.
I am also using Leak Canary, and it hasn't detected any leaks. (Not sure if it can anyway)
Am I using Realm in a way it's not supposed to be used? Should I close Realm instances onPause instead of onDestroy? Should I have only one realm instance in an activity and have all it's fragmetns (up to 5 in my case) use this instance? What kind of changes can I make in my app, and perhaps my app architecture to solve this problem?
I appreciate any help in trying to solve this problem.
EDIT: I'm sharing the realm open-close logic in my background threads.
All my jobs share the same realm usage, which is the following:
Realm is injected lazily via:
#Inject protected transient Lazy<Realm> lazyRealm;
The realm object reference is held at the private transient Realm realm; field. I am using Android Priority Job Queue. When the job is added:
#Override
public void onAdded() {
realm = lazyRealm.get();
realm.executeTransaction(realm1 -> {
//write some stuff into realm
});
realm.close();
}
And when the job is run realm is retreived once, and every possible ending of this method has a call to realm.close()
#Override public void onRun() throws Throwable {
synchronized (syncAdapterLock) {
realm = lazyRealm.get();
Answer answer = realm.where(Answer.class).equalTo(AnswerQuery.ID, answerId).findFirst();
if (answer == null) {
realm.close();
throw new RealmException("File not found");
}
final File photoFile = new File(answer.getFilePath());
final Response response = answerService.uploadPhotoAnswer(answerId, RequestBody.create(MediaType.parse("multipart/form-data"), photoFile)).execute();
if (!response.isSuccessful()) {
realm.close();
throw new HttpError(statusCode);
}
realm.executeTransaction(realm1 -> {
answer.setSyncStatus(SyncStatus.SYNCED.getCode());
});
}
realm.close();
}
}
As you can see, these background threads do close their realm instances properly as far as I'm concerned.

While it was true that all my background tasks did call realm.close(), one of them called it too late in it's lifecycle. That was my GPSService, which is a background service. The problem was that GPS service is initialized at the launch of the App as an Android Service, which is rarely destroyed. I was injecting a realm instance onCreate and closing it onDestroy. After the comments of #EpicPandaForce and reading his articles about using realm properly. I realized that this was the cause of the leak. A non-looper thread was keeping an open realm reference for an extremely long time, thus, the mmap was bloating every time a write transaction occures. Now that I moved the realm get/close to happen every time the service runs, my problem is fixed.
My take away is that one needs to treat background thread realm access very delicately. Thank you both for your quick responses and help!

Which conditions cause SQLiteDiskIOException code 3850

I have an app that uses ContentProvider to access SQLite. An instance of SQLiteOpenHelper is created in providers onCreate:
#Override
public boolean onCreate() {
final Context context = getContext();
mDBHelper = new MyDatabase(context);
return true;
}
SQLiteDatabase instances retrieved in methods insert/update/delete/query are not manually closed. None of these methods are marked synchronized. ContentProvider is accessed from multiple threads started from UI and services.
Sample stacktrace:
android.database.sqlite.SQLiteDiskIOException: disk I/O error (code 3850)
at android.database.sqlite.SQLiteConnection.nativeExecuteForChangedRowCount(Native Method)
at android.database.sqlite.SQLiteConnection.executeForChangedRowCount(SQLiteConnection.java:734)
at android.database.sqlite.SQLiteSession.executeForChangedRowCount(SQLiteSession.java:754)
at android.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:64)
at android.database.sqlite.SQLiteDatabase.updateWithOnConflict(SQLiteDatabase.java:1574)
at android.database.sqlite.SQLiteDatabase.update(SQLiteDatabase.java:1520)
at com.sample.provider.MyProvider.update(SourceFile:0)
at android.content.ContentProvider$Transport.update(ContentProvider.java:260)
at android.content.ContentResolver.update(ContentResolver.java:1040)
Things worked fine up to the moment when I added a class that serializes writes to certain tables using a Handler initialized by Looper from HandlerThread. After this I started seeing plenty of SQLiteDiskIOExceptions with error codes 3850 and 0 (not an error?). Interestingly 90% of these crashes occur with Nexus 4 and on a handful of other devices.
I have been running Unit tests trying to simulate the condition but have been unable to reproduce the problem. There are other questions that already discuss related issues (e.g. here: Synchronize access to Content Provider) but to me the original cause for this error seems still a bit unclear. So what really are the reasons for error 3850?

net.sqlcipher.database.SQLiteException while opening database after resuming app

I'm getting the following error when using the encrypted SQLCipher database in my Android app, but only off and on:
net.sqlcipher.database.SQLiteException: not an error
at net.sqlcipher.database.SQLiteDatabase.dbopen(Native Method)
at net.sqlcipher.database.SQLiteDatabase.<init>(SQLiteDatabase.java:1950)
at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:900)
at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:947)
at net.sqlcipher.database.SQLiteOpenHelper.getReadableDatabase(SQLiteOpenHelper.java:195)
at com.android.storage.DatabaseHelper.getReadable(DatabaseHelper.java:99)
...
I've got the proper files in the assets/ and libs/ folders because the database works fine most of the time. However, every once in awhile I'll see this error. I've seen this twice now on my phone and it's always been after resuming the app after hours of inactivity (I check for user's oauth token in db if it gets cleared from memory).
I call "SQLiteDatabase.loadLibs(this)" only from the Application::onCreate() method so my hunch is that this isn't getting called on a resume and is throwing the error. Does this sound possible? If so, where should I call loadLibs? A user could enter the app in any activity and I access the db if the token isn't in memory. I see my options as either calling loadLibs on each Activity::onCreate or calling it each time I attempt to open the db. Would it cause any harm or performance issues if I called it multiple times like this?

You might consider moving the SQLiteDatabase.loadLibs(this); to your application subclass of net.sqlcipher.database.SQLiteOpenHelper. You can then pass the static instance of your Application subclass as its argument. Something like the following might be an example:
public class SchemaManager extends net.sqlcipher.database.SQLiteOpenHelper {
private static SchemaManager instance;
public static synchronized SchemaManager getInstance() {
if(instance == null) {
SQLiteDatabase.loadLibs(YourApplication.getInstance());
instance = new SchemaManager(…)
}
return instance;
}
}
With regard to the exception that was provided, the Java routine calls into a JNI layer that calls sqlite3_open_v2, setting the soft heap limit and setting the busy timeout. I would suggest adding logging locally to verify you are passing a valid path and a non null passphrase when attempting to acquire the SQLiteDatabase instance when you get a crash. Calling SQLiteDatabase.loadLibs(this); multiple times shouldn't cause a noticeable performance impact, much of what occurs are calls to System.loadLibrary(…) which get mapped into Runtime.getRuntime().loadLibrary(…), once a dynamic library has been loaded, subsequent calls are ignored.