Develop Reference

Android: Migrating from legacy SQLite to Room database using attach

I'm working on an Android app where we need to migrate from some legacy SQLite code to a Room based implementation. But I've run into a couple of issues & I'm not sure how to solve them.
Environment:
Android 6.0+
SQLite implementation is in one database file
Room implementation is in a separate database file
Both will be in /data/user/0/.../databases/
What sorta works, but is slow:
We have a working test implementation to pull all the legacy SQLite tables' data into Kotlin objects, then do inserts into the Room based tables. But, it's fairly slow for our test cases (about 70 secs for 10k+ rows in at least one table). Real life cases could top 100k rows in at least one table. As far as I can tell, the select is one transaction and the inserts are wrapped in a transaction. So, I don't think it's transaction overhead slowing us down.
What we want to make work:
What I'm hoping to do is use SQLite's attach database and detach commands to load the legacy SQLite database into a Room DB connection. Then, I can do an insert into new_table.table_name select * from old_table.table_name. The actual SQL will be slightly different due to slight schema changes. But, the insert-select pattern will be the basic idea.
I'm using ContextWrapper to get the full database path for the legacy database file. That works fine.
What issues I'm seeing:
When I try to do the attach command, though, I was getting an IllegalStateException with something about not being able to enable/disable write-ahead logging while in a transaction or the db is open. But, the legacy DB is not open or in a transaction at that point.
So, I modified the approach to set the journal mode to TRUNCATE in the Room db connection setup. That allowed me to do the attach command. And the insert-select query seems to work as far as I can tell (I haven't fully tested that yet). BUT, when I tried the detach command, I got a SQLite 1 error about the database being locked.
For reference, my basic code for both above attach approaches is:
roomDb.execSQL("attach database '$fullLegacyDbPath' as old_db")
roomDb.execSQL($insertSelectQuery) // the "INSERT INTO ... SELECT ..." query
roomDb.execSQL("detach old_db")
Any ideas how I make this work (preferably without setting the journal mode to truncate)?

java.lang.IllegalStateException when assessing ROOM database first time in app

I have a couple applications which were using com.readystatesoftware.sqliteasset.SQLiteAssetHelper to read in read-only databases (never updated during an application session). For some reason, I decided to refactor them to use ROOM.
All works correctly in my emulators and on the one Android phone that I have, but once released the apps are crashing frequently (maybe every time) for my users. This is occurring at the first time in the app where my code is calling a database query.
Here is the stack trace
java.lang.IllegalStateException:
at androidx.room.RoomOpenHelper.onUpgrade (RoomOpenHelper.java:138)
at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper$OpenHelper.onUpgrade (FrameworkSQLiteOpenHelper.java:9)
at android.database.sqlite.SQLiteOpenHelper.getDatabaseLocked (SQLiteOpenHelper.java:400)
at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase (SQLiteOpenHelper.java:298)
at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper$OpenHelper.getWritableSupportDatabase (FrameworkSQLiteOpenHelper.java:4)
at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper.getWritableDatabase (FrameworkSQLiteOpenHelper.java:2)
at androidx.room.SQLiteCopyOpenHelper.getWritableDatabase (SQLiteCopyOpenHelper.java:13)
at androidx.room.RoomDatabase.inTransaction (RoomDatabase.java:2)
at androidx.room.RoomDatabase.assertNotSuspendingTransaction (RoomDatabase.java)
at com.XXX.YYY.model.bah.dao.BahDao_Impl.getInfoFromDatabase (BahDao_Impl.java:13)
at com.XXX.YYY.model.MyClass.getInfoFromDatabase (MyClass.java:20)
at com.XXX.YYY.controller.MyFragment$UpdateBahInfoTask.run (MyFragment.java:17)
at java.lang.Thread.run (Thread.java:764)
These are the first versions of these applications which are using ROOM, so I am not sure why the onUpgrade method is being called or if that is an indication where the issue lies. I am kind of at a loss of where to go with this. My use of these databases is very simple (e.g. just performing queries based on user actions). I cannot replicate this on the emulators or actual device even when upgrading the old app to the new version using adb.
I did start the database version at the same number used by the Sqlite version of the apps, so that is the only thing that maybe could be causing this.
#Database(entities = {Table1.class, Table2.class, Table3.class}, exportSchema = true, version = 14)
public abstract class BahDatabase extends RoomDatabase {
public abstract BahDao bahDao();
}
I am thinking that I will:
increment the database versions
add a call to .addMigrations(MIGRATION_14_15) when I build the database
//Used for creating singleton
private MyClass(Context context) {
mBahDatabase = Room.databaseBuilder(context, BahDatabase.class, DATABASE_NAME_ROOM)
.createFromAsset(DATABASE_ASSET_FULL_PATH)
.addMigrations(MIGRATION_14_15)
.build();
}
And just put a stub for that migration class that will do nothing (as nothing in the database is changing)
static final Migration MIGRATION_14_15 = new Migration(14, 15) {
#Override
public void migrate(SupportSQLiteDatabase database) {
// Since we didn't alter the table, there's nothing else to do here.
Log.d(TAG, "MIGRATION_14_15 Called");
}
};
UPDATE: The above three steps had no effect as the new version of the app is still crashing with this same error.
UPDATE2: Replaced call to addMigrations(MIGRATION_14_15) with call to fallbackToDestructiveMigration(). As my database created from createFromAsset() is never changed by the app, I could care less if Android balks during some ill conceived migration. The JavaDoc states the below which is fine with my design as the database in my assets folder is always what the app should be using (in regards to tables and data):
If the database was create from an asset or a file then Room will try to use the same file to re-create the database, otherwise this will delete all of the data in the database tables managed by Room.
Will see what happens
Any thoughts?
Thanks in advance.

IllegalStateException occurs when your application invokes a method at inappropriate times. Looking at your stack trace. the issues maybe arising from these areas.
....
at com.XXX.YYY.model.bah.dao.BahDao_Impl.getInfoFromDatabase (BahDao_Impl.java:13)
at com.XXX.YYY.model.MyClass.getInfoFromDatabase (MyClass.java:20)
at com.XXX.YYY.controller.MyFragment$UpdateBahInfoTask.run (MyFragment.java:17)
You should check those methods in model and controller files in your code.

So I made that change to add a call to fallbackToDestructiveMigration() when building the database, and that appears to prevent the crashes. So the relevant part of my singleton which builds the database is below:
mBahDatabase = Room.databaseBuilder(context, BahDatabase.class, DATABASE_NAME_ROOM)
.createFromAsset(DATABASE_ASSET_FULL_PATH)
.fallbackToDestructiveMigration()
.build();
Now, I have no idea why this works or what the underlying problem was. To restate:
This was the first version of my application which used a Room database. Previously I used com.readystatesoftware.sqliteasset.SQLiteAssetHelper to copy the database from the asset folder and use the copy as a read-only database in my app.
Therefore, Room should not be doing any method calling related to migrations.
I suspect that Room was having some kind of issue with the old database on a user's phone which was copied over by com.readystatesoftware.sqliteasset.SQLiteAssetHelper in a previous version of my app.
Based on the above, I suspect that this is some odd Android bug, but not many app devs are likely in the scenario that I am in regards to the refactoring to Room.
Luckily, I could care less about what Room did during its migration processing as the database provided in my apk is the one I want to use (also not modified by the app) and any database updates are always provided in a subsequent apk.
No matter what scenarios I created in regards to what was on the emulator/phone before I installed the new app, I could never get the app to crash (like my users were seeing) when using an emulator or an old Android phone.

You cannot run a new Migration(14, 15) when the #Database annotation still says version = 14. It shouldn't throw an IllegalStateException when the #Database annotation would be version = 15. Generally speaking, when the current version is 14, it makes no sense to migrate to 15 (as 14 currently is the latest version in existence). Also see Understanding migrations with Room.

What is the best approach to backup Room database?

Actually I have more than a question. If I uninstall an app, the room database also removed. I debugged and checked using File(getDatabasePath("MyInfoDatabase.db").absolutePath).exists() before instantiating a room database. Let's say my app crashed and I uninstalled it and then reinstall the app. So it is not possible to get the old room data.
Room.databaseBuilder(appContext, AppDatabase.class, "Sample.db")
.createFromFile(File("mypath"))
.build()
If I am not wrong, then I can copy and save the database to a directory and get the old databack while reinstalling using the above method. Is this the right approach? In this approach getExternalStoragePublicDirectory is deprecated now. Should I use MediaStore api for database backup? On the other hand here, it says that room data exists even after uninstalling. Is the room database exists even after uninstalling an app? AFAIK it doesn't exists.

Android Room requires migration after data insertion by another app

I'm developing an Android app with an embedded SQLite database run with Android Room. I'm moving the app from desktop Java to Android and I need to bring data into the Android database from the old database used with the desktop app. I ran into a problem with Room when I tried using the altered database with my Android app.
I tried taking the database file created by Room and move the data from the old database into this one with a short Java app I coded for this purpose. I simply emptied the relevant tables, read data from the old database and inserted it into the new database.
When I tried moving the database file back to my Android devide and using it with the Android app I got this exception:
Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number.
You can simply fix this by increasing the version number.
If I change the version of the database in my Room database code, Room would then require a piece of code for the migration even though the structure of the database has not been changed:
java.lang.IllegalStateException: A migration from 1 to 2 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods.
I tried clearing the app data cache and clearing all of its data and even uninstalling the app, then reinstalling it with the modified database file. Didn't make the problem go away. I also tried everything here Room cannot verify the data integrity including setting android:allowBackup="false" in my Manifest, and adding fallbackToDestructiveMigration to my database. After trying fallbackToDestructiveMigration, all of my new database's data was gone. But then I put the altered database file back in again and it worked. To be honest, I'm not entirely sure what step made it work. I think it was increasing the version number, running it with fallbackToDestructiveMigration and then moving the new database file back in again when fallbackToDestructiveMigration had wiped data from the database...
My question is how can Room know the database has been modified when there hadn't been any change to its structure that would make it incompatible with the Entities of my app? And how can know even though I've cleared the app's data from the device? And are there any steps I should take the next time that are not mentioned above? I quess I could just include migration code that does nothing. But that seems kind of silly, I'd like the version number to stay at 1 since the app is in development at the moment.

Add an empty Migration.
private val MIGRATION_1_2 = object : Migration(1, 2) {
override fun migrate(database: SupportSQLiteDatabase) {
// empty migration.
}
}
if (sInstance == null) {
if (!checkDataBase())
copyDataBase(context)
synchronized(AppDatabase::class) {
sInstance = Room.databaseBuilder(context.applicationContext, AppDatabase::class.java, DATABASE_NAME)
.addMigrations(MIGRATION_1_2)
.build()
}
}
return sInstance!!
You can check out this Migration Guide from Florina Muntenescu to learn more about Migrations

Room attempt to re-open an already closed database

When using Room from the Android Architecture Components, I received the following error when attempting to access the database using a Dagger component:
java.lang.IllegalStateException: attempt to re-open an already-closed object: SQLiteDatabase: (database path)
I was using Dagger version 2.11 and Room version 1.0.0-alpha7. The error was reproducible on version 1.0.0-alpha5.
This error occurred on any attempt to access the database through a DAO after initialising the database and injecting it into my class.

It's because you are trying to modify the schema of the existing database without giving it any migration information. So basically it attempts to write the new database schema to the existing DB which doesn't work.
There are two ways around this. If you are in your development environment what you can do is fallback to a destructive migration, to do this your database creation code would look something like the following:
MyDatabase myDatabase = Room.databaseBuilder(context, MyDatabase.class, "my-db")
.fallbackToDestructiveMigration()
.build();
This means when you provide the database with an updated or new entity it will do what the answer from #huw said and just delete the database on the application's installation removing all the data from it and give you a fresh install.
The other method is to use a migration function. They are pretty long so unless someone wants me to write it up here I'll leave it for now but basically, the documentation can be found here:
Room DB Migration Documentation
This essentially causes the DB to run some SQL provided by yourself to update the database to the new version. This way you can ensure that none of your data is lost while doing the migration; or as little as possible depending on what you are doing. This is the preferred method for production apps as it means users won't lose their pre-existing data and you won't get a lot of angry reviews/lost customers.
Hope that makes helps!

One solution to this problem was to delete the database file and start again. This was not an issue since I was only testing and could repopulate the database using online data.
To do so either:
App info > Storage > Clear Data
Manually remove the file at /data/data/com.app.example/databases/database.db

I had this kind of exception after not so successful migration. Always double-check the SQL query you use for a migration. You can create a new column with a wrong type of data and the exception's description won't be helpful.