I had this data class:
data class User(
val name:String,
val age: Int
)
I save users' data with this in my database, and now the problem is I want to add a new field
val gender: String
and if I added this the app crashed when retrieving users' data from the database because it is not the same data any more.
This is my code to retrieve data from the database:
#Query("SELECT*FROM users WHERE name =:name")
suspend fun getUser(name:String):Flow<User>
I would be most interested in seeing a pseudocode solution if possible.
You need to do a migration of your database. You can find all the instructions to do it there Migrating Room databases.
If your room version is 2.4.0-alpha01 or higher:
Replace in your database:
#Database(
version = 1,
entities = {User.class}
)
with:
#Database(
version = 2,
entities = {User.class},
autoMigrations = {
#AutoMigration (from = 1, to = 2)
}
)
Else you will need to du the migration manually:
Add this to your Database class:
static final Migration MIGRATION_1_2 = new Migration(1, 2) {
#Override
public void migrate(SupportSQLiteDatabase database) {
database.execSQL("ALTER TABLE users "
+ " ADD COLUMN gender TEXT");
}
};
and add .addMigrations(MIGRATION_1_2) to your Room.databaseBuilder()
You have two solutions, the right one and the fast one:
The right one:
#Database(
version = 2,
entities = [User::class],
autoMigrations = [
AutoMigration (from = 1, to = 2)
]
)
Code taken from: https://developer.android.com/training/data-storage/room/migrating-db-versions#automated
This is a migration , since in this particular case you are only adding a column to a table you can try this approach and it will most likely work. If it doesn't then you have to implement your own migration with some SQL code.
The fast one:
Wherever you are creating your data base, add the following method: .fallbackToDestructiveMigration() . This will make that whenever your database version changes and a migration is needed and not provided it will just delete the old database and recreate it. Take into account that this solution will wipe out all the info you had saved up until that point, so try to use it only if you are in a rush and don't care to maintain the data between updates.
I am using Room with a prepopulated database in the assets folder. For an app update, I would like to alter this database by adding a new column and prepopulating this column with new data.
The database was auto-migrated from version 1 to 2 (a table was added). From version 2 to 3, I would now like to apply abovementioned changes by providing a different 'database.db' file in the assets folder and allowing for destructive migration.
#Database(entities = [Object1::class, Object2::class], version = 3, autoMigrations = [
AutoMigration (from = 1, to = 2)], exportSchema = true)
abstract class AppDatabase : RoomDatabase() {
abstract fun dao(): Dao
companion object {
private const val DB_NAME = "database.db"
#Volatile
private var instance: AppDatabase? = null
fun getInstance(context: Context): AppDatabase {
return instance ?: synchronized(this) {
instance ?: buildDatabase(context).also { instance = it }
}
}
private fun buildDatabase(context: Context): AppDatabase {
return Room.databaseBuilder(
context,
AppDatabase::class.java, "AppDB.db")
.fallbackToDestructiveMigration()
.createFromAsset(DB_NAME)
.build()
}
}
}
The problem is that I still get the following exception:
java.lang.IllegalStateException: A migration from 1 to 3 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods.
I am unsure why this would still happen. I thought it was either providing a migration script or allowing for destructive migration that makes the migration work.
Added Comment:-
I have tried an implemented migration, but the same exception as above happened again. When I try starting over with versionCode 1, I am getting "java.lang.IllegalStateException: Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number. You can simply fix this by increasing the version number." I have also changed the database name and added android:allowBackup="false" in the manifest.
Any ideas?
I had problems using fallbackToDestructiveMigration and createFromAsset together. I would like to share my experience because it took me hours to find it. When you provide an asset db, you have to update the user version pragma of the default database file that you are providing with createFromAsset. If not, you always lose the data that you insert while the app is working.
I finally figured out what the problem was, it had nothing to do with the versioning or anything else related to room or the asset db file.
It was dependency injection.
I provided my database to Dagger in a DatabaseModule class as follows:
private const val DB_NAME = "database.db"
#InstallIn(SingletonComponent::class)
#Module
class DatabaseModule {
#Provides
fun provideDao(appDatabase: AppDatabase): Dao {
return appDatabase.dao()
}
#Provides
#Singleton
fun provideAppDatabase(#ApplicationContext appContext: Context): AppDatabase {
return Room.databaseBuilder(
appContext,
AppDatabase::class.java, "AppDB.db")
.createFromAsset(DB_NAME)
.build()
}
}
It was missing the fallBackToDestructiveMigration() call, so this messed up Room's internal onUpgrade call in RoomOpenHelper.java.
To fix it, I made my buildDatabase call in AppDatabase public and used it to provide the database to Dagger in the DatabaseModule class.
Digging through the room documentation doesn't turn much up, my hunch is that it has to do with the fact that you are using Automigrations instead of implemented migrations. Have you tried changing that Automigration from 1->2 to an implemented migration?
Also, since you are manually replacing it with a new database that has prepopulated data my solution would be to just get rid of the old migrations, change the name of the DB slightly and start over from version 1. There's no reason to maintain the old migrations if anyone going from older versions to the current version are having their DB deleted.
After extensive methodical testing, the only way that I can replicate your (1-3 required) failure is by excluding fallbackToDestructiveMigation. In which case the exception happens if the migration is from 1 to 3 or the migration is 3 to 1 (i.e. Asset Version at 3 but Room version at 1)
as per the spreadsheet screenshot below
1-3 exception when AssetDB Version =3 Database Version = 1 Room Version = 3
also 3-1 exception when AssetDB Version =3 Database Version = -1 Room Version = 1
-1 version means file does not exist (i.e. initial install)
I suspect that you have somehow inadvertently introduced one of the above two scanrios. What I haven't tested is alternative Room library versions. The above was tested with 2.4.0-alpha04 as per :-
implementation 'androidx.core:core-ktx:1.6.0'
implementation 'androidx.appcompat:appcompat:1.3.1'
implementation 'com.google.android.material:material:1.4.0'
implementation 'androidx.constraintlayout:constraintlayout:2.1.0'
implementation 'androidx.room:room-ktx:2.4.0-alpha04'
implementation 'androidx.room:room-runtime:2.4.0-alpha04'
testImplementation 'junit:junit:4.+'
androidTestImplementation 'androidx.test.ext:junit:1.1.3'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
kapt 'androidx.room:room-compiler:2.4.0-alpha0
For the testing, I had two copies of the asset file, one at version 1 the other at version 2 (v1dbbase.db and v3dbbase.db), the data in a common column indicating the if the data was for version3. The actual asset file used was deleted before a test and the appropriate version copied and pasted to database.db
I had the two entities Object1 and Object2 and could comment in or out an extra column in either. e.g.:-
/*TESTING INCLUDE FOR V2+ >>>>>*///, #ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
- as above it is excluded
/*TESTING INCLUDE FOR V2+ >>>>>*/, #ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
- with the two //'s before the comma now included
both the extra columns commented out = Version 1
Object1's extra column included = Version 3
Object1's and Object2's extra column included = Version 3
Object2's extra column included but not Object1's was not considered.
A few constants were added to cater for logging.
Additionally to cater for logging a callback function was added (.addCallback) and onOpen, onCreate and onDestructiveMigration were all overridden to log the Room Version and Database Version.
To further enhance the logging, two functions were added, to get the version from the sqlite database header. One for the asset file, the other for the database. The functions being called/invoked BEFORE the database build.
To run a test it meant:-
Ensuring that the device had the App at the appropriate level.
Deleting the database.db asset
Copying and pasting the appropriate asset file as database.db (from either v1dbbase.db or v3dbbase.db)
Amending the Object1 class to include/exclude the extra column (as explained above)
Amending the Object2 class to include/exclude the extra columns (as explained above)
Amended the Room Version to the appropriate level.
The code used for testing:-
Object1
#Entity(tableName = TABLE_NAME)
data class Object1(
#PrimaryKey
#ColumnInfo(name = COL_ID)
val object1_id: Long,
#ColumnInfo(name = COL_NAME)
val object1_name: String
/*TESTING INCLUDE FOR V2+ >>>>>*///, #ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
) {
companion object {
const val TABLE_NAME = "object1"
const val COL_ID = TABLE_NAME + "_object1_id"
const val COL_NAME = TABLE_NAME + "_object1_name"
const val COL_EXTRAV2 = TABLE_NAME + "_object1_extrav2"
}
}
Object2
#Entity(tableName = TABLE_NAME)
data class Object2(
#PrimaryKey
#ColumnInfo(name = COL_ID)
val object2_id: Long,
#ColumnInfo(name = COL_NAME)
val object2_name: String
/*TESTING INCLUDE FOR V3>>>>>*///, #ColumnInfo(name = COL_EXTRAV3, defaultValue = "x") val object3_extrav3: String
) {
companion object {
const val TABLE_NAME = "object2"
const val COL_ID = TABLE_NAME + "_object2_id"
const val COL_NAME = TABLE_NAME + "_object2_name"
const val COL_EXTRAV3 = TABLE_NAME + "_object2_extrav3"
}
}
Dao
#Dao
abstract class Dao {
#Insert
abstract fun insert(object1: Object1): Long
#Insert
abstract fun insert(object2: Object2): Long
#Query("SELECT * FROM ${Object1.TABLE_NAME}")
abstract fun getAllFromObject1(): List<Object1>
#Query("SELECT * FROM ${Object2.TABLE_NAME}")
abstract fun getAllFromObject2(): List<Object2>
}
AppDatabase
#Database(
entities = [Object1::class, Object2::class],
version = AppDatabase.DBVERSION,
autoMigrations = [AutoMigration (from = 1, to = 2)],
exportSchema = true
)
abstract class AppDatabase : RoomDatabase() {
abstract fun dao(): Dao
companion object {
private const val DB_NAME = "database.db"
private const val DB_FILENAME = "AppDB.db" //<<<<< ADDED for getting header
const val TAG = "DBINFO" //<<<< ADDED for logging
const val DBVERSION = 1 //<<<<<ADDED for logging
#Volatile
private var instance: AppDatabase? = null
fun getInstance(context: Context): AppDatabase {
return instance ?: synchronized(this) {
//ADDED>>>>> to get database version from dbfile and assets before building the database
Log.d(TAG,
"AssetDB Version =${getAssetDBVersion(context, DB_NAME)} " +
"Database Version = ${getDBVersion(context, DB_FILENAME)} " +
"Room Version = ${DBVERSION}")
instance ?: buildDatabase(context).also { instance = it }
}
}
private fun buildDatabase(context: Context): AppDatabase {
return Room.databaseBuilder(
context,
AppDatabase::class.java, DB_FILENAME)
.fallbackToDestructiveMigration()
.createFromAsset(DB_NAME)
.allowMainThreadQueries()
.addCallback(rdc)
.build()
}
/* Call Backs for discovery */
object rdc: RoomDatabase.Callback(){
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
Log.d(TAG,"onCreate called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
}
override fun onOpen(db: SupportSQLiteDatabase) {
super.onOpen(db)
Log.d(TAG,"onOpen called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
}
override fun onDestructiveMigration(db: SupportSQLiteDatabase) {
super.onDestructiveMigration(db)
Log.d(TAG,"onDestructiveMigration called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
}
}
fun getAssetDBVersion(context: Context, assetFilePath: String): Int {
var assetFileHeader = ByteArray(100)
try {
var assetFileStream = context.assets.open(assetFilePath)
assetFileStream.read(assetFileHeader,0,100)
assetFileStream.close()
} catch (e: IOException) {
return -2 // Indicates file not found (no asset)
}
return ByteBuffer.wrap(assetFileHeader,60,4).getInt()
}
fun getDBVersion(context: Context, dbFileName: String): Int {
var SQLiteHeader = ByteArray(100)
val dbFile = context.getDatabasePath(dbFileName)
if(dbFile.exists()) {
var inputStream = dbFile.inputStream()
inputStream.read(SQLiteHeader, 0, 100)
inputStream.close()
return ByteBuffer.wrap(SQLiteHeader, 60, 4).getInt()
} else {
return -1 // Indicates no database file (e.g. new install)
}
}
}
}
you may wish to consider including the logging above, it could very easily detect issues with the version(s) being used.
MainActivity
class MainActivity : AppCompatActivity() {
lateinit var db: AppDatabase
lateinit var dao: Dao
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
db = AppDatabase.getInstance(this)
dao = db.dao()
for(o1: Object1 in dao.getAllFromObject1()) {
logObject1(o1)
}
for(o2: Object2 in dao.getAllFromObject2()) {
logObject2(o2)
}
}
fun logObject1(object1: Object1) {
Log.d(TAG,"ID is ${object1.object1_id}, Name is ${object1.object1_name}")
}
fun logObject2(object2: Object2) {
Log.d(TAG,"ID is ${object2.object2_id}, Name is ${object2.object2_name}")
}
companion object {
const val TAG = AppDatabase.TAG
}
}
In addition to utilising the above code and ensuring that the 6 tasks were undertaken I also kept a spreadsheet of the versions and the results e.g. :-
Previous answer (not the case after testing)
I believe that your issue may be with the pre-populated database, in that it's version number (user_version) hasn't been changed to 3.
you can change the version using the SQL (from an SQlite tool ) PRAGMA user_version = 3;
The documentation says :-
Here is what happens in this situation:
Because the database defined in your app is on version 3 and the database instance already installed on the device is on version 2, a migration is necessary.
Because there is no implemented migration plan from version 2 to version 3, the migration is a fallback migration.
Because the fallbackToDestructiveMigration() builder method is called, the fallback migration is destructive. Room drops the database instance that's installed on the device.
Because there is a prepackaged database file that is on version 3, Room recreates the database and populates it using the contents of the prepackaged database file.
If, on the other hand, you prepackaged database file were on version 2, then Room would note that it does not match the target version and would not use it as part of the fallback migration.
By note perhaps by the way of an exception?
Problem - Room DB getting wiped/cleared when doing force update play store update. I am working on a chat messenger application which uses Room DB as local database. Whenever I do a store update with increasing DB version, the local DB gets cleared and messages history are lost.
I'm Using Room DB. My Application is in the Play Store with the use of Room DB and the version is 4.
My Question is I'm changing the 9 tables schema, and now that I update the DB version, each table schema changes. Should I increase the DB version here? How can I accomplish this without losing the user data using Room DB for force update in Play Store? Ex. DB version is 4, I change the two tables’ elements like in the below query.
Do I need to increase DB version twice as two tables are changed or change to one number incremental will be fine? Example: Do I need DB to increase version to 6 OR keeping it 5 is enough?
private val mMigrationMessageStatus: Migration = object : Migration(4, 5) {
override fun migrate(database: SupportSQLiteDatabase) {
database.execSQL("ALTER TABLE message_status RENAME TO MessageStatus")
database.execSQL("ALTER TABLE MessageStatus ADD COLUMN userId TEXT NOT NULL default ''")
}
}
private val mMigrationGroupMember: Migration = object : Migration(4, 5) {
override fun migrate(database: SupportSQLiteDatabase) {
database.execSQL("ALTER TABLE group_member RENAME TO GroupMember")
database.execSQL("ALTER TABLE GroupMember ADD COLUMN userId TEXT NOT NULL default ''")
}
}
return Room.databaseBuilder(context, AppDatabase::class.java, dbName)
.allowMainThreadQueries()
.addMigrations(mMigrationMessageStatus,mMigrationGroupMember)
.build()
From room version 2.4.0, you can easily update using autoMigrations.
DATABASE CLASS
#Database(
version = 3,
autoMigrations = [
AutoMigration(from = 1, to = 2),
AutoMigration(from = 2, to = 3)
],
.....
)
DATA CLASS
#Entity(tableName = "user")
data class DataUser(
....
// I added this column, like this
#ColumnInfo(defaultValue = "")var test: String = ""
)
see reference below
android developer: room version2.4.0
android developer: autoMigration
I'm using Room and I need to perform a database migration. I've migrated the data but I have a problem in one of the columns. When the migration is performed, the data for that column may still be unavailable.
When the user enters the data needed for that column, I have to get all rows that match a value in that column, update these values by the one provided by the user and drop all other rows that do not match.
I can have a method in my UserDao but the problem is that this does not seem correct because it's a one time only thing and I don't what to expose the method so my idea was to get the database instance and try to do the change myself.
When I use
var myDatabase = Room.databaseBuilder(....)
.addMigrations(... .build()
I keep a reference to it but then, when I do myDatabase.openHelper.writableDatabase I'm always getting an exception
getDatabase called recursively
Any idea how to handle this?
Your issue is that you are trying to use the MyDatabase's openHelper to try to get the database when building the instance of MyDatabase which is in the process of getting the database, so while getting the database you are then trying to get the database.
Instead you need to use the SupportSQLiteDatabase that is passed to the Migration.
As ean example :-
#Database(
version = 1,
entities = [
MyTableEntity::class
]
)
abstract class AppDatabase : RoomDatabase() {
abstract fun MyTableEntityDao(): MyTableEntityDao
companion object {
val MIGRATION_V1_V2: Migration = object : Migration(1, 2) {
override fun migrate(database: SupportSQLiteDatabase) {
//........ code using the already opened database
database.execSQL(????????); //<<<<<<<<<< USES the support database
}
}
}
}
This would then be invoked using something similar to :-
var myDatabase = Room.databaseBuilder(applicationContext,AppDatabase::class.java,"mydatabase")
.allowMainThreadQueries()
.addMigrations(AppDatabase.MIGRATION_V1_V2)
.build()
Why am I getting error Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number. You can simply fix this by increasing the version number. even after adding .fallbackToDestructiveMigration() in Android?
private fun buildDatabase(context: Context): AppDatabase {
val appDatabase = Room.databaseBuilder(context.applicationContext,
AppDatabase::class.java,
DATABASE_NAME
)
if (BuildConfig.DEBUG) {
appDatabase.fallbackToDestructiveMigration()
}
return appDatabase.build()
}
I think you forgot to increase the database version. You have to increase the database version after you update the database schemas.
#Database(
entities = [SampleEntity::class],
version = 1
)
abstract class AppDatabase : RoomDatabase() {
}