In a large scale app, there are chances of my db file (created through android room lib), getting corrupted. For such a user whats the fallback ?
Will room delete the db file and re-create from scratch in production mode or will I have to handle that myself ?
For such a user whats the fallback ?
Backups and restore, noting that backups should not be taken when there are any open transactions. It is best to ensure that if in WAL mode (which is the default mode with room) that the database is fully committed (i.e. the WAL file is empty or doesn't exist).
The backup could be a simple file copy or you can use the VACUUM INTO the latter having the advantage of potentially freeing space but the disadvantage is that it could be more resource intensive.
You could maintain additional databases where changes are applied to other databases as and when the main database is changed. Obviously there would be an overhead. This would have the advantage that if corruption were to occur that it would be less likely to occur in the other databases.
You could maintain a log that allowed roll back and roll forward of changes. The latter being used to roll forward from a backup to the point of corruption or close to the point of corruption.
Will room delete the db file and re-create from scratch in production mode or will I have to handle that myself ?
if detected (when opening) then yes :-
onCorruption
The method invoked when database corruption is detected. Default implementation will delete the database file.
https://developer.android.com/reference/androidx/sqlite/db/SupportSQLiteOpenHelper.Callback#onCorruption(androidx.sqlite.db.SupportSQLiteDatabase)
What happens when room android db gets corrupted?
here's an example of file corruption
2021-10-27 10:36:19.281 7930-7930/a.a.so69722729kotlinroomcorrupt E/SQLiteLog: (26) file is not a database
2021-10-27 10:36:19.285 7930-7930/a.a.so69722729kotlinroomcorrupt E/SupportSQLite: Corruption reported by sqlite on database: /data/user/0/a.a.so69722729kotlinroomcorrupt/databases/thedatabase
2021-10-27 10:36:19.286 7930-7930/a.a.so69722729kotlinroomcorrupt W/SupportSQLite: deleting the database file: /data/user/0/a.a.so69722729kotlinroomcorrupt/databases/thedatabase
2021-10-27 10:36:19.306 7930-7930/a.a.so69722729kotlinroomcorrupt D/TAG: onCreate Invoked.
2021-10-27 10:36:19.312 7930-7930/a.a.so69722729kotlinroomcorrupt D/TAG: onOpen Invoked.
As can be seen by logging from the callbacks that after the file is deleted that onCreate is invoked thus creating a new empty database.
The code used for the above, which you may wish to adapt for testing, is :-
A simple #Entity Something :-
#Entity
data class Something(
#PrimaryKey
val id: Long?=null,
val something: String
)
A Simple #Dao AllDao
#Dao
abstract class AllDao {
#Insert
abstract fun insert(something: Something)
#Query("SELECT * FROM something")
abstract fun getAllFromSomething(): List<Something>
}
The #Database TheDatabase
#Database(entities = [Something::class],version = 1)
abstract class TheDatabase: RoomDatabase() {
abstract fun getAllDao(): AllDao
companion object {
const val DATABASENAME = "thedatabase"
const val TAG = "DBINFO"
private var instance: TheDatabase? = null
var existed: Boolean = false
fun getInstance(context: Context): TheDatabase {
existed = exists(context)
if (exists(context)) {
Log.d(TAG,"Database exists so corrupting it before room opens the database.")
corruptDatabase(context)
}
if (instance == null) {
instance = Room.databaseBuilder(context,TheDatabase::class.java, DATABASENAME)
.allowMainThreadQueries()
.addCallback(cb)
.build()
}
instance!!.openHelper.writableDatabase // Force open
return instance as TheDatabase
}
object cb: Callback() {
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
Log.d("TAG","onCreate Invoked.")
}
override fun onDestructiveMigration(db: SupportSQLiteDatabase) {
super.onDestructiveMigration(db)
Log.d("TAG","onDestructiveMigration Invoked.")
}
override fun onOpen(db: SupportSQLiteDatabase) {
super.onOpen(db)
Log.d("TAG","onOpen Invoked.")
}
}
fun exists(context: Context): Boolean {
val db = File(context.getDatabasePath(DATABASENAME).path)
return db.exists()
}
/**
* Corrupt the database by
* copying the file via buffered reads and write
* BUT only write part of a buffer
* AND skip the 3rd block
* Furthermore, delete the -wal file (possible that this )
* Note that often it is the -wal file deletion that corrupts
*/
fun corruptDatabase(context: Context) {
Log.d("TAG","corruptDatabase Invoked.")
var db: File = File(context.getDatabasePath(DATABASENAME).path)
logFileInfo(db,"Initial")
var tempdb = File(context.getDatabasePath("temp" + DATABASENAME).path)
logFileInfo(tempdb,"Initial")
val blksize = 128
var buffer = ByteArray(blksize)
var i: InputStream
var o: FileOutputStream
try {
i = FileInputStream(db)
o = FileOutputStream(tempdb)
var blocks = 0;
var writes = 0;
while (i.read(buffer) > 0) {
if(blocks++ % 2 == 1) {
writes++
o.write(buffer,buffer.size / 4,buffer.size / 4)
}
}
Log.d(TAG,"${blocks} Read ${writes} Written")
o.flush()
o.close()
i.close()
db = File(context.getDatabasePath(DATABASENAME).path)
logFileInfo(db,"After copy")
tempdb = File(context.getDatabasePath("temp${DATABASENAME}").path)
logFileInfo(tempdb,"After copy")
} catch (e: IOException) {
e.printStackTrace()
}
db.delete()
//(context.getDatabasePath(DATABASENAME+"-wal")).delete()
logFileInfo(db,"After delete")
tempdb.renameTo(context.getDatabasePath(DATABASENAME))
logFileInfo(tempdb,"After rename")
logFileInfo(context.getDatabasePath(DATABASENAME),"After rename/new file")
}
fun logFileInfo(file: File, prefix: String) {
Log.d(TAG,"${prefix} FileName is ${file.name}\n\tpath is ${file.path}\n\tsize is ${file.totalSpace} frespace is ${file.freeSpace}")
}
}
}
Finally an invoking Activity MainActivity
class MainActivity : AppCompatActivity() {
lateinit var db: TheDatabase
lateinit var dao: AllDao
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
db = TheDatabase.getInstance(this);
dao = db.getAllDao()
dao.insert(Something(something = "Something1 " + TheDatabase.existed))
dao.insert(Something(something = "Something2 " + TheDatabase.existed))
for(s: Something in dao.getAllFromSomething()) {
Log.d(TheDatabase.TAG,"Something with ID " + s.id + " is " + s.something)
}
}
}
Note it's the -wal deletion that corrupts for the database above. It is quite resilient in regards to blocks being deleted removed, at least for smaller databases where the WAL file is of sufficient size to recover as such by applying the changes stored within. However, testing based upon the above but with a second table and 100000 rows inserted, then the partial block writes and missed block writes do corrupt the database.
Related
I have 2 fields in the local database(For eg. Name, Password). Now I uploaded the app to the Play Store. After that, I added one field in the database which is mobile number. So now the database has 3 fields(i.e Name, Password, Mobile Number). Now, what happens if I upload this app to the Play Store? Will it affect the database of the old users? How can I update that database without affecting the old local database of the users? I'm using Room Database
The update will be rolled out, via PlayStore, to old users unless it is a different App.
You MUST update the old users otherwise the App will crash. However, you can retain their data but you must cater for the new column.
As the schema has changed (a new column) and if there isn't a migration old users will experience a crash as Room checks to see if the schema, as per the #Entity annotated class (what is expected) against the database (what is found).
The crash would be along the lines of: java.lang.IllegalStateException: Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number. You can simply fix this by increasing the version number. Expected identity hash: e843da3b4913dbc08880c558d759fe82, found: d5c32de20cfd495f9eae5463c1ec7433
hashes will differ (expected(1st) is as per the #Entity the found is as per the schema in the existing database)
What you need to do is
set the default value to a suitable value that indicates that no mobile number has been provided, and
add a migration that introduces the new column, and
increase the version number (which will invoke the migration, perform the migration and then processing continues to the check/open of the database).
if there is no Migration then a crash will ensue e.g. java.lang.IllegalStateException: A migration from 1 to 2 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods.
Demo
The following is a demo that will first create the App with the database at V1 without the Mobile field/column and then will migrate the existing database when the database is upgraded to V2. The existing users will have a value that indicates no mobile.
First the Database code for both versions with the V2 code commented out (The Migration doesn't need to be commented out but would obviously not be present for V1 (just saves having to repeat code)):-
const val DATABASE_VERSION = 1 /*<<<<<<<<<< WILL CHANGE to 2 FOR V2 */
const val USER_TABLE_NAME = "user"
const val USER_NAME_COLUMN = "name"
const val USER_PASSWORD_COLUMN = "password"
#Entity(tableName = USER_TABLE_NAME)
data class User(
#PrimaryKey
#ColumnInfo(name = USER_NAME_COLUMN)
val name: String, /* Original */
#ColumnInfo(name = USER_PASSWORD_COLUMN)
val password: String /* Original */
#Dao
interface UserDAOs {
#Insert(onConflict = OnConflictStrategy.IGNORE)
fun insert(user: User): Long
#Query("SELECT * FROM user")
fun getAllUsers(): List<User>
}
#Database(entities = [User::class], exportSchema = false, version = DATABASE_VERSION)
abstract class TheDatabase: RoomDatabase() {
abstract fun getUserDAOs(): UserDAOs
companion object {
private var instance: TheDatabase?=null
fun getInstance(context: Context): TheDatabase {
if (instance==null) {
instance=Room.databaseBuilder(context,TheDatabase::class.java,"the_database.db")
.allowMainThreadQueries() /* for brevity of the demo */
.build()
}
return instance as TheDatabase
}
}
}
Now some activity code to load some V1 data:-
class MainActivity : AppCompatActivity() {
lateinit var db: TheDatabase
lateinit var dao: UserDAOs
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
db = TheDatabase.getInstance(this)
dao = db.getUserDAOs()
dao.getAllUsers() /*<<<< force open the database in case no code runs (this when the version and schema checking and migration for V2 will take place ) */
if (DATABASE_VERSION == 1) {
dao.insert(User("Fred", "passwordFred")) /* Original */
dao.insert(User("Mary", "passwordMary")) /* Original */
}
/* commented out for V1 as mobile not a field in the User */
/*
if (DATABASE_VERSION == 2) {
dao.insert(User("Jane","passwordJane","1234567890"))
dao.insert(User("John","passwordJohn","0987654321"))
dao.insert(User("Pat","passwordPat"))
}
*/
}
}
When run for a fresh install (aka old user) then the database, via App Inspection:-
room_master_table is where the schema hash is stored and will be the found
as expected the two rows exist and have expected values.
Next the code is changed.
The database code becomes:-
The Database version is increased:-
const val DATABASE_VERSION = 2 /*<<<<<<<<<< WILL CHANGE to 2 FOR V2 */
2 new const vals are added:-
const val USER_MOBILE_COLUMN = "mobile" /*<<<<<<<<<< ADDED for V2 */
const val USER_MOBILE_DEFAULT_VALUE = "xxxxxxxxxx" /*<<<<<<<<<< ADDED for V2 */
The User class becomes:-
#Entity(tableName = USER_TABLE_NAME)
data class User(
#PrimaryKey
#ColumnInfo(name = USER_NAME_COLUMN)
val name: String, /* Original */
#ColumnInfo(name = USER_PASSWORD_COLUMN)
val password: String /* Original */ ,/*<<<<<<<<< ADDED comma FOR V2 */
/*<<<<<<<<<< SCHEMA CHANGES FOR V2 (see comma above) >>>>>>>>>>*/
#ColumnInfo(name = USER_MOBILE_COLUMN, defaultValue = USER_MOBILE_DEFAULT_VALUE) /*<<<<<<<<<< ADDED FOR V2 */
val mobile: String = "not provided" /*<<<<<<<<<< ADDED for V2 (default value allows mobile to not be given for V1 code in Main Activity)*/
)
The #Database annotated class TheDatabase has the migration added:-
#Database(entities = [User::class], exportSchema = false, version = DATABASE_VERSION)
abstract class TheDatabase: RoomDatabase() {
abstract fun getUserDAOs(): UserDAOs
companion object {
private var instance: TheDatabase?=null
fun getInstance(context: Context): TheDatabase {
if (instance==null) {
instance=Room.databaseBuilder(context,TheDatabase::class.java,"the_database.db")
.allowMainThreadQueries() /* for brevity of the demo */
.addMigrations(MIGRATE_1_to_2)
.build()
}
return instance as TheDatabase
}
val MIGRATE_1_to_2: Migration = object: Migration(1,2){
override fun migrate(db: SupportSQLiteDatabase) {
db.execSQL("ALTER TABLE $USER_TABLE_NAME ADD COLUMN $USER_MOBILE_COLUMN TEXT NOT NULL DEFAULT '$USER_MOBILE_DEFAULT_VALUE'")
/* So as to show Migration add a row when migrating (would not be done normally) */
val cv = ContentValues()
cv.put(USER_NAME_COLUMN,"Alice")
cv.put(USER_PASSWORD_COLUMN,"passwordAlice")
cv.put(USER_MOBILE_COLUMN,"1111111111")
db.insert(USER_TABLE_NAME,OnConflictStrategy.IGNORE,cv)
}
}
}
}
The commented out activity code is un-commented for V2:-
class MainActivity : AppCompatActivity() {
lateinit var db: TheDatabase
lateinit var dao: UserDAOs
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
db = TheDatabase.getInstance(this)
dao = db.getUserDAOs()
dao.getAllUsers() /*<<<< force open the database in case no code runs */
if (DATABASE_VERSION == 1) {
dao.insert(User("Fred", "passwordFred")) /* Original */
dao.insert(User("Mary", "passwordMary")) /* Original */
}
/* commented out for V1 as mobile not a field in the User */
if (DATABASE_VERSION == 2) {
dao.insert(User("Jane","passwordJane","1234567890"))
dao.insert(User("John","passwordJohn","0987654321"))
dao.insert(User("Pat","passwordPat"))
}
}
}
When the App is run then App Inspection now shows:-
As can be seen:-
Fred and Mary have have the recognisable indicator that the mobile wasn't provided i.e. it is xxxxxxxxxx
Alice has been added as part of the Migration (not that this would normally be included, it is just to show that the migration was performed)
Jane and John have been added with their provided mobile numbers
Pat has been added with the default value, as per the field default value (the database default value cannot be applied as mobile is not nullable)
Final Test
The remaining proof of concept, is when a new user installs the App i.e. a fresh/new install. In this scenario , for the demo, just the three V2 users will be inserted (Jane, John and Pat):-
Obviously the inserts are reflecting what the App user may do
I have 2 db files in asset. I have to prepopulate 2 tables
#Database(entities = [Quotes::class, Anime::class], version = 1, exportSchema = true)
Here I have tried something but it isn't working
#Provides
#Singleton
fun provideDatabase(
app: Application,
)= Room.databaseBuilder(app, QuotesDatabase::class.java, "quotes_database")
.createFromAsset("quotess.db")
.createFromAsset("animes.db")
.fallbackToDestructiveMigration()
.build()
You have a few options.
The simplest way would be to combine the two into a single asset. This could be done using one of the SQLite tools (SQliteStudio, DBeaver, Navicat for SQLite, DB Browser for SQLite).
You could, without using createFromAsset allow Room to build the database and the open each asset in turn and copy the data in the onCreate callback. When the onCreate callback is invoked, the database has been created along with the tables and it is passed to the callback as a SupportSQLiteDatabase. You could then copy each asset to a suitable location (databases directory), open the asset as an SQLiteDatabase, for each table, extract the data into a Cursor and then load the data from the Cursor into the SupportSQliteDatabase. You can the close the two SQLiteDatabase and then delete them.
A third option, would be to, prior to building the Room database, create the database (according to the SQL that can be found in the generated java), copying the two asset to a suitable location (databases directory) attaching both to the created database, copying the data from both to the respective tables detach the two asset databases and delete them. Then when building the Room database it will exist and be opened.
I don't believe that you need to, but you may have to set the user version of the created database.
Here's an in-principle (untested) example of the second option that you may find useful.
:-
#Database(entities = [Quotes::class,Anime::class], exportSchema = false, version = 1)
abstract class QuotesDatabase: RoomDatabase() {
abstract fun getAllDao(): AllDao
companion object {
#Volatile
private var instance: QuotesDatabase?=null
private var quotesAssetFileName = "quotess.db" /* CHANGE AS REQUIRED */
private var animeAssetFileName = "animes.db" /* CHANGE AS REQUIRED */
private var quotesTableName = "quotes_table" /* CHANGE AS REQUIRED */
private var animeTablename = "anime_table" /* CHANGE AS REQUIRED */
fun getInstance(context: Context): QuotesDatabase {
if (instance==null) {
instance = Room.databaseBuilder(context,QuotesDatabase::class.java,"quotes_database")
.addCallback(cb)
.build()
}
return instance as QuotesDatabase
}
val cb = object: Callback() {
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
db.beginTransaction() /* MAY NOT BE ABLE TO BE USED - IF NOT REMOVE (as well as similar below)*/
copyAsset(Application().applicationContext, quotesAssetFileName)
val asset1db = SQLiteDatabase.openDatabase(Application().getDatabasePath(quotesAssetFileName).path,null,0)
populateFromCursor(asset1db.rawQuery("SELECT * FROM $quotesTableName",null), quotesTableName,db)
copyAsset(Application().applicationContext, animeAssetFileName)
val asset2db = SQLiteDatabase.openDatabase(Application().getDatabasePath(animeAssetFileName).path,null,0)
populateFromCursor(asset2db.rawQuery("SELECT * FROM $animeTablename",null), animeTablename,db)
db.setTransactionSuccessful() /* MAY NOT BE ABLE TO BE USED - IF NOT REMOVE (as well as similar below)*/
db.endTransaction() /* MAY NOT BE ABLE TO BE USED - IF NOT REMOVE (as well as similar below)*/
deleteAssetCopy(asset1db)
deleteAssetCopy(asset2db)
}
}
/* Populates the Room database using the extracted data (Cursor) from the asset copy database */
#SuppressLint("Range")
fun populateFromCursor(csr: Cursor, tableName: String, db: SupportSQLiteDatabase) {
val cv = ContentValues()
while (csr.moveToNext()) {
for (c in csr.columnNames) {
cv.put(c,"'" + csr.getString(csr.getColumnIndex(c))+"'")
}
db.insert(tableName,OnConflictStrategy.IGNORE,cv)
}
csr.close()
}
/* Copies the asset to the asset database */
fun copyAsset(context: Context, assetFileName: String) {
val asset = context.assets.open(assetFileName)
val db = context.getDatabasePath(assetFileName)
val os: OutputStream = db.outputStream()
asset.copyTo(os)
}
/* Deletes the copied assets database */
fun deleteAssetCopy(db: SQLiteDatabase) {
File(db.path).delete()
}
}
}
I have converted my app to use Android Room for SQLite DB. There are some crashes on different devices with my implementation.
Fatal Exception: android.database.sqlite.SQLiteReadOnlyDatabaseException: attempt to write a readonly database (code 1032 SQLITE_READONLY_DBMOVED)
at android.database.sqlite.SQLiteConnection.nativeExecute(SQLiteConnection.java)
at android.database.sqlite.SQLiteConnection.execute(SQLiteConnection.java:707)
at android.database.sqlite.SQLiteConnection.setLocaleFromConfiguration(SQLiteConnection.java:473)
at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:261)
at android.database.sqlite.SQLiteConnection.open(SQLiteConnection.java:205)
at android.database.sqlite.SQLiteConnectionPool.openConnectionLocked(SQLiteConnectionPool.java:505)
at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:206)
at android.database.sqlite.SQLiteConnectionPool.open(SQLiteConnectionPool.java:198)
at android.database.sqlite.SQLiteDatabase.openInner(SQLiteDatabase.java:918)
at android.database.sqlite.SQLiteDatabase.open(SQLiteDatabase.java:898)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:762)
at android.database.sqlite.SQLiteDatabase.openDatabase(SQLiteDatabase.java:751)
at android.database.sqlite.SQLiteOpenHelper.getDatabaseLocked(SQLiteOpenHelper.java:373)
at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:316)
at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper$OpenHelper.getWritableSupportDatabase(FrameworkSQLiteOpenHelper.java:145)
at androidx.sqlite.db.framework.FrameworkSQLiteOpenHelper.getWritableDatabase(FrameworkSQLiteOpenHelper.java:106)
at androidx.room.SQLiteCopyOpenHelper.getWritableDatabase(SQLiteCopyOpenHelper.java:97)
at androidx.room.RoomDatabase.internalBeginTransaction(RoomDatabase.java:482)
at androidx.room.RoomDatabase.beginTransaction(RoomDatabase.java:471)
at com.luzeon.MyApp.sqlite.ViewLogDao_Impl$5.call(ViewLogDao_Impl.java:94)
at com.luzeon.MyApp.sqlite.ViewLogDao_Impl$5.call(ViewLogDao_Impl.java:91)
at androidx.room.CoroutinesRoom$Companion$execute$2.invokeSuspend(CoroutinesRoom.kt:61)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at androidx.room.TransactionExecutor$1.run(TransactionExecutor.java:47)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:923)
I have created the apps Room DB
#Database(entities = [ViewLogModel::class], version = 4)
abstract class MyAppDatabase : RoomDatabase() {
abstract fun viewLogDao(): ViewLogDao
companion object {
// For Singleton Instance
#Volatile
private var INSTANCE: MyAppDatabase? = null
fun getAppDataBase(context: Context): MyAppDatabase {
return INSTANCE ?: synchronized(this) {
INSTANCE ?: Room.databaseBuilder(context.applicationContext, MyAppDatabase::class.java, "MyAppDatabase")
.createFromAsset(“myapp.db")
.setJournalMode(RoomDatabase.JournalMode.TRUNCATE) // disable WAL
.fallbackToDestructiveMigration()
.build()
}
}
fun destroyDataBase(){
INSTANCE = null
}
}
}
And have a DB Helper class
class MyAppDatabaseHelper(private val context: Context, private val coroutineScope: CoroutineScope) {
fun updateViewLog(viewLogModel: ViewLogModel) {
try {
// get the database
val db = MyAppDatabase.getAppDataBase(context)
coroutineScope.launch {
// store in db
db.viewLogDao().insertOrUpdateViewLog(viewLogModel)
}
} catch (e: Exception) {}
}
suspend fun getViewLog(memberId: Int): JSONArray {
try {
val jsonArray = JSONArray()
// get the database
val db = MyAppDatabase.getAppDataBase(context)
val viewLog = db.viewLogDao().getViewLog(memberId)
for (view in viewLog) {
// create the object
val jsonObject = JSONObject()
try {
jsonObject.put("mid", view.mid)
} catch (e: JSONException) {
}
try {
jsonObject.put("uts", view.uts)
} catch (e: JSONException) {
}
jsonArray.put(jsonObject)
}
// clear log (current user records or records older than 24hrs)
db.viewLogDao().deleteViewLog(memberId, Utilities.getUtsYesterday().toFloat())
// return the array
return jsonArray
} catch (e: Exception) {
return JSONArray()
}
}
}
With ViewLogDao
#Dao
interface ViewLogDao {
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insertOrUpdateViewLog(viewLog: ViewLogModel)
#Update
suspend fun updateViewLog(viewLog: ViewLogModel)
#Delete
suspend fun deleteAllViewLog(viewLog: ViewLogModel)
#Query("DELETE FROM viewlog WHERE VID= :vid OR UTS < :uts")
suspend fun deleteViewLog(vid: Int, uts: Float)
#Query("SELECT * FROM viewLog")
suspend fun getAll(): List<ViewLogModel>
#Query("SELECT * FROM viewLog WHERE vid = :vid")
suspend fun getViewLog(vid: Int): List<ViewLogModel>
}
And ViewLogModel
#Entity(tableName = "viewLog")
data class ViewLogModel(
#ColumnInfo(name = "VID") val vid: Int,
#ColumnInfo(name = "UTS") val uts: Float,
#ColumnInfo(name = "MID") #PrimaryKey val mid: Int)
I have not been able to find how to catch the SQLiteReadOnlyDatabaseException in the rare occurrences when the DB is read only. Or is there a way to ensure the ROOM Db is read/write?
I have not been able to find how to catch the SQLiteReadOnlyDatabaseException in the rare occurrences when the DB is read only. Or is there a way to ensure the ROOM Db is read/write?
The message code 1032 SQLITE_READONLY_DBMOVED :-
The SQLITE_READONLY_DBMOVED error code is an extended error code for SQLITE_READONLY. The SQLITE_READONLY_DBMOVED error code indicates that a database cannot be modified because the database file has been moved since it was opened, and so any attempt to modify the database might result in database corruption if the processes crashes because the rollback journal would not be correctly named.
If the message is to be believed then the database has been moved/renamed. From the message it would appear that the (one of the two being handled) database is being renamed whilst it is open.
In the log many of the entries are similar so it looks like two databases are being managed i.e. it is the stage of creating the database from the asset.
This may well be an issue with the createFromAsset handling which I understand to not necessarily be rock solid. e.g. at present there are issues with the prePackagedDatabaseCallback.
As such by using createFromAsset that you can do nothing other than raise an issue.
I would suggest circumventing the issue and pre-copying the asset yourself before passing control to Room.
to undertake the copy you do not need to open the database as a database just as a file.
The other alternative, could be to see if exclusively using WAL mode, overcomes the issue. As you are disabling WAL mode, then I guess that you have no wish to do so (hence why suggested as the last).
this would not only entail not disabling WAL mode but also having the asset set to WAL mode before distribution.
I am using Room with a prepopulated database in the assets folder. For an app update, I would like to alter this database by adding a new column and prepopulating this column with new data.
The database was auto-migrated from version 1 to 2 (a table was added). From version 2 to 3, I would now like to apply abovementioned changes by providing a different 'database.db' file in the assets folder and allowing for destructive migration.
#Database(entities = [Object1::class, Object2::class], version = 3, autoMigrations = [
AutoMigration (from = 1, to = 2)], exportSchema = true)
abstract class AppDatabase : RoomDatabase() {
abstract fun dao(): Dao
companion object {
private const val DB_NAME = "database.db"
#Volatile
private var instance: AppDatabase? = null
fun getInstance(context: Context): AppDatabase {
return instance ?: synchronized(this) {
instance ?: buildDatabase(context).also { instance = it }
}
}
private fun buildDatabase(context: Context): AppDatabase {
return Room.databaseBuilder(
context,
AppDatabase::class.java, "AppDB.db")
.fallbackToDestructiveMigration()
.createFromAsset(DB_NAME)
.build()
}
}
}
The problem is that I still get the following exception:
java.lang.IllegalStateException: A migration from 1 to 3 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods.
I am unsure why this would still happen. I thought it was either providing a migration script or allowing for destructive migration that makes the migration work.
Added Comment:-
I have tried an implemented migration, but the same exception as above happened again. When I try starting over with versionCode 1, I am getting "java.lang.IllegalStateException: Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number. You can simply fix this by increasing the version number." I have also changed the database name and added android:allowBackup="false" in the manifest.
Any ideas?
I had problems using fallbackToDestructiveMigration and createFromAsset together. I would like to share my experience because it took me hours to find it. When you provide an asset db, you have to update the user version pragma of the default database file that you are providing with createFromAsset. If not, you always lose the data that you insert while the app is working.
I finally figured out what the problem was, it had nothing to do with the versioning or anything else related to room or the asset db file.
It was dependency injection.
I provided my database to Dagger in a DatabaseModule class as follows:
private const val DB_NAME = "database.db"
#InstallIn(SingletonComponent::class)
#Module
class DatabaseModule {
#Provides
fun provideDao(appDatabase: AppDatabase): Dao {
return appDatabase.dao()
}
#Provides
#Singleton
fun provideAppDatabase(#ApplicationContext appContext: Context): AppDatabase {
return Room.databaseBuilder(
appContext,
AppDatabase::class.java, "AppDB.db")
.createFromAsset(DB_NAME)
.build()
}
}
It was missing the fallBackToDestructiveMigration() call, so this messed up Room's internal onUpgrade call in RoomOpenHelper.java.
To fix it, I made my buildDatabase call in AppDatabase public and used it to provide the database to Dagger in the DatabaseModule class.
Digging through the room documentation doesn't turn much up, my hunch is that it has to do with the fact that you are using Automigrations instead of implemented migrations. Have you tried changing that Automigration from 1->2 to an implemented migration?
Also, since you are manually replacing it with a new database that has prepopulated data my solution would be to just get rid of the old migrations, change the name of the DB slightly and start over from version 1. There's no reason to maintain the old migrations if anyone going from older versions to the current version are having their DB deleted.
After extensive methodical testing, the only way that I can replicate your (1-3 required) failure is by excluding fallbackToDestructiveMigation. In which case the exception happens if the migration is from 1 to 3 or the migration is 3 to 1 (i.e. Asset Version at 3 but Room version at 1)
as per the spreadsheet screenshot below
1-3 exception when AssetDB Version =3 Database Version = 1 Room Version = 3
also 3-1 exception when AssetDB Version =3 Database Version = -1 Room Version = 1
-1 version means file does not exist (i.e. initial install)
I suspect that you have somehow inadvertently introduced one of the above two scanrios. What I haven't tested is alternative Room library versions. The above was tested with 2.4.0-alpha04 as per :-
implementation 'androidx.core:core-ktx:1.6.0'
implementation 'androidx.appcompat:appcompat:1.3.1'
implementation 'com.google.android.material:material:1.4.0'
implementation 'androidx.constraintlayout:constraintlayout:2.1.0'
implementation 'androidx.room:room-ktx:2.4.0-alpha04'
implementation 'androidx.room:room-runtime:2.4.0-alpha04'
testImplementation 'junit:junit:4.+'
androidTestImplementation 'androidx.test.ext:junit:1.1.3'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
kapt 'androidx.room:room-compiler:2.4.0-alpha0
For the testing, I had two copies of the asset file, one at version 1 the other at version 2 (v1dbbase.db and v3dbbase.db), the data in a common column indicating the if the data was for version3. The actual asset file used was deleted before a test and the appropriate version copied and pasted to database.db
I had the two entities Object1 and Object2 and could comment in or out an extra column in either. e.g.:-
/*TESTING INCLUDE FOR V2+ >>>>>*///, #ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
- as above it is excluded
/*TESTING INCLUDE FOR V2+ >>>>>*/, #ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
- with the two //'s before the comma now included
both the extra columns commented out = Version 1
Object1's extra column included = Version 3
Object1's and Object2's extra column included = Version 3
Object2's extra column included but not Object1's was not considered.
A few constants were added to cater for logging.
Additionally to cater for logging a callback function was added (.addCallback) and onOpen, onCreate and onDestructiveMigration were all overridden to log the Room Version and Database Version.
To further enhance the logging, two functions were added, to get the version from the sqlite database header. One for the asset file, the other for the database. The functions being called/invoked BEFORE the database build.
To run a test it meant:-
Ensuring that the device had the App at the appropriate level.
Deleting the database.db asset
Copying and pasting the appropriate asset file as database.db (from either v1dbbase.db or v3dbbase.db)
Amending the Object1 class to include/exclude the extra column (as explained above)
Amending the Object2 class to include/exclude the extra columns (as explained above)
Amended the Room Version to the appropriate level.
The code used for testing:-
Object1
#Entity(tableName = TABLE_NAME)
data class Object1(
#PrimaryKey
#ColumnInfo(name = COL_ID)
val object1_id: Long,
#ColumnInfo(name = COL_NAME)
val object1_name: String
/*TESTING INCLUDE FOR V2+ >>>>>*///, #ColumnInfo(name = COL_EXTRAV2, defaultValue = "x") val object1_extra: String
) {
companion object {
const val TABLE_NAME = "object1"
const val COL_ID = TABLE_NAME + "_object1_id"
const val COL_NAME = TABLE_NAME + "_object1_name"
const val COL_EXTRAV2 = TABLE_NAME + "_object1_extrav2"
}
}
Object2
#Entity(tableName = TABLE_NAME)
data class Object2(
#PrimaryKey
#ColumnInfo(name = COL_ID)
val object2_id: Long,
#ColumnInfo(name = COL_NAME)
val object2_name: String
/*TESTING INCLUDE FOR V3>>>>>*///, #ColumnInfo(name = COL_EXTRAV3, defaultValue = "x") val object3_extrav3: String
) {
companion object {
const val TABLE_NAME = "object2"
const val COL_ID = TABLE_NAME + "_object2_id"
const val COL_NAME = TABLE_NAME + "_object2_name"
const val COL_EXTRAV3 = TABLE_NAME + "_object2_extrav3"
}
}
Dao
#Dao
abstract class Dao {
#Insert
abstract fun insert(object1: Object1): Long
#Insert
abstract fun insert(object2: Object2): Long
#Query("SELECT * FROM ${Object1.TABLE_NAME}")
abstract fun getAllFromObject1(): List<Object1>
#Query("SELECT * FROM ${Object2.TABLE_NAME}")
abstract fun getAllFromObject2(): List<Object2>
}
AppDatabase
#Database(
entities = [Object1::class, Object2::class],
version = AppDatabase.DBVERSION,
autoMigrations = [AutoMigration (from = 1, to = 2)],
exportSchema = true
)
abstract class AppDatabase : RoomDatabase() {
abstract fun dao(): Dao
companion object {
private const val DB_NAME = "database.db"
private const val DB_FILENAME = "AppDB.db" //<<<<< ADDED for getting header
const val TAG = "DBINFO" //<<<< ADDED for logging
const val DBVERSION = 1 //<<<<<ADDED for logging
#Volatile
private var instance: AppDatabase? = null
fun getInstance(context: Context): AppDatabase {
return instance ?: synchronized(this) {
//ADDED>>>>> to get database version from dbfile and assets before building the database
Log.d(TAG,
"AssetDB Version =${getAssetDBVersion(context, DB_NAME)} " +
"Database Version = ${getDBVersion(context, DB_FILENAME)} " +
"Room Version = ${DBVERSION}")
instance ?: buildDatabase(context).also { instance = it }
}
}
private fun buildDatabase(context: Context): AppDatabase {
return Room.databaseBuilder(
context,
AppDatabase::class.java, DB_FILENAME)
.fallbackToDestructiveMigration()
.createFromAsset(DB_NAME)
.allowMainThreadQueries()
.addCallback(rdc)
.build()
}
/* Call Backs for discovery */
object rdc: RoomDatabase.Callback(){
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
Log.d(TAG,"onCreate called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
}
override fun onOpen(db: SupportSQLiteDatabase) {
super.onOpen(db)
Log.d(TAG,"onOpen called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
}
override fun onDestructiveMigration(db: SupportSQLiteDatabase) {
super.onDestructiveMigration(db)
Log.d(TAG,"onDestructiveMigration called. DB Version = ${db.version}, Room Version is ${DBVERSION}")
}
}
fun getAssetDBVersion(context: Context, assetFilePath: String): Int {
var assetFileHeader = ByteArray(100)
try {
var assetFileStream = context.assets.open(assetFilePath)
assetFileStream.read(assetFileHeader,0,100)
assetFileStream.close()
} catch (e: IOException) {
return -2 // Indicates file not found (no asset)
}
return ByteBuffer.wrap(assetFileHeader,60,4).getInt()
}
fun getDBVersion(context: Context, dbFileName: String): Int {
var SQLiteHeader = ByteArray(100)
val dbFile = context.getDatabasePath(dbFileName)
if(dbFile.exists()) {
var inputStream = dbFile.inputStream()
inputStream.read(SQLiteHeader, 0, 100)
inputStream.close()
return ByteBuffer.wrap(SQLiteHeader, 60, 4).getInt()
} else {
return -1 // Indicates no database file (e.g. new install)
}
}
}
}
you may wish to consider including the logging above, it could very easily detect issues with the version(s) being used.
MainActivity
class MainActivity : AppCompatActivity() {
lateinit var db: AppDatabase
lateinit var dao: Dao
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
db = AppDatabase.getInstance(this)
dao = db.dao()
for(o1: Object1 in dao.getAllFromObject1()) {
logObject1(o1)
}
for(o2: Object2 in dao.getAllFromObject2()) {
logObject2(o2)
}
}
fun logObject1(object1: Object1) {
Log.d(TAG,"ID is ${object1.object1_id}, Name is ${object1.object1_name}")
}
fun logObject2(object2: Object2) {
Log.d(TAG,"ID is ${object2.object2_id}, Name is ${object2.object2_name}")
}
companion object {
const val TAG = AppDatabase.TAG
}
}
In addition to utilising the above code and ensuring that the 6 tasks were undertaken I also kept a spreadsheet of the versions and the results e.g. :-
Previous answer (not the case after testing)
I believe that your issue may be with the pre-populated database, in that it's version number (user_version) hasn't been changed to 3.
you can change the version using the SQL (from an SQlite tool ) PRAGMA user_version = 3;
The documentation says :-
Here is what happens in this situation:
Because the database defined in your app is on version 3 and the database instance already installed on the device is on version 2, a migration is necessary.
Because there is no implemented migration plan from version 2 to version 3, the migration is a fallback migration.
Because the fallbackToDestructiveMigration() builder method is called, the fallback migration is destructive. Room drops the database instance that's installed on the device.
Because there is a prepackaged database file that is on version 3, Room recreates the database and populates it using the contents of the prepackaged database file.
If, on the other hand, you prepackaged database file were on version 2, then Room would note that it does not match the target version and would not use it as part of the fallback migration.
By note perhaps by the way of an exception?
I am writing an application in Kotlin which needs to use data seeded into the database on app install/first start. I am wanting to use Room for database access and cooroutines for async execution.
I see lots of examples on how to use Room and lots of examples on how to use corountines. However i do not see any examples of using both together.
I need to create a database on application install and seed data and then immediately read from that database to have a cached map of values which i can use in the application. I do not understand how to do the create/seed/read together and not on the main thread.
Later in the application i need to empty the map of values and query the database again to repopulate the maps before updating the UI from the new values in the map.
Background information on my application
My application will show a set of images to the user based on two options the user has selected. These two options are stored in the preference store as a number which can be 1 or 0.
Therefore if the user has selected "option1=0" and "option2=1" and my database contained data such as below
id option1 option2 letter image
------------------------------------------------
1 0 0 A image_a_0_0.png
2 0 0 B image_b_0_0.png
3 0 0 C image_c_0_0.png
4 0 0 D image_d_0_0.png
5 1 0 A image_a_1_0.png
6 1 0 B image_b_1_0.png
7 1 0 C image_c_1_0.png
8 1 0 D image_d_1_0.png
9 0 1 A image_a_0_1.png
10 0 1 B image_b_0_1.png
11 0 1 C image_c_0_1.png
12 0 1 D image_d_0_1.png
13 1 1 A image_a_1_1.png
14 1 1 B image_b_1_1.png
15 1 1 C image_c_1_1.png
16 1 1 D image_d_1_1.png
then the following images will be shown to the user:
image_a_0_1.png
image_b_0_1.png
image_c_0_1.png
image_d_0_1.png
My requirements
In my Room database code I have a singleton for my DB instance, and if the DB does not exist i create it. I use a callback to know when the Database is created. In this callback i use a corountine to call the code to seed data into the database.
My Room's Database code
#Database(....)
abstract class AppDatabase : RoomDatabase() {
abstract fun myDao(): MyDao
companion object {
private const val DATABASE_NAME = "my_databse"
// Singleton prevents multiple instances of database opening at the same time.
#Volatile
private var sINSTANCE: AppDatabase? = null
fun getInstance(context: Context): AppDatabase {
val tempInstance = sINSTANCE
if (tempInstance != null) {
return tempInstance
}
synchronized(this) {
val instance = Room.databaseBuilder(
context.applicationContext,
AppDatabase::class.java,
DATABASE_NAME
)
.addCallback(seedDatabaseCallback(context))
.build()
sINSTANCE = instance
return instance
}
}
/**
* Callback which is called when the Database is first created.
*/
private fun seedDatabaseCallback(context: Context): Callback {
return object : Callback() {
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
seedData(context)
}
}
}
suspend fun seedData(context:Context){
val db = AppDatabase.getInstance(context)
var myDao = db.myDao()
myDao.insert(..........)
.....
}
}
}
I have a "DataProvider.kt" file which is responsible for reading values from the preference store and the database (using the DAO) and updating the maps used in the rest of the application. This could be called any time within my app when the user changes any of the two options so that the map always contains the correct images applicable to the users current settings
suspend fun updateMaps(context: Context) {
val mSharedPref = context.getSharedPreferences(OPTIONS_PREFERENCES_KEY, Context.MODE_PRIVATE)
val option1 = mSharedPref.getLong("option1_PREF", 0)
val option2 = mSharedPref.getLong("option2_PREF", 0)
updateMaps(context, option1, option2)
}
private suspend fun updateMaps(context: Context, option1 : Int, option2 : Int) {
withContext(Dispatchers.IO) {
var db = AppDatabase.getInstance(context)
var myValues = db.myDao().getValues(option1, option2)
for (value in myValues){
myMap[value .letter] = value.image
}
}
}
My Issue
In my main activity's onCreate method i am trying to check the database exists (creating and seeding data if need be) before setting up the maps. If the maps have values i set views visible on my main activity.
I do not entirely understand how to put Room and Corountines together. I am finding on app install, in my main activity, it calls updateMaps and the DB is created and data seeded, but it never sets my view to visible as the maps do not have values when it comes to the check in main activity.
I "think" its because the db creation is seeded on one thread, then the db values are being red and maps updated on another thread so they are not dependant on each other.
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
var myView = findViewById<CardView>(R.id.myView)
....
....
GlobalScope.launch {
// Set up the maps, this will create the database and seed the factory
// data the first time this application is run
DataProvider.updateMaps(this#DashboardActivity)
if (DataProvider.myMap.isNotEmpty()) {
// ISSUE DOES NOT COME IN HERE
myView.visibility = View.VISIBLE
}
}
....
}
I have seen comments about async() and await() but I am not entirely clear when you would use those rather than the code i have in this question.
there is an example with prepopulating the database using coroutines in the Google's Room with a View course, only it does repopulate on every open
https://codelabs.developers.google.com/codelabs/android-room-with-a-view-kotlin/index.html?index=..%2F..index#11
EDIT - after more detail read, I think it's better if you look into the sample Sunflower app
https://github.com/android/sunflower
it uses seed worker to populate the database based on JSON file
class SeedDatabaseWorker(
context: Context,
workerParams: WorkerParameters
) : CoroutineWorker(context, workerParams) {
override suspend fun doWork(): Result = coroutineScope {
try {
applicationContext.assets.open(PLANT_DATA_FILENAME).use { inputStream ->
JsonReader(inputStream.reader()).use { jsonReader ->
val plantType = object : TypeToken<List<Plant>>() {}.type
val plantList: List<Plant> = Gson().fromJson(jsonReader, plantType)
val database = AppDatabase.getInstance(applicationContext)
database.plantDao().insertAll(plantList)
Result.success()
}
}
} catch (ex: Exception) {
Log.e(TAG, "Error seeding database", ex)
Result.failure()
}
}