I have a migration where I am adding a new column
The migration is defined as
val MIGRATION_20_21: Migration = object : Migration(20, 21) {
override fun migrate(database: SupportSQLiteDatabase) {
database.execSQL("ALTER TABLE tests ADD COLUMN deleted INTEGER NOT NULL DEFAULT 0")
}
}
The field is dined as
#ColumnInfo(name = "deleted") var deleted: Boolean = false,
And I have a test case for this migration which looks like
#Test
#Throws(IOException::class)
fun migrate20to21() {
var db = helper.createDatabase(TEST_DB, 20).apply {
addTestsToDatabase(this, true)
close()
}
db = helper.runMigrationsAndValidate(TEST_DB, 21, true,
MIGRATION_20_21)
var cursor = db.query("SELECT * FROM tests WHERE id = ?", arrayOf("id0"))
MatcherAssert.assertThat(cursor, Is(notNullValue()))
MatcherAssert.assertThat(cursor.moveToFirst(), `is`(true))
MatcherAssert.assertThat(cursor.getColumnIndex("deleted"), `is`(38))
MatcherAssert.assertThat(cursor.getInt(cursor.getColumnIndex("deleted")), `is`(0))
I am getting the error Migration didn't properly handle: tests
I see a difference in expected and found table info
Expected: deleted=Column{name='deleted', type='INTEGER', affinity='3', notNull=true, primaryKeyPosition=0, defaultValue='null'}, local_created_at=Column{name='local_created_at', type='INTEGER', affinity='3', notNull=false, primaryKeyPosition=0, defaultValue='null'}
Found: deleted=Column{name='deleted', type='INTEGER', affinity='3', notNull=true, primaryKeyPosition=0, defaultValue='0'}
Why there is a difference in default value and how can it be null when I have defined it as not null.
Why there is a difference in default value and how can it be null when I have defined it as not null
The reason is that "Expected" is derived from the Entity (at compile time) and then compared to the schema of the actual database that is being/has been opened (what has been "Found"), so you need to change the Entity to also have a default value of 0, using the #ColumnInfo(defaultValue = "0").
Without then the default value will be null i.e. no default value specified.
The #ColumnInfo's defaultValue was introduced in Room 2.2.0, until that version the DEFAULT clause was (I believe) ignored and and expected/found conflict wouldn't arise.
So the Entity should contain
#ColumnInfo(defaultValue = "0")
val deleted: Int = 0
Or instead of Int = 0, Boolean = false
Amendment re update showing:-
#ColumnInfo(name = "deleted") var deleted: Boolean = false,
then you need to use :-
#ColumnInfo(name = "deleted", defaultValue = "0") var deleted: Boolean = false,
Re the comment
thank you. when I say var deleted: Boolean = false, is it not the same as saying defaultValue?
Correct it isn't the same.
Setting a value for var/val sets the value of the field when the value isn't provided when instantiating the object and effectively assigns the object a default value.
Whilst defaultValue = "0" says that the column definition includes the DEFAULT 0 in the table's column definition.
e.g.
`deleted` INTEGER NOT NULL DEFAULT 0
Without you get
`deleted` INTEGER NOT NULL
This is what Room EXPECTED (as it is without defaultValue =) in your case according to the Entity your ALTER column definition that was FOUND differs from the expected (no DEFAULT so Room's interpretation is defaultValue = null).
In your case the defaultValue = "0" is important as 0 (false) will be used as the value for the column for rows that existed before the ALTER, whilst null might result in run time issues.
using var deleted: Boolean = false, is also potentially important as it provides the default value if the value is omitted when instantiating the object.
Perhaps the best/easiest way to get things as Room expects is to make your Entity changes, then compile (e.g. Ctrl + F9) and to then inspect the generated java (easily visible from Android View in Android Studio) to then find the class that is named as per the #Database class suffixed with _Impl and then find the createAllTables method which includes the EXPECTED table definitions.
e.g.
Another Amendment
Regarding the comment
Does the migration uses the json files created by room?
No, UNLESS AutoMigration is used. AutoMigration builds the code for the migration (I believe) at compile time and hence why it needs schemas and requires exportSchema = true and the underlying gradle schema location directive.
Migrations actually take place at run time and do not need (nor even have access to the saved schemas as I believe that they are not included in the APK).
My current version is 24 but this field was added at version 21. SO even though I add default = 0, it only recreates 24.json. This test case for 20 to 21 fails as 21.json do not have default value.
I suspect that this could be due to different versions of Room. Prior to 2.2.0 defaultValue was not an option and thus it was ignored when comparing expected and found. Noting that this comparison is undertaken after migration(s) have been performed (I believe ALL migrations).
Related
I am about the migrate my room database from 1 to 2.
In version 1 exportSchema was set to false. I was unaware of the impact at the time.
Therefore no 1.json schema file is available on device running the app so far.
In version 1 there is a class, let's call it Mango as follows:
#Entity(tableName="mango)
data class Mango(
#PrimaryKey(autoGenerate = true) val id: Int =0,
val carbs: Float = 0f
){...}
In version 2 the field carbs should change to carbohydrate. This is how I do it in my RoomDatabase class.
#Database(
entities = [Mango::class], version = 2, exportSchema = true
)
abstract class AppDatabase : RoomDatabase() {
...
fun getDatabase(
context: Context
): AppDatabase {
return INSTANCE ?: synchronized(this) {
val instance = Room.databaseBuilder(
context,
AppDatabase::class.java,
"my_database"
).addMigrations(MIGRATION_1_2).build()
INSTANCE = instance
instance
}
}
...
val MIGRATION_1_2 = object : Migration(1, 2) {
override fun migrate(database: SupportSQLiteDatabase) {
database.execSQL("ALTER TABLE Mango RENAME COLUMN cabrs TO carbohydrate")
}
It works on emulator in android studio. And this is how I test it.
Uninstall the app
Run version 1
Switch code to version 2 and run version 2
Result: Working
I have created a release version and sent it via google play to test it in a real life stuation, and I get the following error when I update the app with the new version 2:
Fatal Exception: android.database.sqlite.SQLiteException: near "COLUMN": syntax error (code
1): , while compiling: ALTER TABLE Mango RENAME COLUMN carbs TO carbohydrate
#################################################################
Error Code : 1 (SQLITE_ERROR)
Caused By : SQL(query) error or missing database.
(near "COLUMN": syntax error (code 1): , while compiling: ALTER TABLE Mango RENAME COLUMN
carbs TO carbohydrate).
If it is because of the missing 1.json file, then how can I fix this?
Thanks
If it is because of the missing 1.json file, then how can I fix this?
The issue is probably not the schema but is probably that the "real life run" was on an android version that doesn't include release 3.25.0 or greater of SQLite.
Typically unsupported features result in a Syntax Error that can be confusing is it tends to mention where the error was found as it doesn't know about the new syntax.
i.e. only devices with API 30+ (when a jump was made from 3.22.0 to 3.28.0)
as per https://developer.android.com/reference/android/database/sqlite/package-summary
SQLite 3.25.0 release documentation:-
2018-09-15 (3.25.0)
Add support for window functions
Enhancements the ALTER TABLE command:
Add support for renaming columns within a table using ALTER TABLE table RENAME COLUMN oldname TO newname.
-Fix table rename feature so that it also updates references to the renamed table in triggers and views.
....
If you need to target devices less than API 30, then you may not be able to use AutoMigration but will instead have a manual migration that:-
renames the original table
create the new table (copy the SQL from the createAllTables method in the class that is the same name as the #Database annotated class but suffixed with _Impl that can be found in the java(generated) via the Android View (I believe CRTL B can also be used)).
then execute the SQL INSERT INTO <new_table> SELECT * FROM <the_renamed_original_table>
Note that this assumes that the columns are in exactly the same position. It would be safer to use INSERT INTO new_table (<ALL_THE_COLUMNS_NAMES_OF_THE_NEW_TABLE_COMMA_SEPARATED>) SELECT (<THE_RESPECTIVE_COLUMNS_OF_THE_RENAMED_TABLE_COMMA_SEPARATED>)
Note anything enclosed within < and > should be changed accordingly, the enclosed text explains the change(s)
Note the above is in-principle code, it has not been compiled/run or tested so may contain some minor errors.
How to change affinity?
Expected:
serial_number_prefix=Column{name='serial_number_prefix', type='TEXT', affinity='2', notNull=false, primaryKeyPosition=0, defaultValue='null'},
Actual:
serial_number_prefix=Column{name='serial_number_prefix', type='STRING', affinity='1', notNull=false, primaryKeyPosition=0, defaultValue='null'},
The only change is affinity = 1, but it should be affinity = 2.
How to change it?
This is my migration function
database.execSQL("ALTER TABLE values ADD COLUMN serial_number_prefix STRING DEFAULT null")
in data class
#SerializedName("serial_number_prefix")
#ColumnInfo(name = "serial_number_prefix")
val serialNumberPrefix: String?,
Room, is very specific about what column types can be and thus what Room expects.
Unlike SQLite which is very flexible with column Types, column types in Room must be one of TEXT, INTEGER, REAL or BLOB.
Room determines the TYPE used in the SQL
In SQLite STRING actually equates to the catch all type NUMERIC, which Room does not allow and thus it has an affinity of 1 which is UNDEFINED.
Without going into too much detail the easiest way to conform to Room's expectations is to
create the #Entity annotated classes
Create the #Database annotated class with the #Entity annotated classes specified in the entities parameter of the annotation.
Compile/Build the project (e.g. CTRL + F9).
In the Android View locate the generated Java.
Find the class that is the same name as the #Database annotated class BUT suffixed with _Impl.
Within the class find the createAllTables method.
Copy and paste the SQL for the table(s) and base the ALTER statement on the column definitions therein. This will be the exact definition that Room expects.
How to change it?
In your case you would use:-
ALTER TABLE values ADD COLUMN serial_number_prefix TEXT DEFAULT null
i.e. STRING has been changed to TEXT
Regarding #ColumnInfo(typeAffinity=2)
You could think that overriding the affinity using the typeAffinity parameter of the #ColumnInfo annotation would resolve the issue e.g. using #ColumnInfo(typeAffinity = 1).
However, 1 equates to UNDEFINED, and thus Room then uses the field's type (as per the quote below) to determine the type. Thus as the field types is a Stringm it uses TEXT when it builds the expected schema (and uses TEXT in the SQL to create the table). As TEXT equates to affinity=2, the same issue occurs.
Again use the column type from generated java in the ALTER SQL.
The type affinity for the column, which will be used when constructing the database.
If it is not specified, the value defaults to UNDEFINED and Room resolves it based on the field's type and available TypeConverters.
https://developer.android.com/reference/androidx/room/ColumnInfo#typeAffinity()
I have an issue where autoGenerate is not working on an inherited field in my Entity class.
In my project I have created a base class which has an id field already added to it. This base class is then used by every Entity so I can work with generics and such. Everything seems to work perfectly until I add the autoGenerate to the id field of an Entity. (FYI: this was working in version 2.2.6, but in 2.3.0 this breaks and results in this issue.)
The BaseEntity class
interface BaseEntity {
val id: Any
}
The specific Entity class
#Entity(tableName = DBConstants.FOOD_ENTRY_TABLE_NAME)
data class FoodEntry(
#PrimaryKey(autoGenerate = true)
override val id: Int = 0,
var amount: Float,
var date: Long,
var meal: Meal
) : BaseEntity
If I do something like this it works (but it's not what I need)
#Entity(tableName = DBConstants.FOOD_ENTRY_TABLE_NAME)
data class FoodEntry(
override val id: Int = 0,
#PrimaryKey(autoGenerate = true)
var someOtherId: Int = 0,
var amount: Float,
var date: Long,
var meal: Meal
) : BaseEntity
As far as I can see this is only a problem when you wish to autoGenerate an inherited field.
Anybody else have seen this issue before?
As far as I can see this is only a problem when you wish to autoGenerate an inherited field.
The same behaviour happens if you just have #PrimaryKey or if you define the id column as a primary key.
The issue is that Room interprets the Int as a Type of int (in the underlying Java code (perhaps a bug, you may wish to raise an issue )). Room if it considers the Type as int as opposed to Int treats the 0 default value differently.
In case of Type Int with a 0 then Room doesn't attempt to specify a value if it is a for a primary key column and thus allows SQLite to assign a value.
e.g. SQL along the lines of INSERT INTO the_table (amount,date,meal) VALUES(the_amount, the_date, the_meal);
If the Type is int then the value is always specified so the 0's will result in UNIQUE constrain conflicts.
e.g. SQL along the lines of INSERT INTO the_table VALUES(0,the_amount, the_date, the_meal);
as the column list is omitted ALL columns need a value
Possible Fix
If you instead used
interface BaseEntity {
val id: Any?
}
with :-
#PrimaryKey
override val id: Int? = null,
generated java has :-
_db.execSQL("CREATE TABLE IF NOT EXISTS food (id INTEGER, amount REAL NOT NULL, date INTEGER NOT NULL, meal TEXT NOT NULL, PRIMARY KEY(id))");
or :-
#PrimaryKey(autoGenerate = true)
override val id: Int? = null,
generated java has :-
_db.execSQL("CREATE TABLE IF NOT EXISTS food (id INTEGER PRIMARY KEY AUTOINCREMENT, amount REAL NOT NULL, date INTEGER NOT NULL, meal TEXT NOT NULL)");
then no attempt is made to insert the id value and SQLite assigns the value.
in SQLite terms
Alternative Fix
If you use the original BaseEntity then you could insert using a Query where you exclude the id column and thus allow it to be generated.
e.g. you could have :-
#Query("INSERT INTO food (amount,date,meal) VALUES(:amount,:date,:meal)")
fun insertFE(amount: Float, date: Long, meal: String): Long
but that doesn't insert using a FoodEntry object so you could then have (dependant upon the above) :-
fun insertFE(FoodEntry: FoodEntry): Long {
return insertFE(FoodEntry.amount,FoodEntry.date,your_type_converter(FoodEntry.meal))
}
obviously your_type_converter would be changed accordingly
About autoGenerate (AUTOINREMENT in SQLite)
autogenerate = true does not noticeably effect the internally generated value until you reach the maximum allowed value (9223372036854775807). If autogenerate = true is coded and the last value was the 9223372036854775807 the next insert will result in a an SQLITE_FULL error and an exception.
If autoGenerate = false is coded or autoGenerate is not specified then then SQLite will, if 9223372036854775807 has been assigned (and the row still exists), attempt to allocate an unassigned value between 1 and 9223372036854775807 which would likely succeed as it's basically impossible to have 9223372036854775807 rows.
note that if any row has been assigned a negative value then the range is extended.
of course specifying Int or int imposes a much lower restriction outside of SQLite. Really id's should be Long or long.
autoGenerate= true means that the AUTOINCREMENT keyword is included in the column definition. This is a constraint that says that when the value is determined by SQLite that the value must be greater than any value that either exists or has been used.
To ascertain this AUTOINCREMENT uses an internal table namely sqlite_sequence to store the last assigned/determined value. Having the additional table and having to access and maintain the table has overheads.
SQLite https://sqlite.org/autoinc.html has as it's first sentence :- The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and disk I/O overhead and should be avoided if not strictly needed. It is usually not needed.
I know it's not the best practice to store an image in DB directly and we should store a path instead. But in my case this is a must.
I am able to store list of images perfectly fine defined as:
#ColumnInfo(name = "picture")
var picture: ByteArray? = null
I came across solutions that suggests using (typeAffinity = ColumnInfo.BLOB). So I changed my column to:
#ColumnInfo(name = "picture", typeAffinity = ColumnInfo.BLOB)
var picture: ByteArray? = null
I haven't noticed anything significant in performance. I wonder what are the possible advantages of using typeAffinity or disadvantages of not using it?
It maybe worth mentioning my images are always under 1 megabytes.
There is no real advantage/disadvantage, certainly not at run time perhaps marginally at compile time.
That is all that using typeAffinity=? does is override the typeAffinity being determined by the type of the field/column of the variable.
As you have var picture: ByteArray this would be resolved to a column type of BLOB anyway.
If you wished you could compile with both and see the resultant SQL used by looking at the generated java.
Perhaps consider the following Entity that uses both:-
#Entity(tableName = "user")
data class UserEntity(
#PrimaryKey
val userid: Long = 0,
var picture1: ByteArray? = null,
#ColumnInfo(typeAffinity = ColumnInfo.BLOB)
var pitcure2: ByteArray? = null
)
In the generated Java (use Android view as highlihted) then in the #Database class (UserDatabase in the example) suffixed by _Impl (so UserDatabase_Impl in the example) the following is a screen shot of the generated Java :-
Android highlighted indicates where to select the Android view.
The highlight in the code explorer shows the respective code (UserDatabase_Impl) in the expanded java (generated) directory
The createAllTables method is the method used to create the table(s)
room_master_table is a room specific table used for verification of an existing table with the schema to detect if there are differences.
The code (SQL) generated for the creation of the table is :-
_db.execSQL("CREATE TABLE IF NOT EXISTS `user` (`userid` INTEGER NOT NULL, `picture1` BLOB, `pitcure2` BLOB, PRIMARY KEY(`userid`))");
i.e. the definition of the columns picture1 and picture2 are identical bar the column names.
NOTE please heed the WARNING in regards to not changing the generated code.
I am scratching my head with this error. I couldn't find any answer so far. I have old database which I am migrating to Persistence Room library. However whenever I do migration, I am getting following error,
java.lang.IllegalStateException: Migration didn't properly handle.
Code I am using is as follows:
#Entity(tableName = ROUTE_TABLE)
public class RouteData {
static final String ROUTE_TABLE = "name";
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "id")
private int id;
#ColumnInfo(name = "col1")
private String col1;
//Other columns with exactly same as 'col1' just different names
//Getters and Setters
}
For migration,
Room.databaseBuilder(context.getApplicationContext(), MyData.class, DATABASE_NAME)
.addMigrations(MIGRATION_LATEST)
.fallbackToDestructiveMigration()
.build();
private static final Migration MIGRATION_LATEST = new Migration(9, 10) {
#Override
public void migrate(SupportSQLiteDatabase db) {
//Log.i("Tag" , "Migration Started");
//Migration logic
//Log.i("Tag" , "Migration Ended");
}
};
When I run program. I get "Migration Ended" log in my LogCat but when I am trying access database again, it is giving me following error.
Caused by: java.lang.IllegalStateException: Migration didn't properly handle xxx.
Expected:
TableInfo{name='name', columns={col1=Column{name='col1', type='TEXT', notNull=false, primaryKeyPosition=0}, ....}, foreignKeys=[]}
Found:
TableInfo{name='name', columns={col1=Column{name='col1', type='INTEGER', notNull=false, primaryKeyPosition=0}, ....}, foreignKeys=[]}
I have no idea where where this "INTEGER" is coming. I tried uninstalling, invalidating cache and any other closely related solution. Any idea what is going on? It works perfectly fine if you freshly install app. Error is coming up only after migration. According to my log cat, all migration steps seems to be completed but it is still showing migration not handled properly.
Thank you all for trying to help me, however I have finally found the answer after days of frustrating debugging. In my app manifest auto backup was on,
android:allowBackup="true"
So while trying out various databases, google was actually backing up my any newly created databases and their structure and restoring them automatically when I was reinstalling app. Hence I was getting this wired error. Once I switch of auto backup android:allowBackup="false" and reinstall app, I can test migration properly.
My suggestion for all developers in future encountering such problem, SWITCH OFF auto backup while development. You can switch it on once you have tested your migration.
The main problem in your migration script is bellow:
Caused by: java.lang.IllegalStateException: Migration didn't properly handle xxx.
Expected:
TableInfo{name='name', columns={col1=Column{name='col1', type='TEXT', notNull=false, primaryKeyPosition=0}, ....}, foreignKeys=[]}
Found:
TableInfo{name='name', columns={col1=Column{name='col1', type='INTEGER', notNull=false, primaryKeyPosition=0}, ....}, foreignKeys=[]}
Here, in your Entity Class you have :
#ColumnInfo(name = "col1")
private String col1;
but in your migration query you may be adding col1 as INTEGER
From your error :
Expected :
columns={col1=Column{name='col1', type='TEXT', notNull=false, primaryKeyPosition=0}
Found :
columns={col1=Column{name='col1', type='INTEGER', notNull=false, primaryKeyPosition=0}
Solution: change your migration query like :
ALTER TABLE 'YOUR_TABLE' ADD COLUMN 'col1' TEXT
It will solve your problem.
Thanks :)