Develop Reference

Optimize Room database migrations

In migration (3,4), I delete table TableA and create two new tables TableB and `TableC.
static final Migration MIGRATION_3_4 = new Migration(3, 4) {
#Override
public void migrate(SupportSQLiteDatabase database) {
database.execSQL("DROP TABLE IF EXISTS `TableA`");
database.execSQL("CREATE TABLE IF NOT EXISTS " +
"`TableB` (`id` INTEGER NOT NULL, " +
"PRIMARY KEY(`id`))");
database.execSQL("CREATE TABLE IF NOT EXISTS " +
"`TableB` (`id` INTEGER NOT NULL, " +
"PRIMARY KEY(`id`))");
// bulky method
databaseWriteExecutor.execute(AppDatabase::populateProblem);
}
};
In migration (4, 5), I want to remove TableB and TableC:
static final Migration MIGRATION_4_5 = new Migration(4, 5) {
#Override
public void migrate(SupportSQLiteDatabase database) {
database.execSQL("DROP TABLE IF EXISTS `TableB`");
database.execSQL("DROP TABLE IF EXISTS `TableC`");
}
};
As you can see, migration (4, 5) is the opposite of migration (3, 4). Now suppose a user wants to migrate from 3 to 5, as far as I know Room will run two successive migrations in this case 3 to 4 then 4 to 5, creating two tables and then dropping them is a waste of time, so to optimize this I add a new migration (3, 5):
static final Migration MIGRATION_3_5 = new Migration(3, 5) {
#Override
public void migrate(SupportSQLiteDatabase database) {
database.execSQL("DROP TABLE IF EXISTS `TableA`");
}
};
Question: Can I safely remove migration(3, 4) from the source code? because I think it is no longer used. In this migration I am calling the populateProblem() method which is a very big method, removing the migration will reduce the size of the apk.

Yes, you can drop the intermediate migrations. You only need to keep migrations from every previous version to the latest version. So, if you latest DB version is 5, and assuming that you had versions 1 - 4 in production, you only need the following migrations:
Migration(1, 5)
Migration(2, 5)
Migration(3, 5)
Migration(4, 5)

Android Room migration with updated pre-populated database

I'm having a little bit of a headache with Room and migration with a pre-populated database.
EXPLANATION
I'm currently using Room and a pre-populated database. With the first version (version 1) the database loads fine and everything works correctly.
The problem is that at this point I'm in need to add three new tables to the database with data in them. So I started updating the version 1 database that I had and created all the tables and rows with data in them that I needed.
The first time that I tried, I pushed directly the new .sqlite database into the assets folder and changed the version number to 2, but of course, Room gave the error that it needs to know how to handle migration 1_2 so I added a migration rule
.addMigrations(new Migration(1,2) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
}
...
Thinking that maybe if I tell Room to create these tables it will then connect to the new database in assets and populate the tables accordingly.
But that of course didn't work and by looking at the database inspector it was clear that the tables were present but they were empty.
A SOLUTION I DON'T REALLY LIKE
After tinkering around for a little bit in the end what I found that worked is to have a copy of the updated database, navigate in it (I'm currently using DB Browser for SQLite), get the SQL query for the newly populated rows, and format a database.execSQL statement accordingly to insert the new data into the tables:
.addMigrations(new Migration(1,2) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
database.execSQL("INSERT INTO ...");
database.execSQL("INSERT INTO ...");
database.execSQL("INSERT INTO ...");
database.execSQL("INSERT INTO ...");
}
...
I find this an "acceptable" solution for cases in which we're working on rows that contain small data but in my case, I'm handling rows with very long strings and this creates a series of inconveniences:
the SQL statements that are extracted from the database data need to be well formatted: ' symbols need to be handled as well as " that could be present in the long strings as well as line breaks;
consistency between the database and the insert statements for the rows needs to be kept;
QUESTION
Mind that fallbackToDestructiveMigration() is not an acceptable option since the database in Version 1 has user-created data in it and it needs to be kept between migrations.
So, is there a solution that allows me to directly push the new .sqlite database into assets without writing tons and tons of INSERT and CREATE TABLE statements and let Room handle the new data within it automatically and also while keeping the old tables data?
Thank you for your time!

Perhaps consider
Place the new database into the assets folder suitable for a new install of the App so the createFromAsset would copy this version 2 database for a new install.
In the migration copy the asset to a the database folder with a different database name.
in the migration create the new tables.
still in the migration, for each new table, extract all of the data from the differently named new database then use the Cursor to insert the data into the existing database.
still in the migration, close the differently name database and delete the file.
Here's the migration code for something along those lines (no schema change, just new pre-populated data) and it's Kotlin not Java from a recent answer:-
val migration1_2 = object: Migration(1,2) {
val assetFileName = "appdatabase.db"
val tempDBName = "temp_" + assetFileName
val bufferSize = 1024 * 4
#SuppressLint("Range")
override fun migrate(database: SupportSQLiteDatabase) {
val asset = contextPassed?.assets?.open(assetFileName) /* Get the asset as an InputStream */
val tempDBPath = contextPassed?.getDatabasePath(tempDBName) /* Deduce the file name to copy the database to */
val os = tempDBPath?.outputStream() /* and get an OutputStream for the new version database */
/* Copy the asset to the respective file (OutputStream) */
val buffer = ByteArray(bufferSize)
while (asset!!.read(buffer,0,bufferSize) > 0) {
os!!.write(buffer)
}
/* Flush and close the newly created database file */
os!!.flush()
os.close()
/* Close the asset inputStream */
asset.close()
/* Open the new database */
val version2db = SQLiteDatabase.openDatabase(tempDBPath.path,null,SQLiteDatabase.OPEN_READONLY)
/* Grab all of the supplied rows */
val v2csr = version2db.rawQuery("SELECT * FROM user WHERE userId < ${User.USER_DEMARCATION}",null)
/* Insert into the actual database ignoring duplicates (by userId) */
while (v2csr.moveToNext()) {
database.execSQL("INSERT OR IGNORE INTO user VALUES(${v2csr.getLong(v2csr.getColumnIndex("userId"))},'${v2csr.getString(v2csr.getColumnIndex("userName"))}')",)
}
/* close cursor and the newly created database */
v2csr.close()
version2db.close()
tempDBPath.delete() /* Delete the temporary database file */
}
}
Note when testing the above code. I initially tried ATTACH'ing the new (temp) database. This worked and copied the data BUT either the ATTACH or DETACH (or both) prematurely ended the transaction that the migration runs in, resulting in Room failing to then open the database and a resultant exception.
If this wasn't so then with the new database attached a simple INSERT INTO main.the_table SELECT * FROM the_attached_schema_name.the_table; could have been used instead of using the cursor as the go-between.
without writing tons and tons of INSERT and CREATE TABLE statements
INSERT dealt with above.
The CREATE SQL could, in a similar way, be extracted from the new asset database, by using:-
`SELECT name,sql FROM sqlite_master WHERE type = 'table' AND name in (a_csv_of_the_table_names (enclosed in single quotes))`
e.g. SELECT name,sql FROM sqlite_master WHERE type = 'table' AND name IN ('viewLog','message');;
results in (for an arbitrary database used to demonstrate) :-
name is the name of the table and sql then sql that was used to create the tables.
alternately the SQL to create the tables can be found after compiling in the generated java (visible from the Android View) in the class that has the same name as the class annotated with #Database but suffixed with _Impl. There will be a method called createAlltables which has the SQL to create all the tables (and other items) e.g. (again just an arbitrary example) :-
note the red stricken-through lines are for the room_master table, ROOM creates this and it is not required in the asset (it's what room uses to check to see if the schema has been changed)
Working Example
Version 1 (preparing for the migration to Version 2)
The following is a working example. Under Version 1 a single table named original (entity OriginalEnity) with data (5 rows) via a pre-populated database is used, a row is then added to reflect user suplied/input dat. When the App runs the contents of the table are extracted and written to the log :-
D/DBINFOoriginal: Name is name1 ID is 1 - DB Version is 1
D/DBINFOoriginal: Name is name2 ID is 2 - DB Version is 1
D/DBINFOoriginal: Name is name3 ID is 3 - DB Version is 1
D/DBINFOoriginal: Name is name4 ID is 4 - DB Version is 1
D/DBINFOoriginal: Name is name5 ID is 5 - DB Version is 1
D/DBINFOoriginal: Name is App User Data ID is 6 - DB Version is 1
Database Inspector showing :-
Version 2
The 3 new Entities/Tables added (newEntity1,2 and 3 table names new1, new2 and new3 respectively) same basic structure.
After creating the Entities and compiling the SQL, as per the createAlltables method in the java generated was extracted from the TheDatabase_Impl class (including the 3 additional indexes) :-
This SQL was then used in the SQLite tool to create the new tables and populate them with some data :-
/* FOR VERSION 2 */
/* Create statments copied from TheDatabase_Impl */
DROP TABLE IF EXISTS new1;
DROP TABLE IF EXISTS new2;
DROP TABLE IF EXISTS new3;
CREATE TABLE IF NOT EXISTS `new1` (`new1_id` INTEGER, `new1_name` TEXT, PRIMARY KEY(`new1_id`));
CREATE INDEX IF NOT EXISTS `index_new1_new1_name` ON `new1` (`new1_name`);
CREATE TABLE IF NOT EXISTS `new2` (`new2_id` INTEGER, `new2_name` TEXT, PRIMARY KEY(`new2_id`));
CREATE INDEX IF NOT EXISTS `index_new2_new2_name` ON `new2` (`new2_name`);
CREATE TABLE IF NOT EXISTS `new3` (`new3_id` INTEGER, `new3_name` TEXT, PRIMARY KEY(`new3_id`));
CREATE INDEX IF NOT EXISTS `index_new3_new3_name` ON `new3` (`new3_name`);
INSERT OR IGNORE INTO new1 (new1_name) VALUES ('new1_name1'),('new1_name2');
INSERT OR IGNORE INTO new2 (new2_name) VALUES ('new2_name1'),('new2_name2');
INSERT OR IGNORE INTO new3 (new3_name) VALUES ('new3_name1'),('new3_name2');
The database saved and copied into the assets folder (original renamed) :-
Then the Migration code (full database helper), which :-
is driven simply by a String[] of the table names
copies the asset (new database) an opens it via the SQLite API
creates the tables, indexes and triggers according to the asset (must match the schema generated by room (hence copying sql from generated java previously))
it does this by extracting the respective SQL from the sqlite_master table
populates the newly created Room tables by extracting the data from the asset database into a Cursor and then inserting into the Room database (not the most efficient way BUT Room runs the Migration in a transaction)
is:-
#Database(entities = {
OriginalEntity.class, /* on it's own for V1 */
/* ADDED NEW TABLES FOR V2 */NewEntity1.class,NewEntity2.class,NewEntity3.class
},
version = TheDatabase.DATABASE_VERSION,
exportSchema = false
)
abstract class TheDatabase extends RoomDatabase {
public static final String DATABASE_NAME = "thedatabase.db";
public static final int DATABASE_VERSION = 2; //<<<<<<<<<< changed */
abstract AllDao getAllDao();
private static volatile TheDatabase instance = null;
private static Context currentContext;
public static TheDatabase getInstance(Context context) {
currentContext = context;
if (instance == null) {
instance = Room.databaseBuilder(context, TheDatabase.class, DATABASE_NAME)
.allowMainThreadQueries() /* for convenience run on main thread */
.createFromAsset(DATABASE_NAME)
.addMigrations(migration1_2)
.build();
}
return instance;
}
static Migration migration1_2 = new Migration(1, 2) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
/* Copy the asset into the database folder (with different name) */
File assetDBFile = getNewAssetDatabase(currentContext,DATABASE_NAME);
/* Open the assetdatabase */
SQLiteDatabase assetDB = SQLiteDatabase.openDatabase(assetDBFile.getPath(),null,SQLiteDatabase.OPEN_READWRITE);
/* Build (create and populate) the new ROOM tables and indexes from the asset database */
buildNewTables(
new String[]{
NewEntity1.TABLE_NAME,
NewEntity2.TABLE_NAME,
NewEntity3.TABLE_NAME},
database /* ROOM DATABASE */,
assetDB /* The copied and opened asset database as an SQliteDatabase */
);
/* done with the asset database */
assetDB.close();
assetDBFile.delete();
}
};
private static void buildNewTables(String[] tablesToBuild, SupportSQLiteDatabase actualDB, SQLiteDatabase assetDB) {
StringBuilder args = new StringBuilder();
boolean afterFirst = false;
for (String tableName: tablesToBuild) {
if (afterFirst) {
args.append(",");
}
afterFirst = true;
args.append("'").append(tableName).append("'");
}
/* Get SQL for anything related to the table (table, index, trigger) to the tables and build it */
/* !!!!WARNING!!!! NOT TESTED VIEWS */
/* !!!!WARNING!!!! may not cope with Foreign keys as conflicts could occur */
Cursor csr = assetDB.query(
"sqlite_master",
new String[]{"name","sql", "CASE WHEN type = 'table' THEN 1 WHEN type = 'index' THEN 3 ELSE 2 END AS sort"},
"tbl_name IN (" + args.toString() + ")",
null,
null,null, "sort"
);
while (csr.moveToNext()) {
Log.d("CREATEINFO","executing SQL:- " + csr.getString(csr.getColumnIndex("sql")));
actualDB.execSQL(csr.getString(csr.getColumnIndex("sql")));
}
/* Populate the tables */
/* !!!!WARNING!!!! may not cope with Foreign keys as conflicts could occur */
/* no set order for the tables so a child table may not be loaded before it's parent(s) */
ContentValues cv = new ContentValues();
for (String tableName: tablesToBuild) {
csr = assetDB.query(tableName,null,null,null,null,null,null);
while (csr.moveToNext()) {
cv.clear();
for (String columnName: csr.getColumnNames()) {
cv.put(columnName,csr.getString(csr.getColumnIndex(columnName)));
actualDB.insert(tableName, OnConflictStrategy.IGNORE,cv);
}
}
}
csr.close();
}
private static File getNewAssetDatabase(Context context, String assetDatabaseFileName) {
String tempDBPrefix = "temp_";
int bufferSize = 1024 * 8;
byte[] buffer = new byte[bufferSize];
File assetDatabase = context.getDatabasePath(tempDBPrefix+DATABASE_NAME);
InputStream assetIn;
OutputStream assetOut;
/* Delete the AssetDatabase (temp DB) if it exists */
if (assetDatabase.exists()) {
assetDatabase.delete(); /* should not exist but just in case */
}
/* Just in case the databases folder (data/data/packagename/databases)
doesn't exist create it
This should never be the case as Room DB uses it
*/
if (!assetDatabase.getParentFile().exists()) {
assetDatabase.mkdirs();
}
try {
assetIn = context.getAssets().open(assetDatabaseFileName);
assetOut = new FileOutputStream(assetDatabase);
while(assetIn.read(buffer) > 0) {
assetOut.write(buffer);
}
assetOut.flush();
assetOut.close();
assetIn.close();
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException("Error retrieving Asset Database from asset " + assetDatabaseFileName);
}
return assetDatabase;
}
}
The code in the Activity is :-
public class MainActivity extends AppCompatActivity {
TheDatabase db;
AllDao dao;
private static final String TAG = "DBINFO";
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
/* Original */
db = TheDatabase.getInstance(this);
dao = db.getAllDao();
OriginalEntity newOE = new OriginalEntity();
newOE.name = "App User Data";
dao.insert(newOE);
for(OriginalEntity o: dao.getAll()) {
Log.d(TAG+OriginalEntity.TABLE_NAME,"Name is " + o.name + " ID is " + o.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
/* Added for V2 */
for (NewEntity1 n: dao.getAllNewEntity1s()) {
Log.d(TAG+NewEntity1.TABLE_NAME,"Names is " + n.name + " ID is " + n.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
for (NewEntity2 n: dao.getAllNewEntity2s()) {
Log.d(TAG+NewEntity2.TABLE_NAME,"Names is " + n.name + " ID is " + n.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
for (NewEntity3 n: dao.getAllNewEntity3s()) {
Log.d(TAG+NewEntity3.TABLE_NAME,"Names is " + n.name + " ID is " + n.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
}
}
see Version 1 section and comments in the code for V1 run.
The resultant output to the log for the V2 run (initial) is
:-
2021-10-11 13:02:50.939 D/CREATEINFO: executing SQL:- CREATE TABLE `new1` (`new1_id` INTEGER, `new1_name` TEXT, PRIMARY KEY(`new1_id`))
2021-10-11 13:02:50.941 D/CREATEINFO: executing SQL:- CREATE TABLE `new2` (`new2_id` INTEGER, `new2_name` TEXT, PRIMARY KEY(`new2_id`))
2021-10-11 13:02:50.942 D/CREATEINFO: executing SQL:- CREATE TABLE `new3` (`new3_id` INTEGER, `new3_name` TEXT, PRIMARY KEY(`new3_id`))
2021-10-11 13:02:50.942 D/CREATEINFO: executing SQL:- CREATE INDEX `index_new1_new1_name` ON `new1` (`new1_name`)
2021-10-11 13:02:50.943 D/CREATEINFO: executing SQL:- CREATE INDEX `index_new2_new2_name` ON `new2` (`new2_name`)
2021-10-11 13:02:50.944 D/CREATEINFO: executing SQL:- CREATE INDEX `index_new3_new3_name` ON `new3` (`new3_name`)
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name2 ID is 2 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name3 ID is 3 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name4 ID is 4 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name5 ID is 5 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is App User Data ID is 6 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is App User Data ID is 7 - DB Version is 2
2021-10-11 13:02:51.010 D/DBINFOnew1: Names is new1_name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.010 D/DBINFOnew1: Names is new1_name2 ID is 2 - DB Version is 2
2021-10-11 13:02:51.012 D/DBINFOnew2: Names is new2_name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.012 D/DBINFOnew2: Names is new2_name2 ID is 2 - DB Version is 2
2021-10-11 13:02:51.013 D/DBINFOnew3: Names is new3_name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.013 D/DBINFOnew3: Names is new3_name2 ID is 2 - DB Version is 2
Note that the user data has been retained (1st App User Data ...., 2nd is added when the activity is run).
The Dao (AllDao) is :-
#Dao
abstract class AllDao {
/* Original Version 1 Dao's */
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(OriginalEntity originalEntity);
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long[] insert(OriginalEntity ... originalEntities);
#Query("SELECT * FROM original")
abstract List<OriginalEntity> getAll();
/* New Version 2 Dao's */
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(NewEntity1 newEntity1);
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(NewEntity2 newEntity2);
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(NewEntity3 newEntity3);
#Query("SELECT * FROM " + NewEntity1.TABLE_NAME)
abstract List<NewEntity1> getAllNewEntity1s();
#Query("SELECT * FROM " + NewEntity2.TABLE_NAME)
abstract List<NewEntity2> getAllNewEntity2s();
#Query("SELECT * FROM " + NewEntity3.TABLE_NAME)
abstract List<NewEntity3> getAllNewEntity3s();
}
The Entities are :-
#Entity(tableName = OriginalEntity.TABLE_NAME)
class OriginalEntity {
public static final String TABLE_NAME = "original";
public static final String COL_ID = TABLE_NAME +"_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COL_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
and for V2 :-
#Entity(tableName = NewEntity1.TABLE_NAME)
class NewEntity1 {
public static final String TABLE_NAME = "new1";
public static final String COl_ID = TABLE_NAME + "_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COl_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
and :-
#Entity(tableName = NewEntity2.TABLE_NAME)
class NewEntity2 {
public static final String TABLE_NAME = "new2";
public static final String COl_ID = TABLE_NAME + "_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COl_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
and :-
#Entity(tableName = NewEntity3.TABLE_NAME)
class NewEntity3 {
public static final String TABLE_NAME = "new3";
public static final String COl_ID = TABLE_NAME + "_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COl_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
Finally Test new App install (i.e. no Migration but created from asset)
When run the output to the log is (no user supplied/input data) :-
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name2 ID is 2 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name3 ID is 3 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name4 ID is 4 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name5 ID is 5 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is App User Data ID is 6 - DB Version is 2
2021-10-11 13:42:48.275 D/DBINFOnew1: Names is new1_name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.275 D/DBINFOnew1: Names is new1_name2 ID is 2 - DB Version is 2
2021-10-11 13:42:48.276 D/DBINFOnew2: Names is new2_name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.276 D/DBINFOnew2: Names is new2_name2 ID is 2 - DB Version is 2
2021-10-11 13:42:48.277 D/DBINFOnew3: Names is new3_name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.277 D/DBINFOnew3: Names is new3_name2 ID is 2 - DB Version is 2
Note
Room encloses names of items (tables, columns) in grave accents's, This makes invalid column names valid e.g 1 not enclosed is invalid 1 enclosed is valid. Use of otherwise invalid names may, although I suspect not, cause issues (I haven't tested this aspect). SQLite itself strips the grave accents when storing the name e.g :-
CREATE TABLE IF NOT EXISTS `testit` (`1`);
SELECT * FROM sqlite_master WHERE name = 'testit';
SELECT * FROM testit;
results in :-
i.e. the grave accents are kept when the SQL is stored, hence the generated CREATE's are safe.
and :-
i.e. the grave accents have been stripped and the column is named just 1, which may cause an issue (but likely not) when traversing the columns in the cursor.

Android Room persistence library. Alter Database View in migration

I am using Room persistence library in my android project. I have a database view and I want to add a column to it in new version of my application. what is the proper code for migration?
database.execSQL("????")
PS: I want to change a View, not Table and I tried this:
database.execSQL("ALTER TABLE table_name ADD COLUMN column_name data_type")
I got this error: Cannot add a column to a view (code 1 SQLITE_ERROR)
Update: the old version of my view:
#Data
#DatabaseView("SELECT site.name AS address, group_site.name AS groupName, group_site.member_id AS memberId " +
"FROM site, group_site " +
"INNER JOIN groupsite_join_site " +
"ON site.id = groupsite_join_site.site_id AND group_site.id = groupsite_join_site.group_site_id "
)
public class SiteDetail {
long memberId;
String address;
String groupName;
}
new version:
#Data
#DatabaseView("SELECT site.id as id, site.name AS address, group_site.name AS groupName, group_site.member_id AS memberId " +
"FROM site, group_site " +
"INNER JOIN groupsite_join_site " +
"ON site.id = groupsite_join_site.site_id AND group_site.id = groupsite_join_site.group_site_id "
)
public class SiteDetail {
long id;
long memberId;
String address;
String groupName;
}
As can be seen I want to add id column to my database view.

Before version 3.25.0 of SQLite (anything below Android API 30) Views were not changed in accordance with table changes, as per
Compatibility Note: The behavior of ALTER TABLE when renaming a table was enhanced in versions 3.25.0 (2018-09-15) and 3.26.0 (2018-12-01) in order to carry the rename operation forward into triggers and views that reference the renamed table.
If any views refer to table X in a way that is affected by the schema change, then drop those views using DROP VIEW and recreate them with whatever changes are necessary to accommodate the schema change using CREATE VIEW.
https://www.sqlite.org/lang_altertable.html
In your migration you need to DROP the view (before the ALTER TABLE) and then CREATE the View (The SQL to create the View can be obtained from the generated Java after successfully compiling the project), if the API is less than 30 (or irrespective).

You can add a new column using alter query like
database.execSQL("ALTER TABLE table_name ADD column_name datatype")

At first create migration object as follow
val MigrationFrom1To2 = object : Migration(1, 2) {
override fun migrate(database: SupportSQLiteDatabase) {
database.execSQL("ALTER TABLE your_table_name ADD COLUMN your_column_name data_type")
}
}
After this, add above object in your database configuration
Room.databaseBuilder(
applicationContext,
MyAppDatabase::class.java,
"your_database_name"
).addMigrations(MigrationFrom1To2)
.build()

After creating a Trigger via Callback in Room db, the trigger is not getting created

I tried to add a trigger to limit the number of rows in my Room DB,
but the trigger is not getting created- as I see in the DB and in the generated JSON file that automatically created.
Also, migration code is not getting reached as well.
My code for creating the DB is:
#Database(entities ={Sensor.class, Meter.class, MeterHistory.class, SensorHistory.class,
ConnectivityHistory.class, GraphTypesHistory.class,
SensorSettingsHistory.class, UserInterfaceSettings.class, Temperature.class, LogSettingsHistory.class, BatteryUsage.class,
FunctionsHistory.class, MathChannelData.class}, version = 5)
#TypeConverters(UserUsageManager.DateConverter.class)
public abstract class UserUsageDB extends RoomDatabase {
public abstract UserUsageDao userUsageDao();
private static volatile UserUsageDB userUsageDBInstance;//singleton
static UserUsageDB getDatabase(final Context context) {
if (userUsageDBInstance == null) {
synchronized (UserUsageDB.class) {
if (userUsageDBInstance == null) {
userUsageDBInstance = Room.databaseBuilder(context.getApplicationContext(),
UserUsageDB.class, "user_usage_database")
.addCallback(LIMIT_CALLBACK)
.addMigrations(MIGRATION_1_2)
.fallbackToDestructiveMigration()
.build();
}
}
}
return userUsageDBInstance;
}
and the code of the migration and trigger callback:
static final Migration MIGRATION_1_2 = new Migration(1, 2) {
#Override
public void migrate(SupportSQLiteDatabase database) {
// Create the new meters table
database.execSQL("CREATE TABLE meters_new ( id TEXT NOT NULL, name TEXT NOT NULL, PRIMARY KEY(id))");
// Copy the data
database.execSQL("INSERT INTO meters_new ( id, name) SELECT id, name FROM meters");
// Remove the old table
database.execSQL("DROP TABLE meters");
// Change the table name to the correct one
database.execSQL("ALTER TABLE meters_new RENAME TO meters");
// Create the new sensors table
database.execSQL("CREATE TABLE sensors_new (id TEXT NOT NULL, name TEXT NOT NULL, type TEXT NOT NULL, PRIMARY KEY(id))");
// Copy the data
database.execSQL("INSERT INTO sensors_new ( id, name) SELECT id, name FROM sensors");
// Remove the old table
database.execSQL("DROP TABLE sensors");
// Change the table name to the correct one
database.execSQL("ALTER TABLE sensors_new RENAME TO sensors");
}
};
//create a trigger to limit row count of tables
static final RoomDatabase.Callback LIMIT_CALLBACK = new RoomDatabase.Callback(){
#Override
public void onCreate(#NonNull SupportSQLiteDatabase db) {
super.onCreate(db);
db.execSQL("Create Trigger IF NOT EXISTS Trigr_Limit_battery_usage_Delete \n" +
"AFTER Insert On battery_usage \n" +
" BEGIN \n"+
" DELETE from battery_usage\n"+
" where ts in (select ts from battery_usage order by date limit -1 offset 10);"+
" END");
db.execSQL("Create Trigger IF NOT EXISTS Trigr_Limit_sensor_settings_history_Delete \n" +
"AFTER Insert On sensor_settings_history \n" +
" BEGIN \n"+
" DELETE from sensor_settings_history\n"+
" where ts in (select ts from sensor_settings_history order by date limit -1 offset 10);"+
" END");
}
#Override
public void onOpen(#NonNull SupportSQLiteDatabase db) {
super.onOpen(db);
}
};
Can anyone help me out?
Thanks!!

but the trigger is not getting created
You are Overriding the onCreate method, this only gets called when the database is actually created (first time the App is installed). You want to Override the onOpen method to add the Triggers.
migration code is not getting reached as well.
Migration is from version 1 to 2 as per (new Migration(1, 2)), you have version 5 coded in the App as per version = 5 . Your migration will only be called when the version stored in the database is 1 and the version coded in the App is 2. You may find Understanding migrations with Room useful. Or perhaps Migrating Room databases

Room cannot verify the data integrity even when I change the version number

I'm creating my first Room SQLite Migration and I can't figure out what I'm supposed to do now, the version number in the old Database is 2 and I'm changing it to 3. I get the following error in the console when I try to compile the App:
java.lang.IllegalStateException: Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number. You can simply fix this by increasing the version number.
My Database code looks like this:
#Database(entities = {Vehicle.class, ShockAbsorver.class, Customer.class, Review.class, BadRatingOption.class}, version = 3)
public abstract class AppDatabase extends RoomDatabase{
private static AppDatabase INSTANCE;
public abstract VehicleDao vehicleDao();
public abstract ShockAbsorverDao absorverDao();
public abstract CustomerDao customerDao();
public abstract ReviewDao reviewDao();
public abstract OptionDao optionDao();
static final Migration MIGRATION_2_3 = new Migration(2,3) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
database.execSQL("PRAGMA foreign_keys=off;");
// REMOVE COLUMN 'original' FROM Vehicle
database.beginTransaction();
database.execSQL("ALTER TABLE vehicle RENAME TO temp_vehicle;");
database.execSQL("DROP INDEX vehicleRemoteId");
database.execSQL("CREATE TABLE vehicle(id TEXT NOT NULL PRIMARY KEY,idSuperk INTEGER,idRemote TEXT,model TEXT,year INTEGER,plate TEXT,km INTEGER, FOREIGN KEY(idRemote) REFERENCES review(vehicleId));");
database.execSQL("CREATE INDEX vehicleRemoteId ON vehicle(idRemote)");
database.execSQL("INSERT INTO vehicle (id,idRemote,model,year,plate,km) SELECT id,idRemote,model,year,plate,km FROM temp_vehicle;");
database.execSQL("DROP TABLE temp_vehicle;");
database.endTransaction();
// ADD CLOUMN 'id' INTO ShockAbsorver
database.execSQL("ALTER TABLE shockAbsorver ADD COLUMN id TEXT");
// CREATE TABLE BadRatingOption
database.execSQL("CREATE TABLE options (id TEXT NOT NULL PRIMARY KEY,absorverId TEXT,option INTEGER);");
database.execSQL("CREATE UNIQUE INDEX id ON options(id)");
database.execSQL("CREATE INDEX absorverId ON options(absorverId)");
//POPULATE BadRatingOption WITH OPTIONS FROM ShockAbsorver
Cursor cursor = database.query("SELECT * FROM shockAbsorver");
ArrayList<String> reviewIds = new ArrayList<>();
ArrayList<Integer> indexes = new ArrayList<>();
ArrayList<Integer> options = new ArrayList<>();
try {
while (cursor.moveToNext()) {
reviewIds.add(cursor.getString(cursor.getColumnIndex("reviewId")));
indexes.add(cursor.getInt(cursor.getColumnIndex("index")));
options.add(cursor.getInt(cursor.getColumnIndex("option")));
}
} catch (Exception e) {
e.printStackTrace();
}finally{
cursor.close();
}
for(int i = 0;i<reviewIds.size();i++){
String absorverId = UUID.randomUUID().toString();
String optionId = UUID.randomUUID().toString();
database.execSQL("UPDATE shockAbsorver SET id='"+absorverId+"' WHERE reviewId = '"+reviewIds.get(i)+"' AND `index` = "+String.valueOf(indexes.get(i))+";");
database.execSQL("INSERT INTO options (id,absorverId,option) VALUES ('"+optionId+"','"+absorverId+"','"+options.get(i)+"');");
}
reviewIds = null;
indexes = null;
options = null;
//REMOVE 'option' FROM ShockAbsorver
database.beginTransaction();
database.execSQL("ALTER TABLE shockAbsorver RENAME TO temp_shockAbsorver;");
database.execSQL("CREATE TABLE shockAbsorver(`index` INTEGER NOT NULL,id TEXT,remoteId TEXT,isGood INTEGER,rating INTEGER,isNew INTEGER NOT NULL,isEdited INTEGER NOT NULL,isOriginal INTEGER,observation TEXT,filename TEXT,vehicle TEXT NOT NULL,reviewId TEXT NOT NULL,remoteReviewId TEXT,PRIMARY KEY(reviewId,`index`), FOREIGN KEY(reviewId) REFERENCES review(id));");
database.execSQL("INSERT INTO shockAbsorver (`index`,remoteId,isGood,rating,isNew,isEdited,isOriginal,observation,filename,vehicle,reviewId,remoteReviewId) SELECT `index`,remoteId,isGood,rating,isNew,isEdited,isOriginal,observation,filename,vehicle,reviewId,remoteReviewId FROM temp_shockAbsorver;");
database.execSQL("DROP TABLE temp_shockAbsorver;");
database.endTransaction();
database.execSQL("PRAGMA foreign_keys=on;");
}
};
public static AppDatabase getAppDatabase(Context context){
if(INSTANCE == null){
INSTANCE = Room.databaseBuilder(context.getApplicationContext(),
AppDatabase.class,
"shock-absorver-database")
.allowMainThreadQueries()
.addMigrations(MIGRATION_2_3)
.build();
}
return INSTANCE;
}
public static void destroyInstance(){
INSTANCE = null;
}
}
While I was writing the migration code I was getting "Migration didn't properly handle" until I fixed then all, after that I started getting the data integrity error.
Thanks in advance.

For production type:
You need to write code for migrating from one DB to another
https://developer.android.com/training/data-storage/room/migrating-db-versions
For develop debuging version:
Try to uninstall app from your phone. (it will remove old database).
And than simply build and run again.

You need to call database.setTransactionSuccessful()
Here is the standard idiom for transactions:
db.beginTransaction();
try {
...
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}

#Database(entities = {Vehicle.class, ShockAbsorver.class, Customer.class, Review.class,
BadRatingOption.class}, version = 3)
Another variant(answer above also works) - just increase version from 3 to 4