I have an Android app with a database with version 1.
Now I like to change the structure of the database and write all the migration code etc.
The problem I have is how to test this.
I need to repeatedly have the old version of the app with its old DB so I can test the update process to the new app with the new DB several times.
I thought to simplify things I make a copy of my project and just rename it new - leaving the manifest and everything unchanged!!!!
The idea is to run/install the old version of the app with the old DB structure using Eclipse so I can create the start situation of my update.
Now to simulate the user update to the new app version I install it just over the old version using Eclipse again) - but even without having changed the db I get an error that the DB already exists ???
I am confused (I have only changed the project name, not the manifest) I would have expected that I can just install the new version over the old version and hereby test the user update of the app.
But that does not work.
How would I best do this? (without having the old and the new database code in the very same project. since if I have the same project I have two different data base structures in there and need to build in a switch how it should start up, ie as old or new. I find it cleaner to just write the new version of the app with the new database structure in there)
Room is one of the Android Architecture Components for database stuff.
Room provides a testing Maven artifact to assist with this testing process.
First you need to export db scheme by set the room.schemaLocation annotation processor property in your build.gradle file
android {
...
defaultConfig {
...
javaCompileOptions {
annotationProcessorOptions {
arguments = ["room.schemaLocation":
"$projectDir/schemas".toString()]
}
}
}
}
#RunWith(AndroidJUnit4.class)
public class MigrationTest {
private static final String TEST_DB = "migration-test";
#Rule
public MigrationTestHelper helper;
public MigrationTest() {
helper = new MigrationTestHelper(InstrumentationRegistry.getInstrumentation(),
MigrationDb.class.getCanonicalName(),
new FrameworkSQLiteOpenHelperFactory());
}
#Test
public void migrate1To2() throws IOException {
SupportSQLiteDatabase db = helper.createDatabase(TEST_DB, 1);
// db has schema version 1. insert some data using SQL queries.
// You cannot use DAO classes because they expect the latest schema.
db.execSQL(...);
// Prepare for the next version.
db.close();
// Re-open the database with version 2 and provide
// MIGRATION_1_2 as the migration process.
db = helper.runMigrationsAndValidate(TEST_DB, 2, true, MIGRATION_1_2);
// MigrationTestHelper automatically verifies the schema changes,
// but you need to validate that the data was migrated properly.
}
}
You can read an official document for more info
You can create the main file 'create.sql' and the migrations files called '1.sql, 2.sql,.., n.sql'.
Store all these files in assets for example.
If the user has no database, apply create.sql
If the user needs a database upgrade from A to Z, then apply a+1.sql, a+2.sql, …, z.sql.
You need implement applyUpgrades method that you'll use on upgrade version of DB and for migration tests. It's look like:
fun Context.applyUpgrades(db: SQLiteDatabase, from: Int, to: Int): Unit =
(from+1..to).forEach { index ->
val migrationSql = getMigrationFromAssets(index)
db.execSQL(migrationSql)
}
Your migration tests will look like:
#Test
fun upgrade10to11() { // when you upgrade from 10 to 11
val db = SQLiteDatabase.create(null)
// apply all previous migrations
applyUpgrades(db, from = INITIAL, to = 10)
val valuesBefore = ..
// our new migration
applyUpgrades(db, from = 10, to = 11)
val valuesAfter = ..
// assert that valuesBefore and valuesAfter are correct
}
Also you can check that you did not forget write test for new migration:
#Test
public void hasAllNecessaryTests() {
assertEquals(DB_VERSION, 11);
}
Trello bless
As you only need the data and structures to test migration, you could make backups of the old databases and use them in your tests.
So for each test:
install app, put data in, make backup (once)
restore backup, run test (often)
That way you only have to make the backup once and can restore them on every test.
If your tests are automated, you could even restore the backups automatically as well.
For SQLite, making backups is as easy as copying the database file, but pretty much every database has a way to make and restore backups.
Your updated app seems to try to create the database (not the data inside) again, but with the database of the old app still being around that won't work.
This may be caused by the app not noticing it was installed before.
Related
I have an application with prefilled database (I use Room, Kotlin). Database class:
#Database(
entities = [FurnitureModel::class, ImageModel::class],
version = Database.VERSION,
exportSchema = true
)
abstract class Database : RoomDatabase() {
abstract val furnitureDao: FurnitureDao
abstract val imageDao: ImageDao
companion object {
const val NAME = "application-data"
const val VERSION = 3
#Volatile
private var instance: Database? = null
fun getInstance(context: Context): Database {
synchronized(this) {
var inst = instance
if (inst == null) {
inst = Room.databaseBuilder(
context.applicationContext,
Database::class.java,
"$NAME-local.db"
)
.fallbackToDestructiveMigration()
.createFromAsset("$NAME.db")
.build()
instance = inst
}
return inst
}
}
}
}
It worked fine. Once I decided to update some data inside database. No stucture changes, just adding and changing some data. For versions, I changed constant VERSION from 3 to 4. When I run the application, I've got empty database. As I understand, it is possible to get empty database with fallbackToDestructiveMigration in few situations. I could made some mistake somewhere in data. To get more details I decided to run application without preinstalled previous version. Clean install, if I can say like this. To do it, I wiped data for Virtual device in AVD Manager, removed build directories in application and app directories and run 'Invalidate cache/restart...' in Android Studio (I know that there is no need to do it). But the result is always the same: trying to install my application I always get error
Caused by: java.lang.IllegalStateException: A migration from 3 to 4 was required but not found. Please provide the necessary Migration path via RoomDatabase.Builder.addMigration(Migration ...) or allow for destructive migrations via one of the RoomDatabase.Builder.fallbackToDestructiveMigration* methods.
As I understand, the error must not be shown when you install the application for the first time. But it is shown. I tried to create new virtual device and run app there, but the error is still there.
What should I do to forse Android Studio to forget that the application was installed before?
Thank you.
UPD1: I don't need to keep any data, actually I want to delete all data from previous versions of application. But after wiping data it still works as if I didn't wipe it. This is the problem.
RESULT: I found that the problem was in prefilled database. Unfortunatedly I can't find that is exactly wrong with database. I disabled creating database from assets, run application so that Room created bew empty database, copied the database from emulator and compared each table with my database. No any difference. So to solve the problem I just moved all data in each table to the same table in new empty database (using DataGrid). After running the application new database the problem disapeared.
I've been struggling performing a simple migration. What I just want to achieve is add a new Class in realm.
The code below is inside a method that is called inside onCreate.
Realm.init(this)
val config = RealmConfiguration.Builder()
.name("db_name")
.schemaVersion(5L)
.migration { realm, oldVersion, newVersion ->
val schema = realm.schema
var _oldVersion = oldVersion
if (_oldVersion == 4L) {
if (schema.contains(XModel::class.java.simpleName))
schema.remove(XModel::class.java.simpleName)
if (!schema.contains(XModel::class.java.simpleName))
schema.create(XModel::class.java.simpleName)
.addField(XModel::id.name, Long::class.javaPrimitiveType,
FieldAttribute.PRIMARY_KEY)
...
.addField(XModel::N.name, Int::class.javaPrimitiveType)
_oldVersion += 1
}
}
.build()
Realm.setDefaultConfiguration(config)
As what the title suggest, the new class in the schema was created inside the migration object, but when I try to access it in other parts of the application using a realm query or a simple call to schema.get("XModel") it will throw an error XModel doesn't exist in current schema. Any comment will really help. Thank you...
Edit:
Additional information. I have 2 realm objects, each are in different android modules, one module is dependent to the other. I somehow have some progress, now Im a bit confuse, do I need to declare 2 configurations? Then it would mean 2 realm instance? How to switch from both, I want to merge them into 1 realm.
Edit2:
Another clarification about realm. If you have 2 android modules, each of them using realm, will they have different realm even if in the realm configuration they have the same name?
Background
I want to give you a background of what im doing because I think its needed to fully understand my case.
Originally I only have one module, but then after refactoring and also because of future apps to be develop, I need to pull out the common classes from the existing module and put it in a separate lower-level module that the future apps can depend on. This new lower-level module will also be responsible for most of the data layer, so realm was transferred to this module. But I can't just ignore the realm of the existing app because some users might already populated it, and I need to transfer those data to the new database.
I am using Room Persistence Library 1.1.0. I could find the database file at /data/data/<package_name>/databases/ using Android Studio's Device File Explorer.
It contains multiple tables and I can access contents of that tables without any problem using room-DAOs. However when opening with sqlite-browser, is shows no table.
What might be the reason? Is it possible to resolve the issue without switching back to old SQLiteOpenHelper from room?
Solution
To open such databases* with sqlite-browser, you need to copy all three files. All must be in the same directory.
* Databases stored in multiple files as stated in the question.
Why three files?
As per docs, Starting from version 1.1.0, Room uses write-ahead logging as default journal mode for devices which has sufficient RAM and running on API Level 16 or higher. It was Truncate for all devices until this version. write-ahead logging has different internal structure compared to Truncate.
Take a look at the files temporary files used by SQLite now and then :
Until version 1.1.0
From version 1.1.0
If you want to change the journal mode explicitly to Truncate, you can do it this way. But, it is not recommended because WAL is much better compared to Truncate.
public static void initialize(Context context) {
sAppDatabase = Room.databaseBuilder(
context,
AppDatabase.class,
DATABASE_NAME)
.setJournalMode(JournalMode.TRUNCATE).build();
}
Is it possible to move it to single file without changing to Truncate ?
Yes, it is. Query the following statement against the database.
pragma wal_checkpoint(full)
It is discussed in detail here here.
Copy all three files from Device File Explorer in AndroidStudio to your PC directory and open the db file in Db Browser for SQLite (http://sqlitebrowser.org). Make sure all three files are in the same folder.
You can use the wal_checkpoint pragma to trigger a checkpoint which will move the WAL file transactions back into the database.
theRoomDb.query("pragma wal_checkpoint(full)", null)
or
// the result
// contains 1 row with 3 columns
// busy, log, checkpointed
Cursor cursor = theRoomDb.query("pragma wal_checkpoint(full)", null)
See PRAGMA Statements for more details about the pragma parameter values and results.
If the WAL is not enabled the pragma does nothing.
By the way, I tested with Room 1.1.1, and the WAL mode was not used by default, I had to enable it.
Room database Export and Import Solution
Im facing same problem in one of my project, i spend two days to resolve this issue.
Solution
Don't create multiple instance for Room library. Multiple instance creating all the problems.
MyApplication
class MyApplication: Application()
{
companion object {
lateinit var mInstanceDB: AppDatabase
}
override fun onCreate() {
super.onCreate()
mInstanceDB = AppDatabase.getInstance(this)
}
}
AppDatabase
fun getInstance(context: Context): AppDatabase
{
if (sInstance == null) {
sInstance = Room.databaseBuilder(context.applicationContext,AppDatabase::class.java, "database").allowMainThreadQueries().build()
return sInstance!!
}
}
Now use this instance in any number of activity or fragment just like that
{
var allcustomer = MyApplication.mInstanceDB.customerDao.getAll()
}
Export and Import use this library
implementation 'com.ajts.androidmads.sqliteimpex:library:1.0.0'
Github link
MainActivity class
public class MainActivity extends BaseActivity {
private AppDatabase db;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// Database creation
db = Room.databaseBuilder(getApplicationContext(),
AppDatabase.class, "Medimap-db").build();
// Profile profile=new Profile("rajitha","12-2-345","male","no 345","test med","test emergency","testurl");
// db.profileDao().insert(profile);
// Toast.makeText(getApplicationContext(),"success",Toast.LENGTH_SHORT).show();
new DatabaseAsync().execute();
}
private class DatabaseAsync extends AsyncTask<Void, Void, Void> {
#Override
protected void onPreExecute() {
super.onPreExecute();
//Perform pre-adding operation here.
}
#Override
protected Void doInBackground(Void... voids) {
//Let's add some dummy data to the database.
Profile profile = new Profile("rajitha", "12-2-345", "male", "no 345", "test med", "test emergency", "testurl");
db.profileDao().insert(profile);
return null;
}
#Override
protected void onPostExecute(Void aVoid) {
super.onPostExecute(aVoid);
Toast.makeText(getApplicationContext(), "success", Toast.LENGTH_SHORT).show();
//To after addition operation here.
}
}
}
AppDatabase class
#Database(entities = {Profile.class, Medicine.class}, version = 1)
public abstract class AppDatabase extends RoomDatabase {
public abstract ProfileDao profileDao();
public abstract MedicineDao medicineDaoDao();
}
ProfileDao
#Dao
public interface ProfileDao {
#Query("SELECT * FROM profile")
List<Profile> getAll();
// #Query("SELECT * FROM user WHERE uid IN (:userIds)")
// List<Profile> loadAllByIds(int[] userIds);
// #Query("SELECT * FROM user WHERE first_name LIKE :first AND "
// + "last_name LIKE :last LIMIT 1")
// User findByName(String first, String last);
#Insert
void insertAll(Profile... profiles);
#Insert
void insert(Profile profile);
#Delete
void delete(Profile profile);
}
Here I got an error after the first run of the app. It seems like the app is trying to create DATABASE once again, but there already is an existing one so they suggested to change the version code. My requirement is that I only need to insert a new data set. How can I achieve that? Thanks in advance. Here is the logcat error:
Caused by: java.lang.IllegalStateException: Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number. You can simply fix this by increasing the version number.
Android 6.0+
For those of you who come across this now a days, simply uninstalling might not help and it will drive you nuts, so before you have to figure it out, let me share the issue: Google introduced the "autobackup" functionality in
Android 6.0, which brings back a DB when you reinstall. (https://developer.android.com/guide/topics/data/autobackup.html)
You can simply add...
<application ...
android:allowBackup="false">
</app>
...to disable this functionality and you are good to go (now you can simply uninstall to delete the database).
If you're only developing the application at the moment, it means that you are not on Production and uninstalling the app from device and install it again will work as required.
Increase the version:
#Database(entities = {EmployeeEntry.class}, version = 2, exportSchema = false)
Took from
link
If you don't care about migrations when update entities:
Increase database version
Add fallbackToDestructiveMigration() method at the time of building database, e.g:
Room.databaseBuilder(appContext, AppDatabase.class, "appdb.db")
.fallbackToDestructiveMigration()
.build();
Hope this help someone ;)
There are 3 ways to fix this data integrity exception.
1) If you don't want to increase your Database version and you can bear data loss
Just clear your App data from App settings after installing new build in device and launch App again, it would work without data integrity exception
2) If you can increase your Database version but still you can bear data loss
If you want to make it seamless( no crash due to schema update), you can simply increase Database version (let say from 1 to 2) and mention Default migration strategy in Database Builder fallbackToDestructiveMigration(). Then on installing this App, your previous DB data would clear up and you wouldn't get this data integrity exception.
3) If you can increase your Database version and also want to save your previous data
You need to increase your Database version(let say from 1 to 2) but also need to implement proper migrations logic (based on your lower version and higher version) in your App. Then on installing new version, you will not get this data integrity exception.
For migrations of Room you can read out this nice article written for migrations
For official documentation of Room migrations go to link
In my case, reinstalling didn't help and also I couldn't set android:allowBackup:true because one of the libraries I had used required it. So I went to Settings > Apps > My app, and cleared data and memory for that app. Problem gone;)
Step 1: Install new build
Step 2: Clear all app data from Settings.
Step 3: Open app ( will work fine now )
You can also use adb shell pm uninstall com.package.app in your terminal to fully uninstall the app. After that, install it again and the error will disappear.
I'm doing my first Realm migration and started thinking about the version number. On what is this version number based?
Because if it is based on what is on your phone, how do I handle it if a new person installs the app and gets a migration? Because it will also update the fields which where already set because of a fresh install.
Christian from Realm here. The migration API is still in a very experimental state and kinda ugly, so right now the version number always start with 0, and the only way for changing that is through a migration.
This means that if you want a fresh install with a different version other than 0, you will have to do something like:
// Pseudo code
public class RealmHelper() {
private static SharedPreferences prefs;
public static Realm getInstance() {
if (!prefs.getBoolean("versionSet", false)) {
String path = new File(context.getFilesDir(), Realm.DEFAULT_REALM_NAME).getAbsolutePath();
Realm.migrateRealmAtPath(path, new RealmMigration() {
#Override
public long execute(Realm realm, long version) {
return 42; // Set version numbers
}
})
prefs.edit().putBoolean("versionSet", true).apply();
}
return Realm.getInstance();
}
}
This is going to be a lot better soon though: https://github.com/realm/realm-java/pull/929
Realm migrations in 0.84.2 are changed quite a bit (see Christian's hint on the new API), the key points on making a realm (0.84.2) migration work for me were understanding that:
The schemaVersion is always 0 when your app has a realm db without
specifying the schemaVersion. Which is true in most cases since you
probably start using the schemaVersion in the configuration once you
need migrations & are already running a live release of your app.
The schemaVersion is automatically stored and when a fresh install of your app occurs and you are already on schemaVersion 3, realm
automatically checks if there are exceptions, if not it sets the
schemaVersion to 3 so your migrations aren't run when not needed.
This also meens you don't have to store anything anymore in
SharedPreferences.
In the migration you have to set all values of new columns when the type is not nullable, current version of realm, ...
Empty Strings can be inserted but only when setting convertColumnToNullable on the column