Rails-like database schema version upgrades for Android - android

So, I'm working with an Android SQLite database, with SQLiteOpenHelper. It seems to have a concept of database versions and upgrading... but it looks like you're supposed to roll your own code to actually do the upgrades, and keep your onCreate method up-to-date with your updates.
Coming from a Rails development background, this seems a little primitive. For the uninitiated, Rails allows you to just write a class-per-version-upgrade, and Rails takes care of applying whichever versions need to be... this applies for DB creation, too; you only have one representation of your database schema, the set of migrations. Rails also does a DB-independent representation of the schema changes, but that isn't necessary for Android because it only supports SQLite (which is fine).
Has anyone written a decent schema migration helper class for Android that allows me to get closer to DB schema management nirvana (RailsEdition(TM))? It'll save me rolling my own ugly implementation.

Given that I didn't find anything that supported Android, actually worked, didn't require me to subscribe to an insane database world view, and didn't cost a lot (hobby project, no dice), I came up with the following bodgy hack. It's not clever, but it at least allows me to think about my schemas in a way that I'm familiar with. I don't expect it'd work real well for a large codebase/database schema, but if you've got that you can probably afford to pay for something.
public class AppDatabase extends SQLiteOpenHelper {
public static final String DATABASE_NAME = "main";
public static final int LATEST_VERSION = 4;
public static SQLiteDatabase open(Context ctx) {
AppDatabase db = new AppDatabase(ctx);
return db.getWritableDatabase();
}
public AppDatabase(Context ctx) {
super(ctx, DATABASE_NAME, null, LATEST_VERSION);
}
#Override
public void onCreate(SQLiteDatabase db) {
onUpgrade(db, 0, LATEST_VERSION);
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
for (int i = oldVersion+1; i <= newVersion; i++) {
switch (i) {
case 1:
db.execSQL("CREATE TABLE blah ( " +
"id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT," +
"start CHAR(4)," +
"end CHAR(4)" +
")");
break;
case 2:
db.execSQL("CREATE TABLE fortnights ( " +
"first_day DATE PRIMARY KEY" +
")");
break;
case 3:
db.execSQL("ALTER TABLE shifts ADD top CHAR(4)");
db.execSQL("ALTER TABLE shifts ADD bottom CHAR(4)");
db.execSQL("UPDATE shifts set top=start, bottom=end");
break;
case 4:
db.execSQL("ALTER TABLE shifts ADD callout BOOLEAN DEFAULT 0");
break;
}
}
}
}

Related

SQLite database migration appears to only partially apply in Espresso test

We have an SQLite database and a corresponding SQLiteOpenHelper subclass. This helper has an onDowngrade implementation that I would like to write an Espresso test for.
The full onDowngrade implementation is available here. This is a simplified version of it:
#Override
public void onDowngrade(SQLiteDatabase db, int oldVersion, int newVersion) {
db.execSQL("CREATE TABLE IF NOT EXISTS foo_tmp (_id integer primary key, bar text not null, baz text not null);");
db.execSQL("INSERT INTO foo_tmp(_id,bar,baz) SELECT _id,bar,baz FROM foo;");
db.execSQL("DROP TABLE IF EXISTS foo;");
db.execSQL("RENAME TABLE foo_tmp TO foo;");
}
The test loads a database dump with a very high version number and added or removed columns. It then gets a readable database and ensures that the version has been downgraded to the current expected version and that the column names are the expected column names. The full source is available here. This is what it looks like:
#Test
public void testMigration() throws IOException {
writeDatabaseFile("database" + File.separator + dbFilename);
InstancesDatabaseHelper databaseHelper = new InstancesDatabaseHelper();
SQLiteDatabase db = databaseHelper.getReadableDatabase();
assertThat(db.getVersion(), is(InstancesDatabaseHelper.DATABASE_VERSION));
List<String> newColumnNames = InstancesDatabaseHelper.getInstancesColumnNames(db);
assertThat(newColumnNames, contains(InstancesDatabaseHelper.CURRENT_VERSION_COLUMN_NAMES));
}
Everything works as intended if I manually load the same database dumps into the app. However, when I run this test, it looks like the last RENAME in the migration is not executed. If I comment out the last two SQL statements in the migration (dropping the original table and renaming the temporary table to the original table name), I can assert that the temporary table has the expected contents (here is a commit that shows this).
With some experimentation, we have found that adding databaseHelper.getReadableDatabase().close(); in the test after instantiating the SQLiteOpenHelper makes the tests pass. Given that the onDowngrade call is wrapped in a transaction, I don't understand how this is possible.
Could this point to a bug in our onDowngrade implementation? Is triggering migrations in Espresso tests different in some way?
There probably is a race condition, because SQLite is a shared resource.
eg. when the test runs before the last one COMMIT statement was issued.
Wrap it into a transaction (also see Isolation In SQLite):
if(! BuildConfig.DEBUG) {
db.beginTransaction();
} else {
db.beginTransactionWithListener(new SQLiteTransactionListener() {
#Override public void onBegin() {Log.d(LOG_TAG, "onBegin()");}
#Override public void onCommit() {Log.d(LOG_TAG, "onCommit()");}
#Override public void onRollback() {Log.d(LOG_TAG, "onRollback()");}
});
}
try {
db.execSQL("CREATE TABLE IF NOT EXISTS foo_tmp (_id integer primary key, bar text not null, baz text not null);");
db.execSQL("INSERT INTO foo_tmp(_id,bar,baz) SELECT _id,bar,baz FROM foo;");
db.execSQL("DROP TABLE IF EXISTS foo;");
db.execSQL("RENAME TABLE foo_tmp TO foo;");
db.setTransactionSuccessful();
} catch(SQLException e){
Log.d(LOG_TAG, "" + e.getMessage());
} finally {
db.endTransaction();
}
db.close();

Why does SQLiteOpenHelper drop the table in "onUpgrade" method?

I learned a way which use SQLite3 on Android OS recently.
But it has a non acceptable part.
I can't understand why drop the table when called "onUpgrade" method of SQLiteOpenHelper.
Why need "onUpgarde" method?
If code executes "drop table", table data of old version DB will be removed, isn't it?
Why delete existing data of old DB?
How to restore existing DB data when drop the table?
[Here is learned code]
public class MySQLiteOpenHelper extends SQLiteOpenHelper {
final String CONFIRMED_SHEETS_TABLE = "confirmed_sheets";
public MySQLiteOpenHelper(Context context, String name, SQLiteDatabase.CursorFactory factory, int version) {
super(context, name, factory, version);
}
#Override
public void onCreate(SQLiteDatabase sqLiteDatabase) {
String sql = "create table" +
CONFIRMED_SHEETS_TABLE +
"_id integer primary key autoincrement, " +
"group text, " +
"num001 text, " +
"num002 text, " +
"num003 text, " +
"num004 text, " +
"num005 text, " +
"num006 text, " +
"date text)";
sqLiteDatabase.execSQL(sql);
}
#Override
public void onUpgrade(SQLiteDatabase sqLiteDatabase, int oldVersion, int newVersion) {
// non acceptable part //
String sql = "drop table if exists " + CONFIRMED_SHEETS_TABLE;
sqLiteDatabase.execSQL(sql);
onCreate(sqLiteDatabase);
}
}
If code executes "drop table", DB table data of old version DB will be
removed, isn't it?
Yup
Why need "onUpgrade" method?
If you are switching databases (for example because you added a new column), your app (usually) now depends on that change. Increasing the database version in your Helper class calls onUpgrade(), which allows you to take care of any migration to prepare the app to use your new schema.
Did you know why learned code executes "drop table"?
Convenience. It's not necessarily the right approach, but a database change can make it hard to take old data and merge it in the new table. Thus, it is easier logic-wise to simply start anew.
If you want to merge an existing and new table, have a look at this question.
You do not need to perform a DROP TABLE in onUpgrade(), as it is currently written in your code. The purpose of onUpgrade() is for your app to check if a new version of your app's database is being installed on a user's device, and if so, if there are any changes to your database, such as adding a new column, you can make those changes within onUpgrade(). If you never change your database schema, you never need to do anything in onUpgrade().
For more information and a introductory tutorial on Databases in Android, refer to the Notepad sample code here.
EDIT: also, here's an example of an onUpgrade() I wrote for one of my apps:
/**
* Handle upgrades to the database.
*/
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
if(DBG) Utils.log(TAG, "DBHelper: onUpgrade()...");
// Logs that the database is being upgraded
Utils.log(TAG, "w", "Upgrading database from version " + oldVersion + " to "
+ newVersion);
/*
* Every time you add new columns to the database in the phone, you will want
* to increment the Database version above and then add a condition in here for
* upgrading to it. Otherwise it will cause upgrading users to be nontrivial and
* lead to unnecessary crashes or upgrade instructions.
*/
if (newVersion > oldVersion && newVersion <= 4) {
db.execSQL("ALTER TABLE " + TrackerDb.SomeTable.TABLE_NAME + " ADD COLUMN "
+ TrackerDb.SomeTable.COLUMN_NAME_DATE_MODIFIED + " TEXT");
}
}
OnUpgrade() is called when db versions "old and new" does not match; which means that the user wants to change Database Structure (Dropping a table, Adding new Table, Modifying Table definition).
So the OnUpgrade() should contain the logic as:
Take Backup of the existing data (e.g. Generate insert statements for existing
data).
Modify Database Structure accordingly.
Restore the data from the backup.

Manage DB before uploading to playstore in android

I have uploaded an application to play store couple of weeks back. This application involves sqlite database that stores information on username, password, other details that given are by user while using the application locally.
Now I have couple of more tables and fields added to database and wanna upload the application to playstore as an update?
My worry is if the user updates the application from playstore - After update - all the data stored in database will be saved or will the user has to recreate everything from scratch?
Let me know!
Thanks!
You have to override the onUpgade method of SQLiteOpenHelper. In the OnUpgrade method you can either erase the data(drop sqlite command) or maintain the data with the additional columns(alter sqlite command) or create new table (create sqlite command).
Refer the following snippet.
I assume your version would be 1.(Plz check the constructor of your SqliteOpenHelper class)
Increment the version by 1.
class DatabaseHelper extends SqliteOplenHelper{
private static final int DATABASE_VERSION = 2; //new version of the database
private static final int Database_name = "MyDatabase";
private static final String alterUserName = "alter table users add name text";
private static final String table_users = "create table if not exists "
+ users + "(" + "_id integer primary key autoincrement,"
+ "email text" + ")";
public DatabaseHelper(Context context) {
super(context, Database_name, null, DATABASE_VERSION);
cntxt = context;
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
db.execSQL(table_users);
db.execSQL(alterUserName);
}
}
Now everytime when you roll the next update with database changes be sure to increment the database version by 1 else let it remain the same.
This isn't done for you automatically. In your SQLiteOpenHelper, you need to increment the Schema integer. This will trigger the on upgrade method for your existing users.
Adding a table is not a problem, just do this in onUpgrade, nothing breaks.
However to add fields, you should use the 'ALTER TABLE' SQL command
If you add new columns you can use ALTER TABLE to insert them into a live table. If you rename or remove columns you can use ALTER TABLE to rename the old table, then create the new table and then populate the new table with the contents of the old tab
See the official reference here

Storing application database schema

In example apps database is in most cases single table, so db schema is stored in static variable.
Storing large schema in seperate file is more friendly for me.
How can I do that? I thought about using resources (R.strings.db_schema) but probably there is a better way.
Could somebody give me any advice?
You could put the schema data in a raw file under res/raw. Then you can just load and parse that file the first time.
The way I do is to have a class per table, named after the table with "Table" suffix (e.g. PlayerTable or EventTable).
These classes contain all the static variable for the table name and all the field names, and they also contain two static methods:
public static void onCreate(SQLiteDatabase database)
public static void onUpgrade(SQLiteDatabase database, int oldVersion, int newVersion)
So that my SQLiteOpenHelper can just call all of them, without having hundreds of static variables with all the fields and create queries. E.g:
#Override
public void onCreate(SQLiteDatabase database) {
PlayerTable.onCreate(database);
EventTables.onCreate(database);
..... any other table you have .....
}
This class is then injected into all my data access objects (select / update / insert queries). For them I have dedicated classes that contain all my methods, by functionality (e.g. EventHandlingDAO for all the queries that deal with event handling).
And finally, theses DAO are injected into the activities that need them, when needed.
EDIT: A few more details about my code:
My main objects are the DAO (data access objects), in which I have methods like:
// in EventHandlingDAO:
public void addEvent(Event event) {
SQLiteDatabase database = databaseHelper.getWritableDatabase();
try {
database.execSQL("INSERT INTO " + EventTable.EVENT_TABLE_NAME + " (...."); // list of fields and values
} finally {
database.close();
}
}
public List<Event> getAllEvents() {
final List<Event> result = new ArrayList<Event>();
SQLiteDatabase database = databaseHelper.getReadableDatabase();
try {
final Cursor cursor = database.rawQuery("SELECT " + EventTable.KEY_NAME + ", " + EventTable.KEY_DATE_AS_STRING + " FROM " + EventTable.TABLE_NAME, null);
cursor.moveToFirst();
// ... rest of the logic, that iterates over the cursor, creates Event objects from the cursor columns and add them to the result list
return result;
} finally {
database.close();
}
}
So in that DAO, I have my databaseHelper object, which instanciates my class that extends SQLiteOpenHelper with the methods I talked about above.
And of course, I have interfaces to all my DAO, so that I can inject a Stub or mocked implementation in my tests (or experiment with different implementations if I want to try another solution based on SharedPreference for example)
And the code for my PlayerTable table:
public static void onCreate(SQLiteDatabase database) {
database.execSQL(TABLE_CREATE); // TABLE_CREATE is my "CREATE TABLE..." query
}
public static void onUpgrade(SQLiteDatabase database, int oldVersion, int newVersion) {
// A bit blunt, that destroys the data unfortunately, I'll think about doing something more clever later ;)
database.execSQL("DROP TABLE IF EXISTS " + TABLE_NAME);
onCreate(database);
}

Android: duplicate values are being inserted in to database

Each time i run the project same values are also being inserted in the android database. Even i have given the drop if exists query.What i need is that the data in the database gets updated only if there are some changes in the response from the server side instead of cresting database every time but what is happening with me is that same values got insertes again in the tables. How do I solve this? Following is my code:
public void onCreate(SQLiteDatabase database) {
try{
// onUpgrade(database, oldVersion, newVersion)
database.execSQL(DATABASE_CREATE);
}
catch (Exception e) {
System.out.println("in on create db"+e);
}}
public void onUpgrade(SQLiteDatabase database, int oldVersion,
int newVersion) {
database.execSQL("DROP TABLE IF EXISTS" +DATABASE_CREATE);
onCreate(database);
}
private static final String INSERT = "insert into "
+ DATABASE_TABLE + "(KEY_TYPE,KEY_ENCODING,KEY_WIDTH,KEY_HEIGHT,KEY_DATA,KeyIId)"+" values (?,?,?,?,?,?)";
public WineDatabaseAdapter(Context context) {
try{
this.context = context;
openHelper = new WineDatabaseHelper(context);
this.db=openHelper.getWritableDatabase();
this.insertStmt=this.db.compileStatement(INSERT);
}
catch(Exception e)
{
System.out.println(e);
}
}
Can anyone help me how to solve this problem.
Thanks
DROP TABLE seems a pretty extreme way of trying to stop duplicate values. It's a bit hard to follow the code you've posted, but the normal way of stopping duplicates is to add a unique index on the appropriate column(s). Have you tried that yet ? E.g. something like
CREATE UNIQUE INDEX idx_keytype ON tableName (key_type)
What does your schema look like? If you don't want duplicate rows and you know a certain column will be unique use the "UNIQUE" specifier on it. If what you really want is for the row to be replaced you have to use the databaseHelper command "replace" ie. dbHelper.replace(...);

Categories

Resources