Develop Reference

Android Room migration with updated pre-populated database

I'm having a little bit of a headache with Room and migration with a pre-populated database.
EXPLANATION
I'm currently using Room and a pre-populated database. With the first version (version 1) the database loads fine and everything works correctly.
The problem is that at this point I'm in need to add three new tables to the database with data in them. So I started updating the version 1 database that I had and created all the tables and rows with data in them that I needed.
The first time that I tried, I pushed directly the new .sqlite database into the assets folder and changed the version number to 2, but of course, Room gave the error that it needs to know how to handle migration 1_2 so I added a migration rule
.addMigrations(new Migration(1,2) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
}
...
Thinking that maybe if I tell Room to create these tables it will then connect to the new database in assets and populate the tables accordingly.
But that of course didn't work and by looking at the database inspector it was clear that the tables were present but they were empty.
A SOLUTION I DON'T REALLY LIKE
After tinkering around for a little bit in the end what I found that worked is to have a copy of the updated database, navigate in it (I'm currently using DB Browser for SQLite), get the SQL query for the newly populated rows, and format a database.execSQL statement accordingly to insert the new data into the tables:
.addMigrations(new Migration(1,2) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
database.execSQL("CREATE TABLE ...);
database.execSQL("INSERT INTO ...");
database.execSQL("INSERT INTO ...");
database.execSQL("INSERT INTO ...");
database.execSQL("INSERT INTO ...");
}
...
I find this an "acceptable" solution for cases in which we're working on rows that contain small data but in my case, I'm handling rows with very long strings and this creates a series of inconveniences:
the SQL statements that are extracted from the database data need to be well formatted: ' symbols need to be handled as well as " that could be present in the long strings as well as line breaks;
consistency between the database and the insert statements for the rows needs to be kept;
QUESTION
Mind that fallbackToDestructiveMigration() is not an acceptable option since the database in Version 1 has user-created data in it and it needs to be kept between migrations.
So, is there a solution that allows me to directly push the new .sqlite database into assets without writing tons and tons of INSERT and CREATE TABLE statements and let Room handle the new data within it automatically and also while keeping the old tables data?
Thank you for your time!

Perhaps consider
Place the new database into the assets folder suitable for a new install of the App so the createFromAsset would copy this version 2 database for a new install.
In the migration copy the asset to a the database folder with a different database name.
in the migration create the new tables.
still in the migration, for each new table, extract all of the data from the differently named new database then use the Cursor to insert the data into the existing database.
still in the migration, close the differently name database and delete the file.
Here's the migration code for something along those lines (no schema change, just new pre-populated data) and it's Kotlin not Java from a recent answer:-
val migration1_2 = object: Migration(1,2) {
val assetFileName = "appdatabase.db"
val tempDBName = "temp_" + assetFileName
val bufferSize = 1024 * 4
#SuppressLint("Range")
override fun migrate(database: SupportSQLiteDatabase) {
val asset = contextPassed?.assets?.open(assetFileName) /* Get the asset as an InputStream */
val tempDBPath = contextPassed?.getDatabasePath(tempDBName) /* Deduce the file name to copy the database to */
val os = tempDBPath?.outputStream() /* and get an OutputStream for the new version database */
/* Copy the asset to the respective file (OutputStream) */
val buffer = ByteArray(bufferSize)
while (asset!!.read(buffer,0,bufferSize) > 0) {
os!!.write(buffer)
}
/* Flush and close the newly created database file */
os!!.flush()
os.close()
/* Close the asset inputStream */
asset.close()
/* Open the new database */
val version2db = SQLiteDatabase.openDatabase(tempDBPath.path,null,SQLiteDatabase.OPEN_READONLY)
/* Grab all of the supplied rows */
val v2csr = version2db.rawQuery("SELECT * FROM user WHERE userId < ${User.USER_DEMARCATION}",null)
/* Insert into the actual database ignoring duplicates (by userId) */
while (v2csr.moveToNext()) {
database.execSQL("INSERT OR IGNORE INTO user VALUES(${v2csr.getLong(v2csr.getColumnIndex("userId"))},'${v2csr.getString(v2csr.getColumnIndex("userName"))}')",)
}
/* close cursor and the newly created database */
v2csr.close()
version2db.close()
tempDBPath.delete() /* Delete the temporary database file */
}
}
Note when testing the above code. I initially tried ATTACH'ing the new (temp) database. This worked and copied the data BUT either the ATTACH or DETACH (or both) prematurely ended the transaction that the migration runs in, resulting in Room failing to then open the database and a resultant exception.
If this wasn't so then with the new database attached a simple INSERT INTO main.the_table SELECT * FROM the_attached_schema_name.the_table; could have been used instead of using the cursor as the go-between.
without writing tons and tons of INSERT and CREATE TABLE statements
INSERT dealt with above.
The CREATE SQL could, in a similar way, be extracted from the new asset database, by using:-
`SELECT name,sql FROM sqlite_master WHERE type = 'table' AND name in (a_csv_of_the_table_names (enclosed in single quotes))`
e.g. SELECT name,sql FROM sqlite_master WHERE type = 'table' AND name IN ('viewLog','message');;
results in (for an arbitrary database used to demonstrate) :-
name is the name of the table and sql then sql that was used to create the tables.
alternately the SQL to create the tables can be found after compiling in the generated java (visible from the Android View) in the class that has the same name as the class annotated with #Database but suffixed with _Impl. There will be a method called createAlltables which has the SQL to create all the tables (and other items) e.g. (again just an arbitrary example) :-
note the red stricken-through lines are for the room_master table, ROOM creates this and it is not required in the asset (it's what room uses to check to see if the schema has been changed)
Working Example
Version 1 (preparing for the migration to Version 2)
The following is a working example. Under Version 1 a single table named original (entity OriginalEnity) with data (5 rows) via a pre-populated database is used, a row is then added to reflect user suplied/input dat. When the App runs the contents of the table are extracted and written to the log :-
D/DBINFOoriginal: Name is name1 ID is 1 - DB Version is 1
D/DBINFOoriginal: Name is name2 ID is 2 - DB Version is 1
D/DBINFOoriginal: Name is name3 ID is 3 - DB Version is 1
D/DBINFOoriginal: Name is name4 ID is 4 - DB Version is 1
D/DBINFOoriginal: Name is name5 ID is 5 - DB Version is 1
D/DBINFOoriginal: Name is App User Data ID is 6 - DB Version is 1
Database Inspector showing :-
Version 2
The 3 new Entities/Tables added (newEntity1,2 and 3 table names new1, new2 and new3 respectively) same basic structure.
After creating the Entities and compiling the SQL, as per the createAlltables method in the java generated was extracted from the TheDatabase_Impl class (including the 3 additional indexes) :-
This SQL was then used in the SQLite tool to create the new tables and populate them with some data :-
/* FOR VERSION 2 */
/* Create statments copied from TheDatabase_Impl */
DROP TABLE IF EXISTS new1;
DROP TABLE IF EXISTS new2;
DROP TABLE IF EXISTS new3;
CREATE TABLE IF NOT EXISTS `new1` (`new1_id` INTEGER, `new1_name` TEXT, PRIMARY KEY(`new1_id`));
CREATE INDEX IF NOT EXISTS `index_new1_new1_name` ON `new1` (`new1_name`);
CREATE TABLE IF NOT EXISTS `new2` (`new2_id` INTEGER, `new2_name` TEXT, PRIMARY KEY(`new2_id`));
CREATE INDEX IF NOT EXISTS `index_new2_new2_name` ON `new2` (`new2_name`);
CREATE TABLE IF NOT EXISTS `new3` (`new3_id` INTEGER, `new3_name` TEXT, PRIMARY KEY(`new3_id`));
CREATE INDEX IF NOT EXISTS `index_new3_new3_name` ON `new3` (`new3_name`);
INSERT OR IGNORE INTO new1 (new1_name) VALUES ('new1_name1'),('new1_name2');
INSERT OR IGNORE INTO new2 (new2_name) VALUES ('new2_name1'),('new2_name2');
INSERT OR IGNORE INTO new3 (new3_name) VALUES ('new3_name1'),('new3_name2');
The database saved and copied into the assets folder (original renamed) :-
Then the Migration code (full database helper), which :-
is driven simply by a String[] of the table names
copies the asset (new database) an opens it via the SQLite API
creates the tables, indexes and triggers according to the asset (must match the schema generated by room (hence copying sql from generated java previously))
it does this by extracting the respective SQL from the sqlite_master table
populates the newly created Room tables by extracting the data from the asset database into a Cursor and then inserting into the Room database (not the most efficient way BUT Room runs the Migration in a transaction)
is:-
#Database(entities = {
OriginalEntity.class, /* on it's own for V1 */
/* ADDED NEW TABLES FOR V2 */NewEntity1.class,NewEntity2.class,NewEntity3.class
},
version = TheDatabase.DATABASE_VERSION,
exportSchema = false
)
abstract class TheDatabase extends RoomDatabase {
public static final String DATABASE_NAME = "thedatabase.db";
public static final int DATABASE_VERSION = 2; //<<<<<<<<<< changed */
abstract AllDao getAllDao();
private static volatile TheDatabase instance = null;
private static Context currentContext;
public static TheDatabase getInstance(Context context) {
currentContext = context;
if (instance == null) {
instance = Room.databaseBuilder(context, TheDatabase.class, DATABASE_NAME)
.allowMainThreadQueries() /* for convenience run on main thread */
.createFromAsset(DATABASE_NAME)
.addMigrations(migration1_2)
.build();
}
return instance;
}
static Migration migration1_2 = new Migration(1, 2) {
#Override
public void migrate(#NonNull SupportSQLiteDatabase database) {
/* Copy the asset into the database folder (with different name) */
File assetDBFile = getNewAssetDatabase(currentContext,DATABASE_NAME);
/* Open the assetdatabase */
SQLiteDatabase assetDB = SQLiteDatabase.openDatabase(assetDBFile.getPath(),null,SQLiteDatabase.OPEN_READWRITE);
/* Build (create and populate) the new ROOM tables and indexes from the asset database */
buildNewTables(
new String[]{
NewEntity1.TABLE_NAME,
NewEntity2.TABLE_NAME,
NewEntity3.TABLE_NAME},
database /* ROOM DATABASE */,
assetDB /* The copied and opened asset database as an SQliteDatabase */
);
/* done with the asset database */
assetDB.close();
assetDBFile.delete();
}
};
private static void buildNewTables(String[] tablesToBuild, SupportSQLiteDatabase actualDB, SQLiteDatabase assetDB) {
StringBuilder args = new StringBuilder();
boolean afterFirst = false;
for (String tableName: tablesToBuild) {
if (afterFirst) {
args.append(",");
}
afterFirst = true;
args.append("'").append(tableName).append("'");
}
/* Get SQL for anything related to the table (table, index, trigger) to the tables and build it */
/* !!!!WARNING!!!! NOT TESTED VIEWS */
/* !!!!WARNING!!!! may not cope with Foreign keys as conflicts could occur */
Cursor csr = assetDB.query(
"sqlite_master",
new String[]{"name","sql", "CASE WHEN type = 'table' THEN 1 WHEN type = 'index' THEN 3 ELSE 2 END AS sort"},
"tbl_name IN (" + args.toString() + ")",
null,
null,null, "sort"
);
while (csr.moveToNext()) {
Log.d("CREATEINFO","executing SQL:- " + csr.getString(csr.getColumnIndex("sql")));
actualDB.execSQL(csr.getString(csr.getColumnIndex("sql")));
}
/* Populate the tables */
/* !!!!WARNING!!!! may not cope with Foreign keys as conflicts could occur */
/* no set order for the tables so a child table may not be loaded before it's parent(s) */
ContentValues cv = new ContentValues();
for (String tableName: tablesToBuild) {
csr = assetDB.query(tableName,null,null,null,null,null,null);
while (csr.moveToNext()) {
cv.clear();
for (String columnName: csr.getColumnNames()) {
cv.put(columnName,csr.getString(csr.getColumnIndex(columnName)));
actualDB.insert(tableName, OnConflictStrategy.IGNORE,cv);
}
}
}
csr.close();
}
private static File getNewAssetDatabase(Context context, String assetDatabaseFileName) {
String tempDBPrefix = "temp_";
int bufferSize = 1024 * 8;
byte[] buffer = new byte[bufferSize];
File assetDatabase = context.getDatabasePath(tempDBPrefix+DATABASE_NAME);
InputStream assetIn;
OutputStream assetOut;
/* Delete the AssetDatabase (temp DB) if it exists */
if (assetDatabase.exists()) {
assetDatabase.delete(); /* should not exist but just in case */
}
/* Just in case the databases folder (data/data/packagename/databases)
doesn't exist create it
This should never be the case as Room DB uses it
*/
if (!assetDatabase.getParentFile().exists()) {
assetDatabase.mkdirs();
}
try {
assetIn = context.getAssets().open(assetDatabaseFileName);
assetOut = new FileOutputStream(assetDatabase);
while(assetIn.read(buffer) > 0) {
assetOut.write(buffer);
}
assetOut.flush();
assetOut.close();
assetIn.close();
} catch (IOException e) {
e.printStackTrace();
throw new RuntimeException("Error retrieving Asset Database from asset " + assetDatabaseFileName);
}
return assetDatabase;
}
}
The code in the Activity is :-
public class MainActivity extends AppCompatActivity {
TheDatabase db;
AllDao dao;
private static final String TAG = "DBINFO";
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
/* Original */
db = TheDatabase.getInstance(this);
dao = db.getAllDao();
OriginalEntity newOE = new OriginalEntity();
newOE.name = "App User Data";
dao.insert(newOE);
for(OriginalEntity o: dao.getAll()) {
Log.d(TAG+OriginalEntity.TABLE_NAME,"Name is " + o.name + " ID is " + o.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
/* Added for V2 */
for (NewEntity1 n: dao.getAllNewEntity1s()) {
Log.d(TAG+NewEntity1.TABLE_NAME,"Names is " + n.name + " ID is " + n.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
for (NewEntity2 n: dao.getAllNewEntity2s()) {
Log.d(TAG+NewEntity2.TABLE_NAME,"Names is " + n.name + " ID is " + n.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
for (NewEntity3 n: dao.getAllNewEntity3s()) {
Log.d(TAG+NewEntity3.TABLE_NAME,"Names is " + n.name + " ID is " + n.id + " - DB Version is " + TheDatabase.DATABASE_VERSION);
}
}
}
see Version 1 section and comments in the code for V1 run.
The resultant output to the log for the V2 run (initial) is
:-
2021-10-11 13:02:50.939 D/CREATEINFO: executing SQL:- CREATE TABLE `new1` (`new1_id` INTEGER, `new1_name` TEXT, PRIMARY KEY(`new1_id`))
2021-10-11 13:02:50.941 D/CREATEINFO: executing SQL:- CREATE TABLE `new2` (`new2_id` INTEGER, `new2_name` TEXT, PRIMARY KEY(`new2_id`))
2021-10-11 13:02:50.942 D/CREATEINFO: executing SQL:- CREATE TABLE `new3` (`new3_id` INTEGER, `new3_name` TEXT, PRIMARY KEY(`new3_id`))
2021-10-11 13:02:50.942 D/CREATEINFO: executing SQL:- CREATE INDEX `index_new1_new1_name` ON `new1` (`new1_name`)
2021-10-11 13:02:50.943 D/CREATEINFO: executing SQL:- CREATE INDEX `index_new2_new2_name` ON `new2` (`new2_name`)
2021-10-11 13:02:50.944 D/CREATEINFO: executing SQL:- CREATE INDEX `index_new3_new3_name` ON `new3` (`new3_name`)
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name2 ID is 2 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name3 ID is 3 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name4 ID is 4 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is name5 ID is 5 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is App User Data ID is 6 - DB Version is 2
2021-10-11 13:02:51.006 D/DBINFOoriginal: Name is App User Data ID is 7 - DB Version is 2
2021-10-11 13:02:51.010 D/DBINFOnew1: Names is new1_name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.010 D/DBINFOnew1: Names is new1_name2 ID is 2 - DB Version is 2
2021-10-11 13:02:51.012 D/DBINFOnew2: Names is new2_name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.012 D/DBINFOnew2: Names is new2_name2 ID is 2 - DB Version is 2
2021-10-11 13:02:51.013 D/DBINFOnew3: Names is new3_name1 ID is 1 - DB Version is 2
2021-10-11 13:02:51.013 D/DBINFOnew3: Names is new3_name2 ID is 2 - DB Version is 2
Note that the user data has been retained (1st App User Data ...., 2nd is added when the activity is run).
The Dao (AllDao) is :-
#Dao
abstract class AllDao {
/* Original Version 1 Dao's */
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(OriginalEntity originalEntity);
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long[] insert(OriginalEntity ... originalEntities);
#Query("SELECT * FROM original")
abstract List<OriginalEntity> getAll();
/* New Version 2 Dao's */
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(NewEntity1 newEntity1);
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(NewEntity2 newEntity2);
#Insert(onConflict = OnConflictStrategy.IGNORE)
abstract long insert(NewEntity3 newEntity3);
#Query("SELECT * FROM " + NewEntity1.TABLE_NAME)
abstract List<NewEntity1> getAllNewEntity1s();
#Query("SELECT * FROM " + NewEntity2.TABLE_NAME)
abstract List<NewEntity2> getAllNewEntity2s();
#Query("SELECT * FROM " + NewEntity3.TABLE_NAME)
abstract List<NewEntity3> getAllNewEntity3s();
}
The Entities are :-
#Entity(tableName = OriginalEntity.TABLE_NAME)
class OriginalEntity {
public static final String TABLE_NAME = "original";
public static final String COL_ID = TABLE_NAME +"_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COL_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
and for V2 :-
#Entity(tableName = NewEntity1.TABLE_NAME)
class NewEntity1 {
public static final String TABLE_NAME = "new1";
public static final String COl_ID = TABLE_NAME + "_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COl_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
and :-
#Entity(tableName = NewEntity2.TABLE_NAME)
class NewEntity2 {
public static final String TABLE_NAME = "new2";
public static final String COl_ID = TABLE_NAME + "_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COl_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
and :-
#Entity(tableName = NewEntity3.TABLE_NAME)
class NewEntity3 {
public static final String TABLE_NAME = "new3";
public static final String COl_ID = TABLE_NAME + "_id";
public static final String COL_NAME = TABLE_NAME + "_name";
#PrimaryKey
#ColumnInfo(name = COl_ID)
Long id = null;
#ColumnInfo(name = COL_NAME, index = true)
String name;
}
Finally Test new App install (i.e. no Migration but created from asset)
When run the output to the log is (no user supplied/input data) :-
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name2 ID is 2 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name3 ID is 3 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name4 ID is 4 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is name5 ID is 5 - DB Version is 2
2021-10-11 13:42:48.272 D/DBINFOoriginal: Name is App User Data ID is 6 - DB Version is 2
2021-10-11 13:42:48.275 D/DBINFOnew1: Names is new1_name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.275 D/DBINFOnew1: Names is new1_name2 ID is 2 - DB Version is 2
2021-10-11 13:42:48.276 D/DBINFOnew2: Names is new2_name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.276 D/DBINFOnew2: Names is new2_name2 ID is 2 - DB Version is 2
2021-10-11 13:42:48.277 D/DBINFOnew3: Names is new3_name1 ID is 1 - DB Version is 2
2021-10-11 13:42:48.277 D/DBINFOnew3: Names is new3_name2 ID is 2 - DB Version is 2
Note
Room encloses names of items (tables, columns) in grave accents's, This makes invalid column names valid e.g 1 not enclosed is invalid 1 enclosed is valid. Use of otherwise invalid names may, although I suspect not, cause issues (I haven't tested this aspect). SQLite itself strips the grave accents when storing the name e.g :-
CREATE TABLE IF NOT EXISTS `testit` (`1`);
SELECT * FROM sqlite_master WHERE name = 'testit';
SELECT * FROM testit;
results in :-
i.e. the grave accents are kept when the SQL is stored, hence the generated CREATE's are safe.
and :-
i.e. the grave accents have been stripped and the column is named just 1, which may cause an issue (but likely not) when traversing the columns in the cursor.

About SQLite 3, is it required to read after write?

In an Android application, concerning SQLITE3,
Is it required to read after write or we can just count on a successful transaction ?
Update to show the function to insert data:
// Insert a post into the database
public void addM0(Store_M0_DataClass mStore_M0_DataClass)
{
// Create and/or open the database for writing
SQLiteDatabase db = getWritableDatabase();
// It's a good idea to wrap our insert in a transaction. This helps with performance and ensures
// consistency of the database.
db.beginTransaction();
try
{
// The user might already exist in the database (i.e. the same user created multiple posts).
ContentValues values = new ContentValues();
values.put(KEY_M0_ID, mStore_M0_DataClass.getM0_ID());
values.put(KEY_M0_IMAGE1, mStore_M0_DataClass.getM0_Image1());
values.put(KEY_M0_IMAGE2, mStore_M0_DataClass.getM0_Image2());
values.put(KEY_M0_ENABLE, mStore_M0_DataClass.getM0_Enable());
db.insertOrThrow(TABLE_M0, null, values);
db.setTransactionSuccessful();
}
catch (Exception e)
{
}
finally
{
db.endTransaction();
}
}

An insert or any single action is automatically enclosed in a transaction, as such there is no benefit in wrapping it in a transaction. However, if you were doing multiple inserts in a loop then beginning a transaction prior to the loop, setting it as successful, if successful, after the loop, and ending the transaction after the loop it would have a benefit.
There is also little need to use insertOrThrow unless you want to trap an exception, as the insert method is effectively INSERT OR IGNORE and the result is the rowid of the inserted row, which will be 1 or greater (in most normal use cases).
rowid is a special, normally hidden column, frequently aliased using by using a column defined using specifically column_name INTEGER PRIMARY KEY or column_name INTEGER PRIMARY KEY AUTOINCREMENT Rowid Tables
(noting that AUTOINCREMENT is generally not required and has overheads) SQLite Autoincrement
As such using :-
public long addM0(Store_M0_DataClass mStore_M0_DataClass)
{
// Create and/or open the database for writing
SQLiteDatabase db = getWritableDatabase();
ContentValues values = new ContentValues();
values.put(KEY_M0_ID, mStore_M0_DataClass.getM0_ID());
values.put(KEY_M0_IMAGE1, mStore_M0_DataClass.getM0_Image1());
values.put(KEY_M0_IMAGE2, mStore_M0_DataClass.getM0_Image2());
values.put(KEY_M0_ENABLE, mStore_M0_DataClass.getM0_Enable());
return db.insert(TABLE_M0, null, values);
}
However, if you wanted to store multiple Store_M0_DataClass objects then you could have :-
public int addManyMo(Store_M0_DataClass[] mStore_M0_DataClasses) {
int rv = 0;
SQLiteDatabase db = this.getWritableDatabase();
db.beginTransaction();// <<<<<<<<<< wrap all inserts in a transaction
for (Store_M0_DataClass m0: mStore_M0_DataClasses) {
if(addM0(m0) > 0) {
rv++;
}
}
// Assumes OK if any inserts have worked, not if none have worked
if (rv > 0) {
db.setTransactionSuccessful();
}
db.endTransaction();
return rv; // Retruns the number of rows inserted
}
As for :-
Is it required to read after write or we can just count on a
successful transaction ?
A transaction being successful commits what was in the transaction. Comitting is effectively writing the data to disk so there is no need to do anything after a commit.
with WAL (Write Ahead logging) the data is written to the WAL file rather than the database (and also the shm file). However when accessing the database if the WAL has data it is effectively read as being part of the database. The WAL data is written to the database when the WAL is checkpionted.

Creating a SQLite database for the game Tic Tac toe and connecting it to the main Activity [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I want to create a SQLite database for the game Tic Tac Toe on Android Studio that has the names and score of the players. Every time a user win, a point is added on the scoreboard.
How would I do this?

Stage 1 - Database Design
First design the database, you have identified Data as User name, and score, assuming you don't want historical data then a single table would suffice.
So design would be a table, perhaps called scoreboard, with columns :-
username
score and
To perhaps aid future changes a column name id that uniquely identifies a user (e.g. say you had two Toms or even two Tom smiths), this identifier (which is generally available) will be an alias of rowid. As Cursor Adapters require the id to be named _id then that will be used.
As such you could have a table that is created using the following SQL :-
CREATE TABLE IF NOT EXISTS scoreboard
(
_id INTEGER PRIMARY KEY,
username TEXT,
score INTEGER
);
Stage 2 - Creating the Database
When getting started with SQLite for Android it is probably best to utilise a subclass of SQLiteOpenHelper as what many refer to as the DBHelper.
So create a class say DBHelper.java which extends the SQLiteOpenHelper class.
Note you must include overrides for the onCreate method and the onUpgrade method.
If using Android Studio when adding a new class;
input, DBHelper in the **Name* field,
type SQLiteOpenHelper in the Superclass field (by the time you've typed SQL you will see SQLiteOpenHelper double click is) and
then tick/check the Show Select Overrides Dialog.
Click OK
You will presented with the Overrides Dialog select (Ctrl + CLick) the following 3 (SQLiteOpenHelper(context"Context,name:String,factory:CursorFactory,version:int) will be selected) :-
SQLiteOpenHelper(context"Context,name:String,factory:CursorFactory,version:int)
onCreate(.....
onUpgrade(.....
Then click OK.
You will then have :-
public class DBHelper extends SQLiteOpenHelper {
public DBHelper(Context context, String name, SQLiteDatabase.CursorFactory factory, int version) {
super(context, name, factory, version);
}
#Override
public void onCreate(SQLiteDatabase sqLiteDatabase) {
}
#Override
public void onUpgrade(SQLiteDatabase sqLiteDatabase, int i, int i1) {
}
}
Define some constants
Between the class and the constructor add some constants so you have a single source for names of tables/columns etc e.g. :-
public static final String DBNAME = "tictactoe.db"; // Database name
public static final int DBVERSION = 1; // Database version #
public static final String TB_SCOREBOARD = "scoreboard"; // table name
public static final String COL_SCOREBOARD_ID = BaseColumns._ID; // use default id column name
public static final String COL_SCOREBOARD_USERNAME = "username";
public static final String COL_SCOREBOARD_SCORE = "score";
Ready to create the Table
The onCreate method will be called when you try to open the database (and the database is actually created). Generally it is here that you create the tables.
Note one of the more common issues newcomers have is that they think that onCreate runs every time a database is opened. It is not it only runs once when the database is first created.
as such any changes (say you add a new column) WILL NOT BE MADE if the databade still exists (easiest solution when developing an App is to delete the App's Data or uninstall the App and rerun the App).
So in the onCreate method :-
create a String of the SQL to create the table (i.e. the CREATE IF NOT EXISTS.... previously shown). However, do so utilising the CONSTANTS (see below).
call the SQLiteDatabase execSQL method to run the SQL.
Alter the constructor's signature (make it easier to call).
As the database name and version are known (they are constant) and that a cursor factory needn't be used (null will signify this) the super call in the constructor can be replaced with :-
super(context, DBNAME, null, DBVERSION);
Therefore the signature for the DBHelper class can be changed to :-
public DBHelper(Context context) {
super(context, DBNAME, null, DBVERSION);
}
So the DBHelper class in full (at present) can be :-
public class DBHelper extends SQLiteOpenHelper {
public static final String DBNAME = "tictactoe.db"; // Database name
public static final int DBVERSION = 1; // Database version #
public static final String TB_SCOREBOARD = "scoreboard"; // table name
public static final String COL_SCOREBOARD_ID = BaseColumns._ID; // use default id column name
public static final String COL_SCOREBOARD_USERNAME = "username";
public static final String COL_SCOREBOARD_SCORE = "score";
public DBHelper(Context context) {
super(context, DBNAME, null, DBVERSION);
}
#Override
public void onCreate(SQLiteDatabase sqLiteDatabase) {
String crtsql = "CREATE TABLE IF NOT EXISTS " +
TB_SCOREBOARD + // The table name
"(" +
COL_SCOREBOARD_ID + " INTEGER PRIMARY KEY," + //The _id column
COL_SCOREBOARD_USERNAME + " TEXT, " + // username column
COL_SCOREBOARD_SCORE + " INTEGER" + // score column (no trailing comma as last)
")" ;
sqLiteDatabase.execSQL(crtsql);
}
#Override
public void onUpgrade(SQLiteDatabase sqLiteDatabase, int i, int i1) {
}
}
Stage 3 - TEST what has been so far
Believe it or not the above is sufficient to create the database and the table and hence the columns within the table (not to actually add any data or anything useful but at least).
Typically you would use the database in an activity. For the purposes of this testing a basic MainActivity will be used.
It's actually very simple we just create a DBHelper instance (passing the Context).
BUT doing so won't create a database it's only when either the getWritableDatabase or getReadableDatabase methods are called that an attempt is made to open or create the database. So a second line will do this (could be done in one line) :-
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
DBHelper mDBHlpr = new DBHelper(this); // get DBHelper instance
mDBHlpr.getWritableDatabase(); // force an open (wouldn't normally do this)
}
}
Note you wouldn't normally have a call to getWritableDatabase as will be seen each method to access the table(s) will do this, so the very first time one of these would result in the onCreate method being called.
Note there is much confusion with getReadableDatabase, in short it doesn't make it so you can't update/change data unless the database can only be read.
i.e. getReadabledatabase will get a writable database unless in the very rare scenario the database can only be read (when getWritableDatabase would fail with an unable to open database error).
VERY RARELY is there any use coding getReadableDatabase.
Do the above and Run the App - nothing much will happen, but hopefully it shouldn't crash.
Now if you can (depends on emulator (I use genymotion which gives you root access)) use Android Studion's Device Explorer* to look at **/data/data/<your_package_name>/databases/ and hopefully you will see :-
package and actual database are highlighted.
database is just a file (can be copied and opened in other SQLite tools (can even be copied to android (emulator/device permitting))).
journal is SQLite's file that records what's being done and in cases of errors allows data to be rolled back (i.e. just accept it exists).
A believe that size should be 16K (depends upon data and structure of the database). It shouldn't be 0 though.
If you can't use Device Explorer then you can go into settings and check the App's data (if you have other uses of App data then check subtract this (check before running)), it should be 0 (after subtracting other data). in which case that's an indication that the database exists.
Stage 4 - Adding and Manipulating Data
At this stage a database exists with a table but no data itself exists. So a means of adding data (inserting rows) (a table has rows a row consisting of the columns as per the definition of the table).
It's no use adding data unless that data can be accessed so a means of extracting the data (querying) is required.
As a method of changing (updating) the score is required a means of doing this is required.
So what is needed now are 3 things :-
an insertRow method
a getAllData method (say to list scoreboard)
a updateScore method (which will adjust the score according to a number)
Typically such methods are added to the Database Helper (so they will be here)
The insertRow method
When inserting a row we need to add the name and the score (we could have defined the score column as score INTEGER DEFAULT 0 and then just the name would be required).
Although you don't know it yet id's can be very useful so the method will return the id of the newly inserted row, which due to using _id INTEGER PRIMARY KEY (and specifically this (or INTEGER PRIMARY KEY AUTOINCREMENT). will be generated automatically (i.e. the _id column is an alias of the very special but normally hidden rowid column (see link below for more info on rowid)).
the latter, AUTOINCREMENT, is very rarely needed but is seen very often more here SQLite Autoincrement, this also explains rowid)
So a method such as the following could be added :-
public long insertRow(String username, int initial_score) {
// SQL equivalent of :-
// INSERT INTO scoreboard (username,score) VALUES('the user name',0)
ContentValues cv = new ContentValues(); // Used by convenience method for column/value pairs
cv.put(COL_SCOREBOARD_SCORE,username); // The username to be added
cv.put(COL_SCOREBOARD_SCORE,initial_score); // The score to be added
SQLiteDatabase db = this.getWritableDatabase(); // Get a SQLiteDatabase instance
return db.insert(TB_SCOREBOARD,null,cv); // Insert it using conveniece method
/*
Note if row cannot be inserted then return will be -1
If inserted the id will be returned,
first ever insert will be 1,
then likely 2
then likely 3
NEVER ASSUME 1,2,3.......... though
ALWAYS ASSUME IT WILL BE A UNIQUE VALUE
i.e. NEVER CODE SPECIFIC ID's
*/
}
You may wish to read insert
The Activity could use this using for example :-
mDBHlpr.insertRow("Rumplestiltskin the 3rd",10000000); // The winner :)
mDBHlpr.insertRow("Fred Blogs",0); // New user would normally start with score 0
Adds 2 rows first with high score, 2nd as you would probably add a new user
The getAllData method
With Android you extract data into what's called a Cursor, which is like a spreadsheet it has rows and columns (columns as you specify so they needn't be all the columns, can also be other columns (e.g. derived/calculated or from other tables).
You create a Cursor (at least a normal one) by querying the table or tables in the database (note this doesn't cover all aspects). So use will be made of the convenience query method (well 1 of the 4) using :-
public Cursor getAlldata() {
// The columns to retrieve
String[] columns = new String[]{
COL_SCOREBOARD_ID,
COL_SCOREBOARD_USERNAME,
COL_SCOREBOARD_SCORE
};
// NOTE normally for all columns you would use the above but
// instead pass null as the 2nd parameter to the query method
return this.getWritableDatabase().query(
TB_SCOREBOARD,
columns,
null,
null,
null,
null,
null
);
}
You may wish to read query
This could be used in the Activity along the lines of :-
Cursor csr = mDBHlpr.getAlldata();
csr.close(); //YOU SHOULD ALWAYS CLOSE A CURSOR WHEN DONE WITH IT
The updateScore method
Without getting too complex and sticking to convenience methods the process of updating a score will :-
get the old score (according to id)
update the new score by adding the new score (if it's minus then reducing the score)
As such 2 parameters are required the id and the amount to adjust the score by.
-Id's should be long (you will see many uses of int but long copes with all possible id's).
-adjustment will be integer (long if very high scores are expected)
A diversion for getScoreById method
As getting a user's score may be useful another method will be created to do this. This also makes use of a Cursor that selects specific data rather than all via an SQL WHERE clause. So a method getScoreById will also be created. This will return the current score as an int and is passed a long as the id.
This could be :-
public int getScoreById(long id) {
int rv = -1; // just in case the id doesn't exist return -1 so invalid adjustment can be detected
String[] columns = new String[]{COL_SCOREBOARD_SCORE}; // only want the score
String whereclause = COL_SCOREBOARD_ID + "=?"; // will be WHERE _id=? (? replaced by respective whereargs element)
String[] whereargs = new String[]{String.valueOf(id)}; // ? will be replaced with id
Cursor csr = this.getWritableDatabase().query(
TB_SCOREBOARD,
columns,
whereclause,
whereargs,
null,
null,
null
);
if (csr.moveToFirst()) {
//rv = csr.getInt(0); // Hard coded column offsets bad so :-
rv = csr.getInt(csr.getColumnIndex(COL_SCOREBOARD_SCORE));
}
csr.close(); // Done with the cursor so close it
return rv; // return the current score
}
Back to the upDateScore method
Now that the score can be retrieved by the id via the getScoreById method then the the updateScore method could be :-
public boolean updateScore(long id, int adjustment) {
int newscore = getScoreById(id) + adjustment; // get the new score
// Check that the new score is valid (i.e. greater than 0)
// If it's invalid then don't do update by returning false but after
// issuing a message to the log (for development should be removed for production)
if (newscore < 0) {
Log.d("INVALIDSCORE",
"An invalid new score (less than 0) was returned. Update cancelled.");
return false;
}
// Prepare to use the update convenience method
String whereclause = COL_SCOREBOARD_ID + "=?";
String[] whereargs = new String[]{String.valueOf(id)};
ContentValues cv = new ContentValues();
cv.put(COL_SCOREBOARD_SCORE,newscore);
SQLiteDatabase db = this.getWritableDatabase();
// WARNING without a WHERE clause update would update ALL ROWS
// update returns number of rows updated as an int, so if this is
// greater than 0 true is returned else false.
return db.update(TB_SCOREBOARD,cv,whereclause,whereargs) > 0;
}
So the whole DBHelper class could be :-
public class DBHelper extends SQLiteOpenHelper {
public static final String DBNAME = "tictactoe.db"; // Database name
public static final int DBVERSION = 1; // Database version #
public static final String TB_SCOREBOARD = "scoreboard"; // table name
public static final String COL_SCOREBOARD_ID = BaseColumns._ID; // use default id column name
public static final String COL_SCOREBOARD_USERNAME = "username";
public static final String COL_SCOREBOARD_SCORE = "score";
public DBHelper(Context context) {
super(context, DBNAME, null, DBVERSION);
}
#Override
public void onCreate(SQLiteDatabase sqLiteDatabase) {
String crtsql = "CREATE TABLE IF NOT EXISTS " +
TB_SCOREBOARD + // The table name
"(" +
COL_SCOREBOARD_ID + " INTEGER PRIMARY KEY," + //The _id column
COL_SCOREBOARD_USERNAME + " TEXT, " + // username column
COL_SCOREBOARD_SCORE + " INTEGER" + // score column (no trailing comma as last)
")" ;
sqLiteDatabase.execSQL(crtsql);
}
#Override
public void onUpgrade(SQLiteDatabase sqLiteDatabase, int i, int i1) {
}
public long insertRow(String username, int initial_score) {
// SQL equivalent of :-
// INSERT INTO scoreboard (username,score) VALUES('the user name',0)
ContentValues cv = new ContentValues(); // Used by convenience method for column/value pairs
cv.put(COL_SCOREBOARD_SCORE,username); // The username to be added
cv.put(COL_SCOREBOARD_SCORE,initial_score); // The score to be added
SQLiteDatabase db = this.getWritableDatabase(); // Get a SQLiteDatabase instance
return db.insert(TB_SCOREBOARD,null,cv); // Insert it
/*
Note if row cannot be inserted then return will be -1
If insert the id will be returned,
first ever insert will be 1,
then likely 2
then likely 3
NEVER ASSUME 1,2,3.......... though
ALWAYS ASSUME IT WILL BE A UNIQUE VALUE
i.e. NEVER CODE SPECIFIC ID's
*/
}
public Cursor getAlldata() {
// The columns to retrieve
String[] columns = new String[]{
COL_SCOREBOARD_ID,
COL_SCOREBOARD_USERNAME,
COL_SCOREBOARD_SCORE
};
// NOTE normally for all columns you would use the above but
// instead pass null as the 2nd parameter to the query method
return this.getWritableDatabase().query(
TB_SCOREBOARD,
columns,
null,
null,
null,
null,
null
);
}
public boolean updateScore(long id, int adjustment) {
int newscore = getScoreById(id) + adjustment; // get the new score
// Check that the new score is valid (i.e. greater than 0)
// If it's invalid then don't do update by returning false but after
// issuing a message to the log (for development should be removed for production)
if (newscore < 0) {
Log.d("INVALIDSCORE",
"An invalid new score (less than 0) was returned. Update cancelled.");
return false;
}
// Prepare to use the update convenience method
String whereclause = COL_SCOREBOARD_ID + "=?";
String[] whereargs = new String[]{String.valueOf(id)};
ContentValues cv = new ContentValues();
cv.put(COL_SCOREBOARD_SCORE,newscore);
SQLiteDatabase db = this.getWritableDatabase();
// WARNING without a WHERE clause update would update ALL ROWS
// update returns number of rows updated as an int, so if this is
// greater than 0 true is returned else false.
return db.update(TB_SCOREBOARD,cv,whereclause,whereargs) > 0;
}
public int getScoreById(long id) {
int rv = -1; // just in case the id doesn't exist return -1 so invalid adjustment can be detected
String[] columns = new String[]{COL_SCOREBOARD_SCORE}; // only want the score
String whereclause = COL_SCOREBOARD_ID + "=?"; // will be WHERE _id=? (? replaced by respective whereargs element)
String[] whereargs = new String[]{String.valueOf(id)}; // ? will be replaced with id
Cursor csr = this.getWritableDatabase().query(
TB_SCOREBOARD,
columns,
whereclause,
whereargs,
null,
null,
null
);
if (csr.moveToFirst()) {
//rv = csr.getInt(0); // Hard coded column offsets bad so :-
rv = csr.getInt(csr.getColumnIndex(COL_SCOREBOARD_SCORE));
}
csr.close(); // Done with the cursor so close it
return rv; // return the current score
}
}
Stage 5 - Testing
The activity (based upon a new empty project) could now be :-
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
DBHelper mDBHlpr = new DBHelper(this); // get DBHelper instance
//mDBHlpr.getWritableDatabase(); // force an open (wouldn't normally do this) Not needed now
mDBHlpr.insertRow("Rumplestiltskin the 3rd",10000000); // The winner :)
mDBHlpr.insertRow("Fred Blogs",0); // New user would normally start with score 0
mDBHlpr.updateScore(1,-9999999); //Set Rumplestiltskins's score to 0
// NOTE id should be 1 BUT hard coding id's is
// should be avoided (just used for demo purposes)
mDBHlpr.updateScore(2,1); // Increment Fred's score (see above re hard coded id's)
Cursor csr = mDBHlpr.getAlldata();
StringBuilder sb = new StringBuilder();
// Do something with the Extracted Data
while (csr.moveToNext()) { // Loop through all rows
long userid = csr.getLong(csr.getColumnIndex(DBHelper.COL_SCOREBOARD_ID));
String username = csr.getString(csr.getColumnIndex(DBHelper.COL_SCOREBOARD_USERNAME));
int userscore = csr.getInt(csr.getColumnIndex(DBHelper.COL_SCOREBOARD_SCORE));
sb.append("\n\tUsername=");
sb.append(username);
sb.append((" (ID="));
sb.append(userid);
sb.append(") Score=");
sb.append(userscore);
sb.append(".");
}
csr.close();
Log.d("SCOREBOARD",sb.toString());
}
}
Note cursor handling added
Result
note after numerous runs which will add duplicate usernames but with different id's)
:-
05-18 12:09:46.750 3018-3018/? D/INVALIDSCORE: An invalid new score (less than 0) was returned. Update cancelled.
05-18 12:09:46.754 3018-3018/? D/SCOREBOARD: Username=null (ID=1) Score=1.
Username=null (ID=2) Score=5.
Username=null (ID=3) Score=10000000.
Username=null (ID=4) Score=0.
Username=null (ID=5) Score=10000000.
Username=null (ID=6) Score=0.
Username=null (ID=7) Score=10000000.
Username=null (ID=8) Score=0.
Username=null (ID=9) Score=10000000.
Username=null (ID=10) Score=0.
Invalid Score is because once ID 1 is down to 1 the adjustment of -99999999 will be less than 0.
ID 2's score is 5 due to 5 runs (i.e. 10 rows/users).
Note
The above is a fully working albeit it not that useful, introduction/answer. As such any subsequent questions should really be other questions on Stack Overflow.*

Drop a column when the schema changes in ORMLite

I am working with Android and ORMLite. I am newbie and I upgrading the database to the Version 2. I have the class “Auto” and need to drop the column Alias.
#DatabaseTable(tableName = "Auto")
public class Auto {
public static final String ALIAS = "alias";
public static final String PLACA = "placa";
#DatabaseField(generatedId = true, columnName = "ID")
private int id;
#DatabaseField(canBeNull = false, columnName = ALIAS)
private String alias;
#DatabaseField(canBeNull = false, columnName = PLACA)
public Auto()
{
//ORMLite needs a no-arg constructor
}
public Auto(String alias, String placa) {
this.alias = alias;
this.placa = placa;
}
…..
…..
}
I change the class Auto to…
#DatabaseTable(tableName = "Auto")
public class Auto {
public static final String PLACA = "placa";
#DatabaseField(generatedId = true, columnName = "ID")
private int id;
#DatabaseField(canBeNull = false, columnName = PLACA)
public Auto()
{
//ORMLite needs a no-arg constructor
}
public Auto(String alias, String placa) { this.placa = placa;
}
…..
…..
}
I need to know if ORMLite automatically drops the column Alias when execute “onUpgradeMethod” or I have to do manually like this.
#Override
public void onUpgrade(SQLiteDatabase db, ConnectionSource connectionSource, int oldVersion, int newVersion) {
if(oldVersion == 1) {
try {
Dao dao;
dao = getAutoDao();
dao.executeRaw("ALTER TABLE `Auto` DROP COLUMN alias;");
} catch (SQLException e) {
Log.e(DatabaseHelper.class.getName(), "Error ", e);
}
}
}

This links help me to solve it.
Drop column in SQLite
FAQ SQLite 11
Delete column from SQL
save data before table upgrade
Drop table if it already exists and then re-create it?
SQLite has limited ALTER TABLE support that you can use to add a column to the end of a table or to change the name of a table. If you want to make more complex changes in the structure of a table, you will have to recreate the table. You can save existing data to a temporary table, drop the old table, create the new table, then copy the data back in from the temporary table.
For example, suppose you have a table named "t1" with columns names
"a", "b", and "c" and that you want to delete column "c" from this
table. The following steps illustrate how this could be done:
BEGIN TRANSACTION;
CREATE TEMPORARY TABLE t1_backup(a,b);
INSERT INTO t1_backup SELECT a,b FROM t1;
DROP TABLE t1; CREATE TABLE t1(a,b); INSERT INTO t1 SELECT a,b FROM t1_backup;
DROP TABLE t1_backup;
COMMIT;
The possible solution according to Phil's comments is this. 4
So your choices are:
leave things as they are,
add a new column, copying all the data from old to new as part of the upgrade, and just ignore the old column altogether, or
Use a drop/create strategy to upgrade: back up the data from table into a temporary table, drop the table, re-create it as you'd like it
to be, copy all the data back into it, and finally drop the temporary
table.
Database upgrades are always nervous affairs (need lots of error
handling, lots of testing), frankly if the name is the only thing that
concerns you, I'd leave it alone (option 1). If you really need to
change it then option 2 is low risk but leaves a 'dead' column and
data lying around. Option 3 may be your choice if, for example, the
data is a significant percentage of the overall database size.
I decided to delete the data from the column but don't drop the column, that is similar to solution 2.

Upgrading Android application data with multiple tables (without destroying unaffected tables)

In my Android application; I have a single database with multiple tables.
Each table is more-or-less separate from each other, but figured (for best practice?) to just have one DB file.
When it comes to Upgrades, it's currently an all or nothing affair. On upgrade, it "DROP"'s all the tables and re-creates them. However, this is rather harsh if only one of the tables has changed as all the other tables' data is also lost.
Is there a built-in way to auto-upgrade just the tables that have changed?
(e.g. using a version number per/table?)
If not, I guess I can see two options:
Use separate databases/files for each table, to use built-in version upgrade functionality.
Use the database version number to know when the "schema" has changed, but have a separate table to store the current TABLE_VERSIONS and manage my own upgrade by checking the version number of each table against the current build and DROP/CREATE Tables where needed.
(I'd rather not re-invent the wheel here, so I'm hoping I'm missing something simple...)

You need an abstract class that implements the upgrade process described here. Then you extend this abstract class for each of your tables. In your abstract class you must store you tables in a way(list, hardcoded) so when the onUpgrade fires you iterate over the table items and for each table item you do the described steps. They will be self upgraded, keeping all their existing details. Please note that the onUpgrade event fires only once per database, that's why you need to iterate over all your tables to do the upgrade of all of them. You maintain only 1 version number over all the database.
beginTransaction
run a table creation with if not exists (we are doing an upgrade, so the table might not exists yet, it will fail alter and drop)
put in a list the existing columns List<String> columns = DBUtils.GetColumns(db, TableName);
backup table (ALTER table " + TableName + " RENAME TO 'temp_" + TableName)
create new table (the newest table creation schema)
get the intersection with the new columns, this time columns taken from the upgraded table (columns.retainAll(DBUtils.GetColumns(db, TableName));)
restore data (String cols = StringUtils.join(columns, ",");
db.execSQL(String.format(
"INSERT INTO %s (%s) SELECT %s from temp_%s",
TableName, cols, cols, TableName));
)
remove backup table (DROP table 'temp_" + TableName)
setTransactionSuccessful
(This doesn't handle table downgrade, if you rename a column, you don't get the existing data transfered as the column names do not match).
.
public static List<String> GetColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("select * from " + tableName + " limit 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null)
c.close();
}
return ar;
}
public static String join(List<String> list, String delim) {
StringBuilder buf = new StringBuilder();
int num = list.size();
for (int i = 0; i < num; i++) {
if (i != 0)
buf.append(delim);
buf.append((String) list.get(i));
}
return buf.toString();
}

If you're using the Android SQLite helper classes (i.e. SQLiteOpenHelper) then you only have one version number representing the database schema. Personally, I put all the schema creation code in my instance of SQLiteOpenHelper and keep the upgrade logic simple:
#Override
public void onUpgrade (SQLiteDatabase db, int oldVersion, int newVersion) {
// Alter all the tables so the schema is brought up-to-date.
if (oldVersion < newVersion) {
db.execSQL("ALTER TABLE foo ADD COLUMN new_column INTEGER NOT NULL");
}
}