In my Android application; I have a single database with multiple tables.
Each table is more-or-less separate from each other, but figured (for best practice?) to just have one DB file.
When it comes to Upgrades, it's currently an all or nothing affair. On upgrade, it "DROP"'s all the tables and re-creates them. However, this is rather harsh if only one of the tables has changed as all the other tables' data is also lost.
Is there a built-in way to auto-upgrade just the tables that have changed?
(e.g. using a version number per/table?)
If not, I guess I can see two options:
Use separate databases/files for each table, to use built-in version upgrade functionality.
Use the database version number to know when the "schema" has changed, but have a separate table to store the current TABLE_VERSIONS and manage my own upgrade by checking the version number of each table against the current build and DROP/CREATE Tables where needed.
(I'd rather not re-invent the wheel here, so I'm hoping I'm missing something simple...)
You need an abstract class that implements the upgrade process described here. Then you extend this abstract class for each of your tables. In your abstract class you must store you tables in a way(list, hardcoded) so when the onUpgrade fires you iterate over the table items and for each table item you do the described steps. They will be self upgraded, keeping all their existing details. Please note that the onUpgrade event fires only once per database, that's why you need to iterate over all your tables to do the upgrade of all of them. You maintain only 1 version number over all the database.
beginTransaction
run a table creation with if not exists (we are doing an upgrade, so the table might not exists yet, it will fail alter and drop)
put in a list the existing columns List<String> columns = DBUtils.GetColumns(db, TableName);
backup table (ALTER table " + TableName + " RENAME TO 'temp_" + TableName)
create new table (the newest table creation schema)
get the intersection with the new columns, this time columns taken from the upgraded table (columns.retainAll(DBUtils.GetColumns(db, TableName));)
restore data (String cols = StringUtils.join(columns, ",");
db.execSQL(String.format(
"INSERT INTO %s (%s) SELECT %s from temp_%s",
TableName, cols, cols, TableName));
)
remove backup table (DROP table 'temp_" + TableName)
setTransactionSuccessful
(This doesn't handle table downgrade, if you rename a column, you don't get the existing data transfered as the column names do not match).
.
public static List<String> GetColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("select * from " + tableName + " limit 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null)
c.close();
}
return ar;
}
public static String join(List<String> list, String delim) {
StringBuilder buf = new StringBuilder();
int num = list.size();
for (int i = 0; i < num; i++) {
if (i != 0)
buf.append(delim);
buf.append((String) list.get(i));
}
return buf.toString();
}
If you're using the Android SQLite helper classes (i.e. SQLiteOpenHelper) then you only have one version number representing the database schema. Personally, I put all the schema creation code in my instance of SQLiteOpenHelper and keep the upgrade logic simple:
#Override
public void onUpgrade (SQLiteDatabase db, int oldVersion, int newVersion) {
// Alter all the tables so the schema is brought up-to-date.
if (oldVersion < newVersion) {
db.execSQL("ALTER TABLE foo ADD COLUMN new_column INTEGER NOT NULL");
}
}
Related
I've been looking on this site for a while but have not found the answer. I am trying to do a bulk update on data that I know is already in the table. I have one column that needs to be set when a certain condition comes back for the row ID. Here is the single method but I want to make this more efficient and do it as a bulk. Our database is not in a Provider so I just using a Helper class.
public void markUnavailable(int myId) {
SQLiteDatabase db = this.getWritableDatabase();
String sql = "UPDATE " + MYTABLE + " SET " + Col.IS_AVAILABLE + "= 0"+ " WHERE " + Col.MY_ID + "=" + myId;
db.execSQL(sql);
db.close();
}
I would like to pass in an array of myIds to do the bulk Update. I can't do a Insert or Replace statement because I don't have access to all the column data and don't want to pass this through due to too many codes changes.
public void markUnavailable(int[] myId) {
// ????
/// need some way to loop through and update in bulk
}
Try UPDATE tablename SET column=0 WHERE ID IN (...), where ... is a comma-delimited list of ID values.
I'm not an Android developer, but according to good database practices, you should:
public void markUnavailable(int[] myId) {
SQLiteDatabase db = this.getWritableDatabase();
db.beginTransaction();
SQLiteStatement upd=db.compileStatement("UPDATE "+MYTABLE+" SET "+Col.IS_AVAILABLE+"=0 WHERE "+Col.MY_ID+"=?";
for (int i = 0; i < myId.length; i++) {
upd.bindLong(1, myId[i]);
upd.execute();
}
db.endTransaction();
}
Android has SQLiteDatabase.update would be very usefull in this case, but String [] whereArgs would not deal well with your int[] myId.
The fastest way to do a bulk update would be to do it as a single transaction,by using begin and end transactions. Also if the size of the database is large it will be a good idea to make myID as the primary key of the table as it will significantly increase the speed of the speed in fetching the rows for update when the WHERE clause is used.[It is said that indexing can reduce the speed of update and insert but when the where clause is used,indexing has always increased my speed by huge margins.
public void markUnavailable(int[] myId) {
SQLiteDatabase db = this.getWritableDatabase();
db.beginTransaction();
SQLiteStatement upd=db.compileStatement("UPDATE "+MYTABLE+" SET "+Col.IS_AVAILABLE+"=0 WHERE "+Col.MY_ID+"=?");
for (int i = 0; i < myId.length; i++) {
upd.clearBindings();
upd.bindLong(1, myId[i]); // this matches the first "?"
upd.execute();
}
db.setTransactionSucessful();
db.endTransaction();
}
My app's got a database with three tables in it: one to store the names of the people it tracks, one to track an ongoing event, and one - for lack of a better term - for settings.
I load the first table when the app starts. I ask for a readable database to load in members to display, and later I write to the database when the list changes. I've had no problems here.
The other two tables, however, I can't get to work. The code in the helper classes is identical with the exception of class names and column names, and (at least until the point where I try to access the table) the code to use the table is nearly identical as well.
Here's the code for my helper class (I've got a separate helper for each table, and as I said, it's identical except for class names and columns):
public class db_MembersOpenHelper extends SQLiteOpenHelper
{
public static final String TABLE_NAME = "members_table";
public static final String[] COLUMN_NAMES = new String[] {
Constants.KEY_ID,
"name",
"score"
};
private static final String TABLE_CREATE = "CREATE TABLE " + TABLE_NAME + " ("
+ COLUMN_NAMES[0] + " INTEGER PRIMARY KEY autoincrement, "
+ COLUMN_NAMES[1] + " TEXT, "
+ COLUMN_NAMES[2] + " INTEGER);";
public db_MembersOpenHelper(Context context)
{
super(context, Constants.DATABASE_NAME, null, Constants.DATABASE_VERSION);
}
#Override
public void onCreate(SQLiteDatabase db) { db.execSQL(TABLE_CREATE); }
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion)
{
Log.w("TaskDBAdapter", "Upgrading from version " + oldVersion + " to " + newVersion + ".");
// Do nothing. We do not have any updated DB version
}
}
Here's how I use it successfully:
db_MembersOpenHelper membersDbHelper = new db_MembersOpenHelper(this);
SQLiteDatabase membersDb = membersDbHelper.getReadableDatabase();
Cursor membersResult = membersDb.query(TABLE_NAME, null, null, null, null, null, null);
members = new HashMap<String, Integer>();
membersResult.moveToFirst();
for(int r = 0; r < membersResult.getCount(); r++)
{
members.put(membersResult.getString(1), membersResult.getInt(2));
membersResult.moveToNext();
}
membersDb.close();
And here's where it fails:
db_PlayersOpenHelper playersDbHelper = new db_PlayersOpenHelper(this);
final SQLiteDatabase playersDb = playersDbHelper.getWritableDatabase();
if(newGame)
{
for(String name : players)
{
ContentValues row = new ContentValues();
row.put(COLUMN_NAMES[1], name);
row.put(COLUMN_NAMES[2], (Integer)null);
playersDb.insert(TABLE_NAME, null, row);
}
}
The first one works like a charm. The second results in ERROR/Database(6739): Error inserting achievement_id=null name=c
android.database.sqlite.SQLiteException: no such table: players_table: , while compiling: INSERT INTO players_table(achievement_id, name) VALUES(?, ?);
...
I did do some testing, and the onCreate method is not being called at all for the tables that aren't working. Which would explain why my phone thinks the table doesn't exist, but I don't know why the method isn't getting called.
I can't figure this out; what am I doing so wrong with the one table that I accidentally did right with the other?
I think the problem is that you are managing three tables with with three helpers, but only using one database. SQLiteOpenHelper manages on database, not one table. For example, it checks to see whether the database, not table, exists when it starts. It already does, so onCreate() does not fire.
I would manage all tables with one helper.
Let me see if I get this right. You are trying to create one database with three tables. But when you create the database, you create just one table; you are somehow instantiating the same database at a different place and wonder why its onCreate method doesn't get called. Is this a correct interpretation?
My strategy would be to try and create all three tables in the single onCreate() method.
If you are working with multiple tables, then you have to create all of the tables at once. If you have run your application first and later you update your database, then it will not upgrade your DB.
Now delete your application, then run it again.
There is one more solution but it is not proper. You can declare onOpen method in which you can call onCreate. And add IF NOT EXISTS before table name in your create table string. – Sourabh just now edit
I am getting inconsistent results between two methods of reading the columns in an Android SQLite database.
First, this is part of a database upgrade routine as per the accepted answer here: Upgrade SQLite database from one version to another?
The technique involves moving the current table away with a temporary name, creating a new table with the new schema, and then copying relevant data from the old table into the new one before deleting the old temporary table.
The particular problem I have is when I remove a column from the schema. So, a particular column exists in the old version of the table, but not the new one.
That answer suggests using a method like this to list the columns in the table:
/**
* Returns a list of the table's column names.
*/
private List<String> getColumns(SQLiteDatabase db, final String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("SELECT * FROM " + tableName + " LIMIT 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} finally {
if (c != null)
c.close();
}
return ar;
}
That works fine on the old table, before I move it away with a temporary name and replace it. When I run the same query again later, on the newly-created empty table, it still lists the old table schema with the name of the column which no longer exists. It looks as if it's reusing stale cached results for that query.
If I read the columns a different way, using this instead, then it returns the new column list as expected:
private void listColumns(SQLiteDatabase db, final String tableName) {
final String query = "PRAGMA table_info(" + tableName + ");";
Cursor c = db.rawQuery(query, null);
while (c.moveToNext()) {
Log.v("MyApp", "Column: " + c.getString(1));
}
c.close();
}
The complete sequence is:
final String tempTableName = "temp_" + tableName;
table.addToDb(db); // ensure it exists to start with
// get column names of existing table
final List<String> columns = getColumns(db, tableName);
// backup table
db.execSQL("ALTER TABLE " + tableName + " RENAME TO " + tempTableName);
// create new table
table.addToDb(db);
// delete old columns which aren't in the new schema
columns.retainAll(getColumns(db, tableName));
// restore data from old into new table
String columnList = TextUtils.join(",", columns);
db.execSQL(String.format("INSERT INTO %s (%s) SELECT %s from %s", tableName, columnList, columnList,
tempTableName));
// remove backup
db.execSQL(DROP_TABLE + tempTableName);
What's the reason for the different results?
I assume you have done something similar to this:
ALTER TABLE "main"."mytable" RENAME TO "newtable";
CREATE TABLE "main"."mytable" ("key1" text PRIMARY KEY,"key2" text,"key3" text);
INSERT INTO "main"."mytable" SELECT "key1","key2","key3" FROM "main"."newtable";
DROP TABLE "main"."newtable";
If you have, please share the equivalent code, just to rule out any errors with this part.
I never got to the bottom of this. I just ended up using the second method I mentioned, which doesn't exhibit the problem.
I am planning to write a free version and a full version of a software. I want the information stored by the free version of the software to be accessible by the full version also (I don't want to use Content Providers). And I also want to make sure this data is not lost when the software is updated. How do I achieve this?
You need to implement an intelligent way of onUpgrade for your sqlite helpers.
You should always have the new table creation query at hand, and use that for upgrade and transfer any existing data. Note: that the onUpgrade methods runs once for your sqlite helper object and you need to handle all the tables in it.
So what is recommended onUpgrade:
beginTransaction
run a table creation with if not exists (we are doing an upgrade, so the table might not exists yet, it will fail alter and drop)
put in a list the existing columns List<String> columns = DBUtils.GetColumns(db, TableName);
backup table (ALTER table " + TableName + " RENAME TO 'temp_" + TableName)
create new table (the newest table creation schema)
get the intersection with the new columns, this time columns taken from the upgraded table (columns.retainAll(DBUtils.GetColumns(db, TableName));)
restore data (String cols = StringUtils.join(columns, ",");
db.execSQL(String.format(
"INSERT INTO %s (%s) SELECT %s from temp_%s",
TableName, cols, cols, TableName));
)
remove backup table (DROP table 'temp_" + TableName)
setTransactionSuccessful
(This doesn't handle table downgrade, if you rename a column, you don't get the existing data transfered as the column names do not match).
.
public static List<String> GetColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("select * from " + tableName + " limit 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null)
c.close();
}
return ar;
}
public static String join(List<String> list, String delim) {
StringBuilder buf = new StringBuilder();
int num = list.size();
for (int i = 0; i < num; i++) {
if (i != 0)
buf.append(delim);
buf.append((String) list.get(i));
}
return buf.toString();
}
I am doing my first app with a database and I am having a little trouble understanding the onUpgrade function. My database has a table with an items and a favorite column so that the user can favorite an item. Most implementations I see simply drop the table and reconstruct it but I don't want to do this. I want to be able to add more items to the table.
When the app is upgraded through the android marketplace does the database know its version number? So could I increment the version number in the code and then export it to the marketplace and when the user boots up the upgraded version for the first time then onUpgrade will be called?
If this is the case my onUpgrade would simply pull from a file and add the database items in. Is this a standard way of doing things or is there a better way of handling this in Android. I am trying to stay as standard as possible.
Thanks
Ok, before you run into bigger problems you should know that SQLite is limited on the ALTER TABLE command, it allows add and rename only no remove/drop which is done with recreation of the table.
You should always have the new table creation query at hand, and use that for upgrade and transfer any existing data. Note: that the onUpgrade methods runs one for your sqlite helper object and you need to handle all the tables in it.
So what is recommended onUpgrade:
beginTransaction
run a table creation with if not exists (we are doing an upgrade, so the table might not exists yet, it will fail alter and drop)
put in a list the existing columns List<String> columns = DBUtils.GetColumns(db, TableName);
backup table (ALTER table " + TableName + " RENAME TO 'temp_" + TableName)
create new table (the newest table creation schema)
get the intersection with the new columns, this time columns taken from the upgraded table (columns.retainAll(DBUtils.GetColumns(db, TableName));)
restore data (String cols = StringUtils.join(columns, ",");
db.execSQL(String.format(
"INSERT INTO %s (%s) SELECT %s from temp_%s",
TableName, cols, cols, TableName));
)
remove backup table (DROP table 'temp_" + TableName)
setTransactionSuccessful
(This doesn't handle table downgrade, if you rename a column, you don't get the existing data transfered as the column names do not match).
.
public static List<String> GetColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("select * from " + tableName + " limit 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null)
c.close();
}
return ar;
}
public static String join(List<String> list, String delim) {
StringBuilder buf = new StringBuilder();
int num = list.size();
for (int i = 0; i < num; i++) {
if (i != 0)
buf.append(delim);
buf.append((String) list.get(i));
}
return buf.toString();
}
Next to Pentium10's excellent answer, here are some good examples from living code:
Android AOSP: com.android.providers.calendar.CalendarDatabaseHelper.java
Android AOSP: com.android.browser.BrowserProvider.java
OpenIntents Notepad: org.openintents.notepad.NotePadProvider.java
Thank you for clarifying that onUpgrade() will not support Remove/Drop statements #Pentium 10
For those of you who would like to know the exact moment when onUpgrade() gets called, it is during a call to either getReadableDatabase() or getWriteableDatabase().
To those who are not clear how it ensure it gets triggered...the answer is: It is triggered when the database version provided to the constructor of SqLiteOpenHelper is updated. Here is a example
public class dbSchemaHelper extends SQLiteOpenHelper {
private String sql;
private final String D_TAG = "FundExpense";
//update this to get onUpgrade() method of sqliteopenhelper class called
static final int DB_VERSION = 2;
static final String DB_NAME = "fundExpenseManager";
public dbSchemaHelper(Context context) {
super(context, DB_NAME, null, DB_VERSION);
// TODO Auto-generated constructor stub
}
now to...onUpgrade()
#Override
public void onUpgrade(SQLiteDatabase arg0, int arg1, int arg2) {
sql = "ALTER TABLE " + fundExpenseSchema.Expense.TABLE_NAME + " ADD COLUMN " + fundExpenseSchema.Expense.FUNDID + " INTEGER";
arg0.execSQL(sql);
}
I've been using the solution proposed by #Pentium10 for a long time but today i had a problem, after doing alter table, getColumns from the original table still returns the same columns (in the new version of the db the table suffer mayor structure changes, some columns added some others), really i don't know why select statement does not reflect the structure changes, more over before creating my table again, select statement still returns the columns! When the table is not re-created yet!
So i manage solving this issue updating getColumns method using pragma table_info, like this:
/**
* Get a list of column base_dictionary for the selected table
*
* #param db
* Database that contains the table
* #param tableName
* Table name to be used
* #return A List of column name
*/
public static List<String> getColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("pragma table_info(" + tableName + ")", null);
ar = new ArrayList<String>();
if (c != null && c.moveToFirst()) {
do {
ar.add(c.getString(c.getColumnIndexOrThrow("name")));
} while (c.moveToNext());
c.close();
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null) c.close();
}
return ar;
}