I am planning to write a free version and a full version of a software. I want the information stored by the free version of the software to be accessible by the full version also (I don't want to use Content Providers). And I also want to make sure this data is not lost when the software is updated. How do I achieve this?
You need to implement an intelligent way of onUpgrade for your sqlite helpers.
You should always have the new table creation query at hand, and use that for upgrade and transfer any existing data. Note: that the onUpgrade methods runs once for your sqlite helper object and you need to handle all the tables in it.
So what is recommended onUpgrade:
beginTransaction
run a table creation with if not exists (we are doing an upgrade, so the table might not exists yet, it will fail alter and drop)
put in a list the existing columns List<String> columns = DBUtils.GetColumns(db, TableName);
backup table (ALTER table " + TableName + " RENAME TO 'temp_" + TableName)
create new table (the newest table creation schema)
get the intersection with the new columns, this time columns taken from the upgraded table (columns.retainAll(DBUtils.GetColumns(db, TableName));)
restore data (String cols = StringUtils.join(columns, ",");
db.execSQL(String.format(
"INSERT INTO %s (%s) SELECT %s from temp_%s",
TableName, cols, cols, TableName));
)
remove backup table (DROP table 'temp_" + TableName)
setTransactionSuccessful
(This doesn't handle table downgrade, if you rename a column, you don't get the existing data transfered as the column names do not match).
.
public static List<String> GetColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("select * from " + tableName + " limit 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null)
c.close();
}
return ar;
}
public static String join(List<String> list, String delim) {
StringBuilder buf = new StringBuilder();
int num = list.size();
for (int i = 0; i < num; i++) {
if (i != 0)
buf.append(delim);
buf.append((String) list.get(i));
}
return buf.toString();
}
Related
My app is using an external SQLite database. The database is created using DB Browser for SQLite software. I am using the following method to query my table with the column ENGLISH (same as en_word). However, problem is the query is slow when my database become large.
public static final String ENGLISH = "en_word";
public static final String TABLE_NAME = "words";
String sql = "SELECT * FROM " + TABLE_NAME +
" WHERE " + ENGLISH + " LIKE ? ORDER BY LENGTH(" + ENGLISH + ") LIMIT 100";
SQLiteDatabase db = initializer.getReadableDatabase();
Cursor cursor = null;
try {
cursor = db.rawQuery(sql, new String[]{"%" + englishWord.trim() + "%"});
List<Bean> wordList = new ArrayList<Bean>();
while(cursor.moveToNext()) {
String english = cursor.getString(1);
String mal = cursor.getString(2);
wordList.add(new Bean(english, mal));
}
return wordList;
} catch (SQLiteException exception) {
exception.printStackTrace();
return null;
} finally {
if (cursor != null)
cursor.close();
}
I tried to create index using DB Browser for SQLite.
CREATE INDEX `kindx` ON `words` ( `en_word` )
However, do I need to modify my code so that my app will query the database using this index? If so, how to do that?
The problem is that SQLite, like most relational databases, can use an index when the parameter to a 'like' clause ends with a wildcard, it cannot use an index when the parameter begins with a wildcard.
So, for this type of query, the index will not be used, and you wind up with a full table scan. This is why it is slower with a large number of rows.
You are actually attempting to do what is known as "full text search", which is not really possible to do efficiently without database features to support it directly.
I have not tried it, but I see that SQLite does have full-text search capabilities, and that it is supported on Android. See Full text search example in Android for an example.
I am developing an app (that uses SQLite) in Android Studio 2.3 on Mac OS and would like to view the SQLite database, but can't seem to find the sqlite file. Where are the AVM files and specifically the sqlite file?
I found the AVD folder
/Users/<my name>/.android/avd/Pixel_XL_API_26.avd but can't find the user generated data. Any suggestions?
You could always include methods to find out information about the database.
For example you can query the sqlite_master table e.g. :-
Cursor csr = db.query("sqlite_master",null,null,null,null,null,null);
which equates to :-
SELECT * FROM sqlite_master
The are PRAGMA statments that allow you to interrogate/change internals PRAGMA Statements.
Note use rawQuery to obtain information into a cursor but use execSQL to make changes e.g. to set and get user_version (Database Version) respectively:-
db.execSQL("PRAGMA user_version=" + Integer.toString(version));
Cursor csr = db.rawQuery("PRAGMA user_version",null);
You could also look at the data, output to the log, in the tables with something based upon (where tablename would be changed to the appropriate tablename) :-
Cursor csr = db.query("tablename",null,null,null,null,null,null);
String logdata;
while (csr.moveToNext()) {
Log.d("DBINFO","Row " + csr.getPosition());
logdata = "";
for (int i =0; i < csr.getColumnCount(); i++) {
String val;
try {
val = csr.getString(i);
}
catch (Exception e) {
val = "unobtainable";
}
logdata = logdata + "\t\nColumn Name=" +
csr.getColumnName(i) +
" Value=" + val;
}
Log.d("DBINFO",logdata);
}
csr.close();
Note! this only uses getString so will not properly represent some values e.g. double's and floats, if a BLOB then the value will be unobtainable.
In my Android application; I have a single database with multiple tables.
Each table is more-or-less separate from each other, but figured (for best practice?) to just have one DB file.
When it comes to Upgrades, it's currently an all or nothing affair. On upgrade, it "DROP"'s all the tables and re-creates them. However, this is rather harsh if only one of the tables has changed as all the other tables' data is also lost.
Is there a built-in way to auto-upgrade just the tables that have changed?
(e.g. using a version number per/table?)
If not, I guess I can see two options:
Use separate databases/files for each table, to use built-in version upgrade functionality.
Use the database version number to know when the "schema" has changed, but have a separate table to store the current TABLE_VERSIONS and manage my own upgrade by checking the version number of each table against the current build and DROP/CREATE Tables where needed.
(I'd rather not re-invent the wheel here, so I'm hoping I'm missing something simple...)
You need an abstract class that implements the upgrade process described here. Then you extend this abstract class for each of your tables. In your abstract class you must store you tables in a way(list, hardcoded) so when the onUpgrade fires you iterate over the table items and for each table item you do the described steps. They will be self upgraded, keeping all their existing details. Please note that the onUpgrade event fires only once per database, that's why you need to iterate over all your tables to do the upgrade of all of them. You maintain only 1 version number over all the database.
beginTransaction
run a table creation with if not exists (we are doing an upgrade, so the table might not exists yet, it will fail alter and drop)
put in a list the existing columns List<String> columns = DBUtils.GetColumns(db, TableName);
backup table (ALTER table " + TableName + " RENAME TO 'temp_" + TableName)
create new table (the newest table creation schema)
get the intersection with the new columns, this time columns taken from the upgraded table (columns.retainAll(DBUtils.GetColumns(db, TableName));)
restore data (String cols = StringUtils.join(columns, ",");
db.execSQL(String.format(
"INSERT INTO %s (%s) SELECT %s from temp_%s",
TableName, cols, cols, TableName));
)
remove backup table (DROP table 'temp_" + TableName)
setTransactionSuccessful
(This doesn't handle table downgrade, if you rename a column, you don't get the existing data transfered as the column names do not match).
.
public static List<String> GetColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("select * from " + tableName + " limit 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null)
c.close();
}
return ar;
}
public static String join(List<String> list, String delim) {
StringBuilder buf = new StringBuilder();
int num = list.size();
for (int i = 0; i < num; i++) {
if (i != 0)
buf.append(delim);
buf.append((String) list.get(i));
}
return buf.toString();
}
If you're using the Android SQLite helper classes (i.e. SQLiteOpenHelper) then you only have one version number representing the database schema. Personally, I put all the schema creation code in my instance of SQLiteOpenHelper and keep the upgrade logic simple:
#Override
public void onUpgrade (SQLiteDatabase db, int oldVersion, int newVersion) {
// Alter all the tables so the schema is brought up-to-date.
if (oldVersion < newVersion) {
db.execSQL("ALTER TABLE foo ADD COLUMN new_column INTEGER NOT NULL");
}
}
I am getting inconsistent results between two methods of reading the columns in an Android SQLite database.
First, this is part of a database upgrade routine as per the accepted answer here: Upgrade SQLite database from one version to another?
The technique involves moving the current table away with a temporary name, creating a new table with the new schema, and then copying relevant data from the old table into the new one before deleting the old temporary table.
The particular problem I have is when I remove a column from the schema. So, a particular column exists in the old version of the table, but not the new one.
That answer suggests using a method like this to list the columns in the table:
/**
* Returns a list of the table's column names.
*/
private List<String> getColumns(SQLiteDatabase db, final String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("SELECT * FROM " + tableName + " LIMIT 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} finally {
if (c != null)
c.close();
}
return ar;
}
That works fine on the old table, before I move it away with a temporary name and replace it. When I run the same query again later, on the newly-created empty table, it still lists the old table schema with the name of the column which no longer exists. It looks as if it's reusing stale cached results for that query.
If I read the columns a different way, using this instead, then it returns the new column list as expected:
private void listColumns(SQLiteDatabase db, final String tableName) {
final String query = "PRAGMA table_info(" + tableName + ");";
Cursor c = db.rawQuery(query, null);
while (c.moveToNext()) {
Log.v("MyApp", "Column: " + c.getString(1));
}
c.close();
}
The complete sequence is:
final String tempTableName = "temp_" + tableName;
table.addToDb(db); // ensure it exists to start with
// get column names of existing table
final List<String> columns = getColumns(db, tableName);
// backup table
db.execSQL("ALTER TABLE " + tableName + " RENAME TO " + tempTableName);
// create new table
table.addToDb(db);
// delete old columns which aren't in the new schema
columns.retainAll(getColumns(db, tableName));
// restore data from old into new table
String columnList = TextUtils.join(",", columns);
db.execSQL(String.format("INSERT INTO %s (%s) SELECT %s from %s", tableName, columnList, columnList,
tempTableName));
// remove backup
db.execSQL(DROP_TABLE + tempTableName);
What's the reason for the different results?
I assume you have done something similar to this:
ALTER TABLE "main"."mytable" RENAME TO "newtable";
CREATE TABLE "main"."mytable" ("key1" text PRIMARY KEY,"key2" text,"key3" text);
INSERT INTO "main"."mytable" SELECT "key1","key2","key3" FROM "main"."newtable";
DROP TABLE "main"."newtable";
If you have, please share the equivalent code, just to rule out any errors with this part.
I never got to the bottom of this. I just ended up using the second method I mentioned, which doesn't exhibit the problem.
I am doing my first app with a database and I am having a little trouble understanding the onUpgrade function. My database has a table with an items and a favorite column so that the user can favorite an item. Most implementations I see simply drop the table and reconstruct it but I don't want to do this. I want to be able to add more items to the table.
When the app is upgraded through the android marketplace does the database know its version number? So could I increment the version number in the code and then export it to the marketplace and when the user boots up the upgraded version for the first time then onUpgrade will be called?
If this is the case my onUpgrade would simply pull from a file and add the database items in. Is this a standard way of doing things or is there a better way of handling this in Android. I am trying to stay as standard as possible.
Thanks
Ok, before you run into bigger problems you should know that SQLite is limited on the ALTER TABLE command, it allows add and rename only no remove/drop which is done with recreation of the table.
You should always have the new table creation query at hand, and use that for upgrade and transfer any existing data. Note: that the onUpgrade methods runs one for your sqlite helper object and you need to handle all the tables in it.
So what is recommended onUpgrade:
beginTransaction
run a table creation with if not exists (we are doing an upgrade, so the table might not exists yet, it will fail alter and drop)
put in a list the existing columns List<String> columns = DBUtils.GetColumns(db, TableName);
backup table (ALTER table " + TableName + " RENAME TO 'temp_" + TableName)
create new table (the newest table creation schema)
get the intersection with the new columns, this time columns taken from the upgraded table (columns.retainAll(DBUtils.GetColumns(db, TableName));)
restore data (String cols = StringUtils.join(columns, ",");
db.execSQL(String.format(
"INSERT INTO %s (%s) SELECT %s from temp_%s",
TableName, cols, cols, TableName));
)
remove backup table (DROP table 'temp_" + TableName)
setTransactionSuccessful
(This doesn't handle table downgrade, if you rename a column, you don't get the existing data transfered as the column names do not match).
.
public static List<String> GetColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("select * from " + tableName + " limit 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null)
c.close();
}
return ar;
}
public static String join(List<String> list, String delim) {
StringBuilder buf = new StringBuilder();
int num = list.size();
for (int i = 0; i < num; i++) {
if (i != 0)
buf.append(delim);
buf.append((String) list.get(i));
}
return buf.toString();
}
Next to Pentium10's excellent answer, here are some good examples from living code:
Android AOSP: com.android.providers.calendar.CalendarDatabaseHelper.java
Android AOSP: com.android.browser.BrowserProvider.java
OpenIntents Notepad: org.openintents.notepad.NotePadProvider.java
Thank you for clarifying that onUpgrade() will not support Remove/Drop statements #Pentium 10
For those of you who would like to know the exact moment when onUpgrade() gets called, it is during a call to either getReadableDatabase() or getWriteableDatabase().
To those who are not clear how it ensure it gets triggered...the answer is: It is triggered when the database version provided to the constructor of SqLiteOpenHelper is updated. Here is a example
public class dbSchemaHelper extends SQLiteOpenHelper {
private String sql;
private final String D_TAG = "FundExpense";
//update this to get onUpgrade() method of sqliteopenhelper class called
static final int DB_VERSION = 2;
static final String DB_NAME = "fundExpenseManager";
public dbSchemaHelper(Context context) {
super(context, DB_NAME, null, DB_VERSION);
// TODO Auto-generated constructor stub
}
now to...onUpgrade()
#Override
public void onUpgrade(SQLiteDatabase arg0, int arg1, int arg2) {
sql = "ALTER TABLE " + fundExpenseSchema.Expense.TABLE_NAME + " ADD COLUMN " + fundExpenseSchema.Expense.FUNDID + " INTEGER";
arg0.execSQL(sql);
}
I've been using the solution proposed by #Pentium10 for a long time but today i had a problem, after doing alter table, getColumns from the original table still returns the same columns (in the new version of the db the table suffer mayor structure changes, some columns added some others), really i don't know why select statement does not reflect the structure changes, more over before creating my table again, select statement still returns the columns! When the table is not re-created yet!
So i manage solving this issue updating getColumns method using pragma table_info, like this:
/**
* Get a list of column base_dictionary for the selected table
*
* #param db
* Database that contains the table
* #param tableName
* Table name to be used
* #return A List of column name
*/
public static List<String> getColumns(SQLiteDatabase db, String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("pragma table_info(" + tableName + ")", null);
ar = new ArrayList<String>();
if (c != null && c.moveToFirst()) {
do {
ar.add(c.getString(c.getColumnIndexOrThrow("name")));
} while (c.moveToNext());
c.close();
}
} catch (Exception e) {
Log.v(tableName, e.getMessage(), e);
e.printStackTrace();
} finally {
if (c != null) c.close();
}
return ar;
}