Develop Reference

Providing my android app with a DB encrypted with sqlcipher doesn't work, can't open it

My android app works with a provided DB populated and located on the asset folder. I recently had to start adding private datas in it so I used sqlcipher to encrypt it. So before starting:
The encryption goes well, since I can decrypt it easily and read it back. This is done with sqlcipher shell.
Everything was working like a charm with the db not crypted
What I was doing was, on the first launch of my app, creating a blank db on my phone, copy and past the content of the provided db in it and then, the user was able to have his own db on his phone without having to recreate it each time ( the db is pretty big ). Indeed, if the user already has the db on his phone for the further launch, he won't have to recreate it this way.
But with sqlcipher, it's not working anymore.
Important: When using sqlcipher with a non crypted db ( using "" as parameter for the related method like openDatabase ), it was working as well.
But when I try with the crypted db and with the password, what I have is the error
file is encrypted or is not a database: create locale table failed
This happens after I created a blank db using this:
//By calling this method and empty database will be created into the default system path
//of your application so we are gonna be able to overwrite that database with our database.
this.getReadableDatabase("password").close();
I then try to open it with the following instruction:
//Open the database
String myPath = DB_PATH + DATABASE_NAME;
myDataBase = SQLiteDatabase.openDatabase(myPath, "password", null, SQLiteDatabase.OPEN_READWRITE);
And then happens the error. Someone has an idea please? I'm saddly not an android expert, using a db was already difficult but as you can guess it's getting on another level now. Any help would be welcome. Thanks in advance !

Finally solve my problem after a long search following this link: Sqlcipher __ CREATE TABLE android_metadata failed
The error was coming from the way I was opening my db, which changes whether you use sqlcipher or sqlite.
Old way to open the db using sqlite:
myDataBase = SQLiteDatabase.openDatabase(myPath, "password", null, SQLiteDatabase.OPEN_READWRITE);
New way using sqlcipher, with a SQLiteDatabaseHook:
SQLiteDatabaseHook hook = new SQLiteDatabaseHook() {
public void preKey(SQLiteDatabase database) {
}
public void postKey(SQLiteDatabase database) {
database.rawExecSQL("PRAGMA cipher_migrate;");
}
};
myDataBase = SQLiteDatabase.openOrCreateDatabase(myPath, "password", null, hook);

Attempt to write a readonly database... but I'm not

I have a read-only database connection. Sometimes, when reading data from the database with a SELECT query, it throws a SQLiteReadOnlyDatabaseException.
I open the connection like this:
return SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READONLY);
The query is:
Select * FROM BudgetVersions WHERE entityId = ?
I read data from the database using db.rawQuery(), like this:
String query = ...;
Cursor c = db.rawQuery(query, new String[]{ activeBudgetId });
try {
if (c.moveToFirst()) {
bv.versionName = c.getString(c.getColumnIndexOrThrow("versionName"));
return bv;
} else {
return null;
}
} finally {
c.close();
}
Very rarely, I get a crash like this, inside the call to c.moveToFirst():
Caused by: android.database.sqlite.SQLiteReadOnlyDatabaseException: attempt to write a readonly database (code 776)
at android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(Native Method)
at android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:845)
at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:836)
at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62)
at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:144)
at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:133)
at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:197)
at android.database.AbstractCursor.moveToFirst(AbstractCursor.java:237)
As a workaround, I could try using a writable database connection instead, but I'd like to know why the crash is happening.
The table I'm reading from is a standard SQLite table:
CREATE TABLE BudgetVersions (
entityId VARCHAR PRIMARY KEY NOT NULL UNIQUE,
budgetId VARCHAR NOT NULL,
versionName VARCHAR NOT NULL,
dateFormat VARCHAR,
currencyFormat VARCHAR,
lastAccessedOn DATETIME,
isTombstone BOOL NOT NULL,
deviceKnowledge NUMERIC NOT NULL
);
I've seen the crash happen on both a KitKat emulator and a device running Lollipop.
There is a separate writeable connection open to the same database at the same time, owned by a WebView. The database is being updated by Javascript code in the WebView, and read from in the native Android/Java layer with this read-only connection.
I expect this may prove to be the ultimate cause of the problem, but I'd like to understand in detail why a read-only connection would interfere with a separate writeable connection.
I am well aware that the general advice is to use a single connection to the database, but since the writeable connection is owned by the WebView, I don't have easy access to it from the Java code.

Solved by changing it to a writeable database connection. The clue was in the documentation for the 776 error code:
(776) SQLITE_READONLY_ROLLBACK
The SQLITE_READONLY_ROLLBACK error code is an extended error code for
SQLITE_READONLY. The SQLITE_READONLY_ROLLBACK error code indicates
that a database cannot be opened because it has a hot journal that
needs to be rolled back but cannot because the database is readonly.
During development, I am frequently interrupting the currently-running app to install and run a new version. This causes the currently-running app to be force-stopped by the system. If the Javascript code in the WebView is in the middle of writing to the database via its separate writeable connection when the app is nuked, then a hot journal will be left behind.
When the new version of the app starts up, the read-only database connection in the native Java code is opened. When this connection spots the journal, it tries to roll back the journal. And because it's a read-only connection, it fails.
(This fits with the crash being observed immediately on startup after I've made a change.)
The correct fix is therefore to make the Java connection a writeable connection. This connection never attempts a write during normal operation, but it must write when recovering from a previous interrupted write through the WebView's writeable connection.

Is there a memory cache for SQLite in Android and how to release or clear it?

Firstly, I create a database called "mydb" in my Android app:
DBHelper dbHelper = new DBHelper(context, "mydb", null, 1);//DBHelper is my custom class
And write some data into it's table:
SQLiteDatabase db = dbHelper.getReadableDatabase();
db.execSQL("insert into mytable(name, text) values ('allen','hello')");
Here, everything is ok. But then, i delete this database manually not by programming, with a software "R.E. explore" (Certainly on a rooted device).
Then, in my code, i read this table of the database. What is astonishing is that i still could get the data I stored.
Cursor cursor = db.query("mytable", new String[]{"name","text"}, null, null, null, null, null);
Why?

Quoting from the Android Developers reference website:
Once opened successfully, the database is cached, so you can call
this method every time you need to write to the database. (Make sure
to call close() when you no longer need the database.)
This is from the description of the getWritableDatabase() method, however both getReadableDatabase() and getWritableDatabase() return basically the same object for reading the database.
Please note that you should use getWritableDatabase() if you want to persist the changes you make to the database on the device's internal memory. Otherwise they will be valid only for the duration of the application's runtime and will be discarded once the app is closed. If you wish to delete the database completely, you should call the SQLiteDatabase's close() method in order to invalidate the cache.

use SQLiteDatabase.deleteDatabase(File file) API to delete the database
Deletes a database including its journal file and other auxiliary files that may have been created by the database engine.
Make sure you have closed all the connections that are open.
In case you are not able to do that,
just cal the deleteDatabase followed by kill process.. - not recommended

You need to delete the app from your phone then install again

sqlite update not releasing memory on HTC desire

Running my app on an HTC Desire (android version 2.3.3) I noticed memory usage increasing when calling the following function:
private static final String TABLE_MODULES = "tbl_module1";
private static final String MOD1_INCLUDE = "mod1_include";
private void setItemInclude (int idx, boolean include)
{
String incString = "";
if (include)
incString = "true";
else
incString = "false";
ContentValues args = new ContentValues();
args.put(MOD1_INCLUDE, incString);
database.update(TABLE_MODULES,args,"_id="+idx,null);
}
where database is of type SQLiteDatabase and was obtained from a call to dbHelper.getWritableDatabase() and where dbHelper is derived from SQLiteOpenHelper and instantiated with the application context.
I get the desired update, but when I check the data usage of my application it appears to increase by about 60KB per call and is not released unless dbHelper.close() is called. Multiple calls to this function result in SQLiteDiskIOException.
I have tried database.rawQuery(), which also updates correctly but hangs on to memory similarly.
String updateQuery = "UPDATE " + TABLE_MODULES + " SET MOD1_INCLUDE = " + "'" + incString + "'" + " WHERE _id ='" + idx + "'";
Cursor cursor = database.rawQuery(updateQuery, null);
cursor.moveToFirst();
cursor.close();
Running the same code on an Asus (android 4.0.3) and an Orange San Francisco (android 2.1.update1) does not have this problem.
getWritableDatabase is called from the activity's onCreate() and the resulting database is closed in onDestroy(). I thought that it was ok to keep the database open for the duration of the activity. Any ideas as to what is the cause?

I have found what is causing the problem.
On the HTC, each time database.update() is called a temporary file is updated in the application's databases directory. This file has the extension .sqlite-wal. This file gets bigger (by 60KB) on each call and is only deleted when I close the database.
The ASUS also creates a temporary file but with the extension .sqlite-journal. This file is also updated on each update() call, but once the transaction completes the file's size reduces to zero. So, no memory problems result.
I am not sure what the difference is between Journal and Wal but believe that they are both to do with rolling back a database after a failed transaction. I have solved the memory problem with the following code, which turns off the journal/wal feature.
Cursor cursor = database.rawQuery("PRAGMA journal_mode=OFF", null);
cursor.moveToFirst();
cursor.close();
Calling this after opening the database appears to prevent these temporary files from being created.
However, I am not confident that this is a good idea. I was surprised not to find others reporting this issue and wonder if leaving the database open ready for multiple transactions is considered bad practice? I am new to SQL and have fairly limited experience with databases.

The documentation is quite clear.
The WAL journaling mode uses a write-ahead log instead of a rollback
journal to implement transactions.
In your case it sounds like automatic checkpointing is either not configured or not working. You probably don't want to turn journaling off as it can result in data loss.

What is the proper technique for updating database tables

I have database updates like the two below, throughout my code. Most of my updates open the database first before updating the record and then closes the database after the record has been updated. I have noticed that not using this statement: mDb = Helper.getWritableDatabase(); before the insertion and this statement: mDb.close(); after insertion, will cause a force close error sometimes, but not always. What is the proper way. Do I use the open and close statements all the time or only when I have to or should I always open and then close during the update process. What is the proper technique. Here is the snippet with the open close statements. Thanks in advance. Is the open statement necessary?
// Open connections to the database
mDb = Helper.getWritableDatabase();
// update 1
String strFilter7 = "_id=" + 7;
ContentValues args7 = new ContentValues();
args7.put(COL_VALUE, newB1ftgvalue);
mDb.update("VarData", args7, strFilter7, null);
// update 2
String strFilter11 = "_id=" + 11;
ContentValues args11 = new ContentValues();
args11.put(COL_VALUE, newB2ftgvalue);
mDb.update("VarData", args11, strFilter11, null);
// closes database
mDb.close();

It is good practice to always call close() after you are done with database updates. If you haven't closed the database you may see errors. Once database is open, you may do multiple updates. It shouldn't be an issue. One thing to take care is, it is better not to keep open connection for long time due to lot of reasons. Here is good discussion on this topic.

Do not close it and do only have one sqlite helper. It's basically a static openhelper. There is no problem with never closing your database. This link gives a good piece of code that works great. You will not have memory leaks with an open database. You will however have problems with open cursors, so make sure to close those.
http://www.touchlab.co/blog/single-sqlite-connection/
A good discussion is here:
What are the best practices for SQLite on Android?
I realized the link I posted has changed since when I once first viewed it. Change:
instance = new DatabaseHelper(context);
to
instance = new DatabaseHelper(context.getApplicationContext());
and
public class DatabaseHelper extends OrmLiteSqliteOpenHelper
to
public class DatabaseHelper extends SqliteOpenHelper

You need to open the db as a writable database in order to modify its data, so yes, you have to open it before updating records.