Develop Reference

How to manually perform checkpoint in SQLite android?

I'm trying to create a backup of my sqlite database and I want to flush the content of the WAL file in the db first.
Here is my SQLiteOpenHelper:
public class MyDBHelper extends SQLiteOpenHelper {
private Context mContext;
private static MyDBHelper mInstance = null;
private MyDBHelper(final Context context, String databaseName) {
super(new MYDB(context), databaseName, null, DATABASE_VERSION);
this.mContext = context;
}
#Override
public void onCreate(SQLiteDatabase db) {
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
public static MyDBHelper getInstance(Context context) {
if (mInstance == null) {
mInstance = new MyDBHelper(context, DATABASE_NAME);
}
return mInstance;
}
private void closeDataBase(Context context) {
getInstance(context).close();
}
}
Now, my understanding is that after a checkpoint is completed, the mydb.db-wal file should be empty. Is that correct?
Here is what I've tried so far:
1.
public Completable flushWalInDB() {
return Completable.fromAction(new Action() {
#Override
public void run() throws Exception {
getInstance(mContext).getReadableDatabase().rawQuery("pragma wal_checkpoint;", null);
}
});
}
This doesn't throw an error but doesn't seem to do anything. After running this, I physically checked my mydb.db-wal file and had the same size. I also checked the db on the device and nothing was added in the database
After some digging around I found this
[https://stackoverflow.com/a/30278485/2610933][1]
and tried this:
2.
public Completable flushWalInDB() {
return Completable.fromAction(new Action() {
#Override
public void run() throws Exception {
getInstance(mContext).getReadableDatabase().execSQL("pragma wal_checkpoint;");
}
});
}
When running this it throws an error:
unknown error (code 0): Queries can be performed using SQLiteDatabase query or rawQuery methods only.
And based on this answer [https://stackoverflow.com/a/19574341/2610933][1] , I also tried to VACUUM the DB but nothing seems to happen.
public Completable vacuumDb() {
return Completable.fromAction(new Action() {
#Override
public void run() throws Exception {
getInstance(mContext).getReadableDatabase().execSQL("VACUUM");
}
});
}
}
Whats is the correct way of flushing the WAL file in the DB before creating a backup?
Thank you.

PRAGMA wal_checkpoint(2) does copy all data from the WAL into the actual database file, but it does not remove the -wal file, and any concurrent connections can make new changes right afterwards.
If you want to be really sure that there is no WAL to interfere with your backup, run PRAGMA journal_mode = DELETE. (You can switch it back afterwards.)

To manually add checkpont use PRAGMA wal_checkpoint, after searching for 2 hours following code worked for me -:
SQLiteDatabase db = this.getWritableDatabase();
String query = "PRAGMA wal_checkpoint(full)";
Cursor cursor = db.rawQuery(query, null);
if (cursor != null && cursor.moveToFirst()) {
int a = cursor.getInt(0);
int b = cursor.getInt(1);
int c = cursor.getInt(2);
}
if (cursor != null) {
cursor.close();
}

junit.framework.AssertionFailedError

I get error of this:
junit.framework.AssertionFailedError at
com.example.sirivan.myfirstapp.test.TestDb.testCreateDb(TestDb.java:22)
TestDb.java
public class TestDb extends AndroidTestCase {
public static final String LOG_TAG = TestDb.class.getSimpleName();
public void testCreateDb() throws Throwable{
assertTrue(mContext.deleteDatabase(WeatherDbHelper.DATABASE_NAME));
SQLiteDatabase db = new WeatherDbHelper(
this.mContext).getWritableDatabase();
assertEquals(true, db.isOpen());
db.close();
}
}

Following the documentation:
http://developer.android.com/reference/android/content/Context.html#deleteDatabase(java.lang.String)
Returns true if the database was successfully deleted; else false.

How use Roboelectric and Fest to test an android app with native binaries?

I'm trying to setup roboelectric and fest in my own project. However when I try to run ./gradlew clean test in the command line I get the following errors in the test report:
http://pastebin.com/5gaJgftf
My project does build the app without errors though. I only get this issue when I try to run tests, so it seems that Roboelectric is not aware is not aware of my native sqlcipher binaries and other binaries.
So I tried loading it with a shadow class for the runner that loads up the necessary binaries:
#Config(emulateSdk = 18, shadows={MyJniClass.class})
#RunWith(RobolectricTestRunner.class)
public class MainActivityBuildTest {
#Test
public void testSomething() throws Exception {
Activity activity = Robolectric.buildActivity(MainActivity.class).create().get();
assertTrue(activity != null);
}
}
Using my custom jniloader shadow class
#Implements(RobolectricTestRunner.class)
class MyJniClass {
static {
try {
System.loadLibrary("libdatabase_sqlcipher");
System.loadLibrary("libdatabase_android");
System.loadLibrary("libstlport_shared");
} catch (UnsatisfiedLinkError e) {
// only ignore exception in non-android env
if ("Dalvik".equals(System.getProperty("java.vm.name"))) throw e;
}
}
}

You have issues to use sql cipher with robolectric?
My workaround is to use two different implementation of the SQLiteOpenHelper. One use sqlcipher and the another one the default database implementation. This both are behind a factory class, which create the SQLiteDatabase based on a static boolean flag, so the unscure database handling will be eliminated from progard.
The next issue is that both have different SQLiteDatabase classes. So again build a wrapper around the SQLiteDatabase which will be created with the right SQLiteDatabase from the SQLiteOpenHelper Wrapper. Take the Cipher variant as your base. you can ignore methods which exist at default SQLiteDatabase but not at the cipher variant. This wrapper class take the same static boolean flag to choose which database should be used. if make a mistake and take the wrong database then it should throw a null pointer exception ;)
in your app code you should now use only the wrapper classes.
example for DatabaseHelper wrapper
public class MyDatabaseHelper {
public static final String DATABASE_NAME = "my.db";
public static final int DATABASE_VERSION = 1;
MyEncryptedDatabaseHelper encryptedDatabase;
MyUnsecureDatabaseHelper unsecureDatabase;
public MyDatabaseHelper(Context context) {
if (ReleaseControl.USE_UNSECURE_DATABASE) {
unsecureDatabase = new MyUnsecureDatabaseHelper(context, DATABASE_NAME, null, DATABASE_VERSION);
return;
}
encryptedDatabase = new MyEncryptedDatabaseHelper(context, DATABASE_NAME, null, DATABASE_VERSION);
}
public MySQLiteDatabase getWritableDatabase(String password) throws MySQLiteException {
if (ReleaseControl.USE_UNSECURE_DATABASE) {
try {
return new MySQLiteDatabase(unsecureDatabase.getWritableDatabase());
} catch (android.database.SQLException e) {
throw new MySQLiteException(e);
}
}
try {
return new MySQLiteDatabase(encryptedDatabase.getWritableDatabase(password));
} catch (net.sqlcipher.database.SQLiteException e) {
throw new MySQLiteException(e);
}
}
}
and short snippet from SQLiteDatabase wrapper
public class MySQLiteDatabase {
private net.sqlcipher.database.SQLiteDatabase encryptedDatabase;
private android.database.sqlite.SQLiteDatabase unsecureDatabase;
public MySQLiteDatabase(SQLiteDatabase database) {
encryptedDatabase = database;
}
public MySQLiteDatabase(android.database.sqlite.SQLiteDatabase database) {
unsecureDatabase = database;
}
public static void loadLibs(android.content.Context context) {
if (ReleaseControl.USE_UNSECURE_DATABASE) { return; }
SQLiteDatabase.loadLibs(context);
}
public static int releaseMemory() {
if (ReleaseControl.USE_UNSECURE_DATABASE) {
return android.database.sqlite.SQLiteDatabase.releaseMemory();
}
return net.sqlcipher.database.SQLiteDatabase.releaseMemory();
}
public static SQLiteDatabase openDatabase(String path, String password, MyCursorFactory factory, int flags) {
if(factory == null) factory = new NullCursorFactory();
if (ReleaseControl.USE_UNSECURE_DATABASE) {
return new MySQLiteDatabase(android.database.sqlite.SQLiteDatabase.openDatabase(path, factory.getUnsecure(), flags));
}
return new MySQLiteDatabase(net.sqlcipher.database.SQLiteDatabase.openDatabase(path, password, factory.getEncrypted(), flags));
}
In robolectric test i set the USE_UNSECURE_DATABASE per reflection true

What is the best way of creating greenDAO DB connection only once for single run of application?

Currently I am creating the greenDAO DB connection in a class (which opens the connection in every static method) and using it wherever I need it. But I am not sure if it's the best way of doing it.
Can anyone suggest a better way of doing it?
My Code:
import com.knowlarity.sr.db.dao.DaoMaster;
import com.knowlarity.sr.db.dao.DaoMaster.DevOpenHelper;
import com.knowlarity.sr.db.dao.DaoSession;
import com.knowlarity.sr.db.dao.IEntity;
public class DbUtils {
private static Object lockCallRecord =new Object();
private DbUtils(){};
public static boolean saveEntity(Context context , IEntity entity){
boolean t=false;
DevOpenHelper helper=null;
SQLiteDatabase db=null;
DaoMaster daoMaster=null;
DaoSession daoSession =null;
try{
helper = new DaoMaster.DevOpenHelper(context, IConstant.DB_STRING, null);
db = helper.getReadableDatabase();
daoMaster = new DaoMaster(db);
daoSession = daoMaster.newSession();
//Some business logic here for fetching and inserting the data.
}catch (Exception e){
Log.e("saveEntity", e.getStackTrace().toString());
}finally{
if(daoSession!=null)daoSession.clear();
daoMaster=null;
if(db.isOpen())db.close();
helper.close();
}
return t;
}

Your approach causes the database to be loaded very often which is not necessary and may slow down your app significantly.
Open the database once and store it somewhere and request it from there if needed.
Personally I use a global DaoSession and local DaoSessions. The local DaoSessions get used where nothing should remain in the session cache (i.e. persisting a new object into the database, that is likely to be used only very infrequent or performing some queries which will load a lot of entities that are unlikely to be reused again).
Keep in mind that updating entities in a local DaoSession is a bad idea if you use the entity in your global session as well. If you do this the cached entity in your global session won't be updated and you will get wrong results unless you clear the cache of the global session!
Thus the safest way is to either just use one DaoSession or new DaoSessions all the time and to not use a global and local sessions!!!
A custom application class is a good place, but any other class will also be ok.
This is how I do it:
class DBHelper:
private SQLiteDatabase _db = null;
private DaoSession _session = null;
private DaoMaster getMaster() {
if (_db == null) {
_db = getDatabase(DB_NAME, false);
}
return new DaoMaster(_db);
}
public DaoSession getSession(boolean newSession) {
if (newSession) {
return getMaster().newSession();
}
if (_session == null) {
_session = getMaster().newSession();
}
return _session;
}
private synchronized SQLiteDatabase getDatabase(String name, boolean readOnly) {
String s = "getDB(" + name + ",readonly=" + (readOnly ? "true" : "false") + ")";
try {
readOnly = false;
Log.i(TAG, s);
SQLiteOpenHelper helper = new MyOpenHelper(context, name, null);
if (readOnly) {
return helper.getReadableDatabase();
} else {
return helper.getWritableDatabase();
}
} catch (Exception ex) {
Log.e(TAG, s, ex);
return null;
} catch (Error err) {
Log.e(TAG, s, err);
return null;
}
}
private class MyOpenHelper extends DaoMaster.OpenHelper {
public MyOpenHelper(Context context, String name, SQLiteDatabase.CursorFactory factory) {
super(context, name, factory);
}
#Override
public void onCreate(SQLiteDatabase db) {
Log.i(TAG, "Create DB-Schema (version "+Integer.toString(DaoMaster.SCHEMA_VERSION)+")");
super.onCreate(db);
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
Log.i(TAG, "Update DB-Schema to version: "+Integer.toString(oldVersion)+"->"+Integer.toString(newVersion));
switch (oldVersion) {
case 1:
db.execSQL(SQL_UPGRADE_1To2);
case 2:
db.execSQL(SQL_UPGRADE_2To3);
break;
default:
break;
}
}
}
In application class:
private static MyApplication _INSTANCE = null;
public static MyApplication getInstance() {
return _INSTANCE;
}
#Override
public void onCreate() {
_INSTANCE = this;
// ...
}
private DBHelper _dbHelper = new DBHelper();
public static DaoSession getNewSession() {
return getInstance()._dbHelper.getSession(true);
}
public static DaoSession getSession() {
return getInstance()._dbHelper.getSession(false);
}
Of course you can also store the DaoMaster instead of the DB itself. This will reduce some small overhead.
I'm using a Singleton-like Application class and static methods to avoid casting the application (((MyApplication)getApplication())) every time I use some of the common methods (like accessing the DB).

I would recommend to create your database in your Application class. Then you can create a method to return the DaoSession to get access to the database in other Activities.

How to make the data to be inside the database without the need to wait for the data to be inserted one by one?

How is it that some dictionaries such as merriam dictionary (Offline dictionary) when the application was installed , the words are there instantly, and time is not required to insert a list of words and definition into the database? I am a beginner and is currently developing an android application that consist of about 30K words and it will take around 15+ minutes for it to insert all the data into the database before the user can search for that particular data. And I am looking for a method that can fix this. Could someone please tell me a way to do it ?
Thank you

My guess is that these apps are using an already SQLite database with all the data they need already populated.
You can import populated databases to your app with something like this :
public class DataBaseAdapter {
String DB_NAME = "DBNAME.db";
String DIR = "/data/data/packageName/databases/";
String DB_PATH = DIR + DB_NAME;
private DataBaseHelper mDbHelper;
private SQLiteDatabase db;
private Context context;
public DataBaseAdapter(Context context) {
this.context = context;
mDbHelper = new DataBaseHelper(this.context);
}
class DataBaseHelper extends SQLiteOpenHelper {
private boolean createDatabase = false;
#SuppressWarnings("unused")
private boolean upgradeDatabase = false;
Context context;
public DataBaseHelper(Context context) {
super(context, DB_NAME, null, 1);
this.context = context;
}
public void initializeDataBase() {
getWritableDatabase();
if (createDatabase) {
try {
copyDataBase();
} catch (IOException e) {
throw new Error("Error copying database");
}
}
}
private void copyDataBase() throws IOException {
InputStream input = context.getAssets().open(DB_NAME);
OutputStream output = new FileOutputStream(DB_PATH);
byte[] buffer = new byte[1024];
int length;
try {
while ((length = input.read(buffer)) > 0) {
output.write(buffer, 0, length);
}
}
finally {
try {
if (output != null) {
try {
output.flush();
} finally {
output.close();
}
}
} finally {
if (input != null) {
input.close();
}
}
}
getWritableDatabase().close();
}
public void onCreate(SQLiteDatabase db) {
createDatabase = true;
}
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
upgradeDatabase = true;
}
public void onOpen(SQLiteDatabase db) {
super.onOpen(db);
}
}
public DataBaseAdapter open() {
mDbHelper.initializeDataBase();
if (db == null)
db = mDbHelper.getWritableDatabase();
return this;
}
public void close() {
db.close();
}
}
you can then add methods to get data from database and this class can be used in your activity by calling open then the method to get data then close.

Your application should include a pre-populated database for offline access with it's install. That will avoid each user having to run the INSERT step on their device.
Is there a particular reason you need to run the INSERTS post-install?