Develop Reference

SQLite database not copied correctly in some Android devices

I've developed an app that uses a SQLite database. It's copied into the data folder of the app on a fresh install and each time the app is updated (since the database is read-only I overwrite the last version).
This is the code I'm using:
public class IfcDatabase extends SQLiteOpenHelper {
private static final Object lock = new Object();
private static volatile IfcDatabase mInstance;
private SQLiteDatabase db = null;
private Context ctxt;
public static final int DATABASE_VERSION = 22;
public static final String DATABASE_NAME = "mydatabase.sqlite";
public static String DB_PATH = "";
public static IfcDatabase getInstance(Context ctx) {
IfcDatabase r = mInstance;
if (r == null) {
synchronized (lock)
{ // While we were waiting for the lock, another
r = mInstance; // thread may have instantiated the object.
if (r == null)
{
DB_PATH = ctx.getDatabasePath(DATABASE_NAME).getParent()+"/";
Log.e("logs", "DB_PATH="+DB_PATH);
r = new IfcDatabase(ctx.getApplicationContext());
mInstance = r;
}
}
}
return r;
}
private IfcDatabase(Context context) {
//super(context, DATABASE_NAME, null, DATABASE_VERSION);
super(context, DB_PATH+DATABASE_NAME, null, DATABASE_VERSION);
ctxt = context;
try {
if (!existsDatabase()) {
SQLiteDatabase tempDb = getWritableDatabase();
Log.e("logs", "Just before copying database");
copyDatabase();
Log.e("logs", "Just after copying database");
}
} catch (SQLException eSQL) {
Log.e("logs", "Error: Can not open database");
} catch (IOException IOe) {
Log.e("logs", "Error: Can not copy initial database");
}
}
private boolean existsDatabase() {
File dbFile = new File(DB_PATH + DATABASE_NAME);
return dbFile.exists();
}
public void copyDatabase() throws IOException {
AssetManager _asset = ctxt.getAssets();
InputStream myInput = _asset.open(DATABASE_NAME);
String outFileName = DB_PATH + DATABASE_NAME;
Log.e("logs", "OUTFILENAME="+outFileName);
OutputStream myOutput = new FileOutputStream(outFileName);
byte[] buffer = new byte[1024];
int length;
while ((length = myInput.read(buffer)) > 0) {
myOutput.write(buffer, 0, length);
}
myOutput.flush();
myOutput.close();
myInput.close();
}
#Override
public void onCreate(SQLiteDatabase db) {
}
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
public void openDatabase() {
db = getWritableDatabase();
Log.e("logs", "Checking if table exists (checking if database has been successfully copied)");
if(!checkIfTableExists("mytable"))
Log.e("logs", "At this moment this table doesn't exist");
else
Log.e("logs", "At this moment this table exists");
Log.e("logs", "OPEN BBDD "+db);
}
private boolean checkIfTableExists(String table_name)
{
Cursor _cursor = db.rawQuery("SELECT name FROM sqlite_master WHERE type='table' AND name='"+table_name+"'", null);
int count = _cursor.getCount();
_cursor.close();
if(count > 0)
return true;
else
return false;
}
public void closeDatabase() {
if (db != null)
db.close();
}
...
}
When the app is started AND when the user logs in it calls:
IfcDatabase.getInstance(this).openDatabase();
And only when the user logs out it calls:
IfcDatabase.getInstance(this).closeDatabase();
On a fresh installation openDatabase is called, database is copied from the assets to the data folder and the user can start making queries to the database. These are the logs:
DB_PATH=/data/user/0/com.mypackage.myapp/databases/
Just before copying database
OUTFILENAME=/data/user/0/com.mypackage.myapp/databases/mydatabase.sqlite
Just after copying database
Checking if table exists (checking if database has been successfully copied)
At this moment this table exists
OPEN BBDD SQLiteDatabase: /data/user/0/com.mypackage.myapp/databases/mydatabase.sqlite
Well, this is happening (this is working correctly) in almost all cases but in some specific Android models (Google Pixel and some ZTE). In these cases, it seems that this code doesn't copy correctly the database. In these cases, on a fresh installation these are the logs:
DB_PATH=/data/user/0/com.mypackage.myapp/databases/
Just before copying database
OUTFILENAME=/data/user/0/com.mypackage.myapp/databases/mydatabase.sqlite
Just after copying database
Checking if table exists (checking if database has been successfully copied)
At this moment this table doesn't exist
As you can see, it seems that the database is empty or doesn't exist. And when the user makes a query the app crashes with this exception:
java.lang.IllegalStateException: attempt to re-open an already-closed object: SQLiteDatabase: /data/user/0/com.mypackage.myapp/databases/mydatabase.sqlite
So, is it possible that Google Pixel and ZTE use the SQLite databases in a different way and my code doesn't work in those cases?
Thank you very much and kind regards!
--- EDIT ---
I should add that I've been thinking if this could be a problem with the permissions. I request the Write external storage permission, if the user denies it I ask for a confirmation and if the user keeps denying it, then the user can not use the app. But I'm almost sure that it's not related to my problem.
--- NEW EDIT ---
I've find out a new thing: it seems that is not just a Google Pixel problem but a Google Pixel WITH Android P. This worries me a lot because if it's an Android P related problem it will happen in all devices in the future.
Currently I have no devices that can install Android P and reading the specifications of Android P I cannot see anything related to databases or internal files or something like that.
Does anyone know if Android P has changed anything related to databases, assets, data folder, etc..?

Maybe you can solve it by adding this to your SQLiteOpenHelper
#Override
public void onOpen(SQLiteDatabase db) {
super.onOpen(db);
db.disableWriteAheadLogging();
}

Replacing database with another database at runtime.

I have two databases, one database is the primary. This primary DB is responsible for holding the current data which is up to date and my secondary DB is populated via a cron job, once the primary DB gets obsolete I want to replace it with the secondary DB via a file operation of just over writing the existing DB and refreshing my views. Is it possible to do this, is there a better way?
So far what I have done is:
public void writeToSD() throws IOException {
File f=new File("/mnt/sdcard/dump.db");
FileInputStream fis=null;
FileOutputStream fos=null;
try{
fis=new FileInputStream(f);
fos=new FileOutputStream("/data/data/com.one.two/databases/Bdr");
while(true){
int i=fis.read();
if(i!=-1){
fos.write(i);
}
else{
break;
}
}
fos.flush();
}
catch(Exception e){
e.printStackTrace();
}
finally{
try{
fos.close();
fis.close();
}
catch(IOException ioe){
System.out.println(ioe);
}
}

How about always using the same database files (let's say dbA, dbB) with two instances of SQLiteOpenHelper and using an utility class like this instead of using raw SQLiteOpenHelper:
class Db {
private SQLiteOpenHelper mPrimaryDb;
private SQLiteOpenHelper mSecondaryDb;
public Db(Context context) {
mPrimaryDb = new MyDbHelper(context, "db_a");
mSecondaryDb = new MyDbHelper(context, "db_b");
}
public SQLiteOpenHelper getPrimaryDb() {
return mPrimaryDb;
}
public SQLiteOpenHelper getSecondaryDb() {
return mSecondaryDb;
}
public void swapDb() {
SQLiteOpenHelper tmp = mPrimaryDb;
mPrimaryDb = mSecondaryDb;
mSecondaryDb = tmp;
// TODO: notify data users that data has changed, cleanup the old primary database, etc.
}
{

If you want to use file operations, renaming the data base files is faster. But during file operations all connections have to be closed before any action.
If insertion is too slow, I would not overwrite the database file. I would generate the new database with a temp name and the same table and view structure. After finishing writing to the temp file I would rename the file to the same name as the invariant part of the old database plus a version number or a timestamp . And in my application I would look periodically for a new version, if found I would close all connections to the old file and open the new database.

Use prepopulated SQLite database in android while using exisntent android API

I wan't to use database file, that I've created on my computer.
While this is working, I'm considering that as a bad workaround, as it's not using existent API, while creating its own api.
I want to be able to use getWritableDataBase(), onCreate(SQLiteDatabase db) and onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) methods.
So, that's what I did and it's not working for some reason. I thought that if I would rewrite existent database it would work, but when querying I'm getting table not exists exception.
public void onCreate(SQLiteDatabase db) {
File f = new File(db.getPath());
try {
InputStream is = context.getResources().getAssets().open("words1.db");
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
FileOutputStream fos = new FileOutputStream(f);
fos.write(buffer);
fos.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
So how could I use prepopulated database, while using existent api?

The easiest way to pre-package a database with your application is to use Jeff Gilfelt's SQLiteAssetHelper. While it does require you to add a small JAR to your project, his code is tested and debugged, and it requires very little additional code on your part.
Here is a sample project demonstrating the use of SQLiteAssetHelper.

Performance SQLite Issue - Can a codechange speed up things?

I use the following code to add rows to my database :
public void insert(String kern, String woord) {
SQLiteDatabase db = getWritableDatabase();
ContentValues values = new ContentValues();
values.put(KERN, kern);
values.put(WOORD, woord);
db.insertOrThrow(TABLE_NAME, null, values);
return;
Currently, I'm invoking this insert() 3.455 times, to add all words to the database, using : insert("Fruits", "Banana"); It takes forever.
How can I change this code to work faster? I'm thinking in the line of foreach, but don't know how to implement.. Thanks!
/Edit; The solution provided by #hovanessyan works and will do the job. AND.. note that if you have a lot of lines that have to be put in, you might be confronted with the method exceeds max byte limit error. In that case, review the other solution, that suggests packing the database in the actual .APK file.

You can wrap-up those inserts into transaction.
db.beginTransaction();
try {
// do all the inserts here
//method call here, that does 1 insert; For example
addOneEntry(kern,woord);
...
db.setTransactionSuccessful();
} catch (SQLException e) {
//catch exceptions
} finally {
db.endTransaction();
}
private void addOneEntry(String kern, String woord) {
//prepare ContentValues
//do Insert
}

You can use bulkInsert:
ContentValues[] cvArr = new ContentValues[rows.size()];
int i = 0;
for (MyObject row : rows) {
ContentValues values = new ContentValues();
values.put(KERN, myObject.getKern());
values.put(WOORD, myObject.getWoord);
cvArr[i++] = values;
}// end for
resolver.bulkInsert(Tasks.CONTENT_URI, cvArr);

Using the tips of both hovanessyan and Damian (remind me to rep+1 you as soon as I reach 15 ;), I came up with the following solution:
For relatively small databases (<1,5Mb)
I created the database using SQLite Database Browser, and put it in my Assets folder.
Then, the following code copies the database to the device, if it's not already there:
boolean initialiseDatabase = (new File(DB_DESTINATION)).exists();
public void copyDB() throws IOException{
final String DB_DESTINATION = "/data/data/happyworx.nl.Flitswoorden/databases/WoordData.db";
// Check if the database exists before copying
Log.d("Database exist", "" + initialiseDatabase);
Log.d("Base Context", "" + getBaseContext());
if (initialiseDatabase == false) {
// Open the .db file in your assets directory
InputStream is = getBaseContext().getAssets().open("WoordData.db");
// Copy the database into the destination
OutputStream os = new FileOutputStream(DB_DESTINATION);
byte[] buffer = new byte[1024];
int length;
while ((length = is.read(buffer)) > 0){
os.write(buffer, 0, length);
}
os.flush();
os.close();
is.close();
}}
In my app, a portion of the database is User-customizable.
I call the code above in onStart() with :
try {
copyDB();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
So, when the user presses "reset database to standard" (in preferences screen), I just set the Boolean initialiseDatabase to "false" and wait for the user to go back to the main activity. (thus calling onstart and copying the original database).
I tried to call the Activity.copyDB() from the preferences.java. It's neater, because it doesn't require the user to go back to the main activity to rebuild the database. However, I get an error about not being able to call static references to non-static methods. I don't understand that, but will look into it.

How can I embed an SQLite database into an application?

I think I have some basic understanding problem so maybe someone's able to help :-)
I'm developing an Android application using Eclipse and this application will make use of a database (only reading from the database will be implemented). The database contains around 4,000 entries i.e. creating and populating the database via source code is not an option. Thus I have created the database in advance with all its records.
But how can I "embed" this database file into my application and then access it? The databse will be around 500 kB in file size. Downloading from a remote server is not an option either as this is not allowed.
Thanks,
Robert

I solved that problem by:
adding file.db into project/assets folder;
writing next class:
public class LinnaeusDatabase extends SQLiteOpenHelper{
private static String DATABASE_NAME = "Dragonfly.db";
public final static String DATABASE_PATH = "/data/data/com.kan.linnaeus/databases/";
private static final int DATABASE_VERSION = 1;
private SQLiteDatabase dataBase;
private final Context dbContext;
public LinnaeusDatabase(Context context) {
super(context, DBActivity.DatabaseName, null, DATABASE_VERSION);
this.dbContext = context;
DATABASE_NAME = DBActivity.DatabaseName;
// checking database and open it if exists
if (checkDataBase()) {
openDataBase();
} else
{
try {
this.getReadableDatabase();
copyDataBase();
this.close();
openDataBase();
} catch (IOException e) {
throw new Error("Error copying database");
}
Toast.makeText(context, "Initial database is created", Toast.LENGTH_LONG).show();
}
}
private void copyDataBase() throws IOException{
InputStream myInput = dbContext.getAssets().open(DATABASE_NAME);
String outFileName = DATABASE_PATH + DATABASE_NAME;
OutputStream myOutput = new FileOutputStream(outFileName);
byte[] buffer = new byte[1024];
int length;
while ((length = myInput.read(buffer))>0){
myOutput.write(buffer, 0, length);
}
myOutput.flush();
myOutput.close();
myInput.close();
}
public void openDataBase() throws SQLException {
String dbPath = DATABASE_PATH + DATABASE_NAME;
dataBase = SQLiteDatabase.openDatabase(dbPath, null, SQLiteDatabase.OPEN_READWRITE);
}
private boolean checkDataBase() {
SQLiteDatabase checkDB = null;
boolean exist = false;
try {
String dbPath = DATABASE_PATH + DATABASE_NAME;
checkDB = SQLiteDatabase.openDatabase(dbPath, null,
SQLiteDatabase.OPEN_READONLY);
} catch (SQLiteException e) {
Log.v("db log", "database does't exist");
}
if (checkDB != null) {
exist = true;
checkDB.close();
}
return exist;
}
}

Nice article on ReignDesign blog titled Using your own SQLite database in Android applications. Basically you precreate your database, put it in your assets directory in your apk, and on first use copy to "/data/data/YOUR_PACKAGE/databases/" directory.

I've done this in the past by storing a JSON file in the application in the res/raw resources and then loading the file on first load. From there, you can use the bulk insert mode to batch-import entries. See my database loader for Units as an example of this technique. One benefit of the res/raw style is that you can use the Android resource selecting system to localize the data to different regions, so you could have a different database (or part thereof) for different languages or regions.
You could also put a raw SQL dump in a similar file and load that on first load. You can get the file from the resources by using openRawResource(int). I'd recommend this instead of storing a pre-made sqlite database file to increase compatibility between versions of sqlite, as well as make maintaining the database easier (from an app-development lifecycle POV).
When loading things in bulk, make sure to use transactions, as that'll help speed things up and make the loading more reliable.

In a comment to this answer, CommonsWare recommends against the same method proposed by #evilone here (ie, copying the database from assets yourself). Instead you should use the tried and tested SQLiteAssetHelper. I've used it in Android Studio and it works well.
The directions are fairly clear. Here are a few key excerpts:
If you are using the Gradle build system, simply add the following
dependency in your build.gradle file:
dependencies {
compile 'com.readystatesoftware.sqliteasset:sqliteassethelper:+'
}
Extend SQLiteAssetHelper as you would normally do SQLiteOpenHelper,
providing the constructor with a database name and version number:
public class MyDatabase extends SQLiteAssetHelper {
private static final String DATABASE_NAME = "northwind.db";
private static final int DATABASE_VERSION = 1;
public MyDatabase(Context context) {
super(context, DATABASE_NAME, null, DATABASE_VERSION);
}
}
Put your database in assets/databases/northwind.db (using whatever database name you have).
The database is made available for use the first time either
getReadableDatabase() or getWritableDatabase() is called.