Develop Reference

How to totally fix SQLite "no such table error" on Android 9.0 in Upgrade

I have also met sqlite "no such table" error on Android 9.0, I have found there is a similar issue on SO. (Android P - 'SQLite: No Such Table Error' after copying database from assets).
Base on above link, I tried below solutions 1 and 2.
1) Close connection after read
private void createDataBase() throws IOException {
this.getReadableDatabase();
this.close();
try {
copyDataBase();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
2) Disable WAL.
#Override
public void onConfigure(SQLiteDatabase db) {
Logger.i(TAG, "onConfigure()");
super.onConfigure(db);
db.disableWriteAheadLogging();
}
private boolean checkIfDBExists() {
File dbFile = new File(DB_PATH + DB_NAME);
return dbFile.exists();
}
public void openDataBase() throws SQLException {
String myPath = DB_PATH + DB_NAME;
myDataBase = SQLiteDatabase.openDatabase(myPath, null,SQLiteDatabase.OPEN_READWRITE);
}
After that, I found a problem, These solutions are only fine when I remove installed App, and fresh install App.
When I have installed a buggy App with "no such table error" on Android 9.0 device, Then I add above fixed code, and install it directly with Android Studio which looks like an App upgrade scenario.
In this case, The db file will be over there, checkIfDBExists() will return true, then we will call openDataBase() instead of createDataBase() The sqlite "no such table" error will still happen, So How can I totally fix this issue no matter fresh install or upgrade? Thanks in advance.

After trying several times, I have found a solution to fix this issue.
In databases folder, there are some temp files xxx.db-shm, xxx.db-wal, These files should be generated when enable WAL, so even after we disable WAL later, they are already there which will affect db query.
So the solution is simple, delete these files when you disable WAL.

Reading SQL from file and SQLiteOpenHelper

I need to include an existing SQLiteDatabase in my Android app and I also want the ability to download and install a new db. I did some research and my first working solution came from here. I didn't like that solution, for one thing, it assumes that the database will always be in a fixed path and other oddities.
So, rather than putting the existing database file in the assets, I exported the database to a SQL file, read it in, and then in the onCreate() method of my SQLiteOpenHelper, I call a new method updateDatabase with an open DataInputStream with the file data.
I ran into a few issues, which I think I solved, but I'm sure I didn't think of all the issues, which are:
When SQLiteOpenHelper's onCreate method is called, the database has been created, it is open, and inTransaction() is true. As a result, if the imported sql file includes BEGIN TRANSACTION, an exception is thrown and if the sql String contains statements creating 'android_metadata' yet another exception. So, I added a simple search using String.contains() looking for these keywords, and set a boolean doExecute to false to avoid executing them. So a question is, is there a better SQL class or method to filter this, or even a better regexp method?
Similar issue with having unexpected line breaks in the SQL file. I read the file with readLine() and to look for line breaks, I simply use String.trim() on the line, then check for endsWith(";"). This puts some constraints on my input file, like not having multiple statements on a single line. So, is there a better way to pre-process SQL from a file?
Here's the code I use to create my db after I've gotten a DataInputStream from the assets resource or from a download:
public boolean updateDatabase(DataInputStream inStream, SQLiteDatabase db, boolean doClear) throws Error {
String sqlStatement = null;
boolean result = true;
boolean inOnCreate = true;
boolean wasInTransaction;
if(doClear) dropDatabase();
// if called from onCreate() db is open and inTransaction, else getWritableDatabase()
if(db == null) {
inOnCreate = false;
db = this.getWritableDatabase();
}
wasInTransaction = db.inTransaction(); // see NB below
boolean doExecute;
try {
while ((sqlStatement = inStream.readLine()) != null) {
// trim, so we can look for ';'
sqlStatement.trim();
if(!sqlStatement.endsWith(";")) {
continue; // line breaks in file, get whole statement
}
// NB - my file (exported from SQLite Database Browser starts with "BEGIN TRANSACTION;".
// executing this throws SQLiteException: cannot start a transaction within a transaction
// According to SQLiteDatabase doc for beginTransaction(), "Transactions can be nested"
// so this is a problem
// but... possibly it is an "exclusive transaction" ?
doExecute = true;
if(wasInTransaction) {
// don't execute BEGIN TRANSACTION; or COMMIT;
if((sqlStatement.contains("BEGIN" ) || sqlStatement.contains("begin" )) &&
(sqlStatement. contains("TRANSACTION") || sqlStatement.contains("transaction" ))) {
doExecute = false;
}
if(sqlStatement.contains("COMMIT") || sqlStatement.contains("commit")) {
doExecute = false;
}
} // inTransaction
// this statement could be in older databases, but this scheme doesn't need, can't have it
if(sqlStatement.contains("android_metadata")) {
doExecute = false;
}
if(doExecute) {
try {
db.execSQL(sqlStatement);
} catch (SQLException e) {
throw(new Error("Error executing SQL " + sqlStatement));
} // try/catch
} // doExecute
} // while()
} catch (IOException e) {
result = false; // which won't matter if we throw
throw(new Error("Error reading " + DB_SQL));
}
if(!inOnCreate) {
db.close();
}
return result;
}

Wouldn't want to have you cop out early on such ambitious and elegant implementations, but if you have your database already made and checked with a database browser and all, have you considered SQLite Asset Helper? If your main issue was being forced to use the asset folder, this method lib let's you use a file from any specified directory. Moreover, it allows for handling the raw .db file. Worth checking out.

Robolectric - Error Copying SQLite Database

I am currently testing the Android API, and to do so, I have developed an application that copies a database that I have made beforehand and testing that application (which calls on several Android classes). Someone directed me to using Robolectric to get coverage of the Android source, however, this has caused some problems. My application runs just fine without it, as well as my test, but now I'm running into errors with the copying of my database. Whenever I run the test, my catch gives me an error from copying the database, and taking out that catch results in an AssertionFailureError
assertTrue(activity.accessAdapter() != null);
assertTrue(activity.accessAdapter().accessHelper().checkDatabase()); // Here
I'm assuming that has something to do with not getting the database copied, as if I put back in the try-catches I had in my code, it results back in setting them off when copying the database.
Here's the code for my SQLiteOpenHelper, or at least what gets called up in copying the database.
public void createDatabase() throws IOException
{
this.getReadableDatabase();
this.close();
try {
copyDatabase();
Log.e(TAG, "Database created.");
} catch (IOException e) {
throw new Error("Error copying database.");
}
}
private void copyDatabase() throws IOException {
InputStream input = ((ShadowContextWrapper) shadowContext).getAssets().open(DATABASE_NAME);
String outFileName = DATABASE_PATH + DATABASE_NAME;
OutputStream output = new FileOutputStream(outFileName);
byte[] buffer = new byte[1024];
int length;
while ((length = input.read(buffer)) > 0) output.write(buffer, 0, length);
output.flush();
output.close();
input.close();
}
I assumed the problem might have to do with the context, so I created a ShadowContext, but that didn't really help either. There are other errors, but those are just methods that call each other up all the way down to the createDatabase() in my SQLiteOpenHelper.
Does anyone know how I can copy this database using Robolectric? I have it saved in assets for my application, but obviously, without the emulator, this is pointless. Thank you.
P.S. I can add any more code if necessary, this is just what I thought was necessary at first.

Performance SQLite Issue - Can a codechange speed up things?

I use the following code to add rows to my database :
public void insert(String kern, String woord) {
SQLiteDatabase db = getWritableDatabase();
ContentValues values = new ContentValues();
values.put(KERN, kern);
values.put(WOORD, woord);
db.insertOrThrow(TABLE_NAME, null, values);
return;
Currently, I'm invoking this insert() 3.455 times, to add all words to the database, using : insert("Fruits", "Banana"); It takes forever.
How can I change this code to work faster? I'm thinking in the line of foreach, but don't know how to implement.. Thanks!
/Edit; The solution provided by #hovanessyan works and will do the job. AND.. note that if you have a lot of lines that have to be put in, you might be confronted with the method exceeds max byte limit error. In that case, review the other solution, that suggests packing the database in the actual .APK file.

You can wrap-up those inserts into transaction.
db.beginTransaction();
try {
// do all the inserts here
//method call here, that does 1 insert; For example
addOneEntry(kern,woord);
...
db.setTransactionSuccessful();
} catch (SQLException e) {
//catch exceptions
} finally {
db.endTransaction();
}
private void addOneEntry(String kern, String woord) {
//prepare ContentValues
//do Insert
}

You can use bulkInsert:
ContentValues[] cvArr = new ContentValues[rows.size()];
int i = 0;
for (MyObject row : rows) {
ContentValues values = new ContentValues();
values.put(KERN, myObject.getKern());
values.put(WOORD, myObject.getWoord);
cvArr[i++] = values;
}// end for
resolver.bulkInsert(Tasks.CONTENT_URI, cvArr);

Using the tips of both hovanessyan and Damian (remind me to rep+1 you as soon as I reach 15 ;), I came up with the following solution:
For relatively small databases (<1,5Mb)
I created the database using SQLite Database Browser, and put it in my Assets folder.
Then, the following code copies the database to the device, if it's not already there:
boolean initialiseDatabase = (new File(DB_DESTINATION)).exists();
public void copyDB() throws IOException{
final String DB_DESTINATION = "/data/data/happyworx.nl.Flitswoorden/databases/WoordData.db";
// Check if the database exists before copying
Log.d("Database exist", "" + initialiseDatabase);
Log.d("Base Context", "" + getBaseContext());
if (initialiseDatabase == false) {
// Open the .db file in your assets directory
InputStream is = getBaseContext().getAssets().open("WoordData.db");
// Copy the database into the destination
OutputStream os = new FileOutputStream(DB_DESTINATION);
byte[] buffer = new byte[1024];
int length;
while ((length = is.read(buffer)) > 0){
os.write(buffer, 0, length);
}
os.flush();
os.close();
is.close();
}}
In my app, a portion of the database is User-customizable.
I call the code above in onStart() with :
try {
copyDB();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
So, when the user presses "reset database to standard" (in preferences screen), I just set the Boolean initialiseDatabase to "false" and wait for the user to go back to the main activity. (thus calling onstart and copying the original database).
I tried to call the Activity.copyDB() from the preferences.java. It's neater, because it doesn't require the user to go back to the main activity to rebuild the database. However, I get an error about not being able to call static references to non-static methods. I don't understand that, but will look into it.

SqliteOpenHelper for Mode_World_Readable, how?

SqliteOpenHelper by default creates database in mode_private. How can we create world readable/writable db using SqliteOpenHelper ?
Or Else Do I need to use Context.openOrCreateDatabase()

How can we create world readable/writable db using SqliteOpenHelper ?
We can't do that. ContextImpl.openOrCreateDatabase() actually opens/creates database using SQLiteDatabase.openOrCreateDatabase() method and then sets the permission for the database file using class android.os.FileUtils which is not part of the public API. So unless you want to use reflection the only possible way to make database world-readable/-writable is to use Context.openOrCreateDatabase().

Opening a Database as world Readable/writable is definitely possible. But then what is the necessity of a Database? You can use files instead..!!
Opening a Database as world Readable/writable is not recommended.
Always remember this:
Open a database only when necessary,
because it is costly.
Open only in the mode necessary either
read or write or both.
Close it as soon as the manipulations
are over.
If you want to share a Database or a resource among your applications you can use SharedUserID. Inorder to use SharedUserID, the applications must be signed by the same Key.
For More info see my post here at sree.cc
http://sree.cc/google/android/sharing-resources-in-different-aplications-using-shareduserid
Here is the code Snippet for the same:
private void getDB() {
//accessing file using SHAREDUSERID
try
{
//creating context from mainAPP for accessing database
ctx = createPackageContext(
"com.schogini.sharedDB.pack",
Context.CONTEXT_IGNORE_SECURITY);
if(ctx==null){
return;
}
}
catch (PackageManager.NameNotFoundException e) {
//package not found
Log.e("Error",e.getMessage());
}
try
{
File myDbFile = ctx.getDatabasePath("sharedDB.db");
if (myDbFile.exists())
{
dbb = openOrCreateDatabase(myDbFile.getPath(), SQLiteDatabase.OPEN_READWRITE, null);
dbb.setVersion(1);
dbb.setLocale(Locale.getDefault());
dbb.setLockingEnabled(true);
try{
cur=dbb.rawQuery("select * from TABLENAME;",null);
try{
cur.moveToFirst();
int k=cur.getColumnCount();
lv_arr=new String[k];
for(int i=0;i<k;i++)
{
lv_arr[i]=""+cur.getString(i);
Toast.makeText(LaunchActivity.this, "Data "+i, Toast.LENGTH_SHORT).show();
}
}
catch(Exception e)
{
//may be an empty database
Log.e("Error",e.getMessage());
dbb.close();
}
}
catch(Exception e)
{
Log.e("Error",e.getMessage());
dbb.close();
}
}
else
{
//database not found
Toast.makeText(LaunchActivity.this, "DataBase Doesnot Exist", Toast.LENGTH_SHORT).show();
}
}
catch(Exception e)
{
Log.i("\n\nTAG",e.toString());
}
}
Not recommended:
If you want the word readable, then create it world readable. Use openFileOutput() or openOrCreateDatabase(). Declare the context that creates the DB as World readable. Note this method is not safe by any means.
Do a reference here.
http://developer.android.com/reference/android/content/Context.html#MODE_WORLD_READABLE