In Android Pie sqlite Write-Ahead logging (WAL) has been enabled by default. This is causing errors for my existing code only in Pie devices.
I have been unable to turn off WAL successfully using SQLiteDatabase.disableWriteAheadLogging() or PRAGMA journal_mode due to the way I access the database. I would like to disable WAL completely with an Android setting called db_compatibility_wal_supported :
Compatibility WAL (Write-Ahead Logging) for Apps
Does anyone know how to configure this? I don't know if this file can be altered programmatically at startup or if it is changed manually.
Further Details about the problem
I have a sqlite database (20mb+ / 250k records) in my app. This db is generated using plain java on my server. It contains a food database and the user of the app can add to the database (and the server is updated). This is stored in the assets folder in android.
During first installation the database is copied from assets to the app folder so that it can be written to, using effectively this method :
Copy SQLite database from assets folder
Unfortunately, once I start writing to the database using SqlDroid wal is enabled and the tables which were in the original db have vanished and only any newly created tables remain. The size of the database however is still 20mb+. All the database errors are due to the missing tables.
The table copying and writing method works perfectly in versions of Android prior to Pie.
The best and simplest way to disable WAL mode in your Database is as follows:
public class MyDbHelper extends SQLiteOpenHelper {
//...
#Override
public void onOpen(SQLiteDatabase db) {
db.disableWriteAheadLogging(); // Here the solution
super.onOpen(db);
}
//...
}
This way, all access to your database will be with WAL mode disabled. As much as you open and close multiple connections throughout the implementation of your App
If you are using Room, you won't have direct access to the database but you can instead set the journal mode when you are building/opening the database:
db = Room.databaseBuilder(context, Database.class, "Database")
.setJournalMode(JournalMode.TRUNCATE)
.build();
one cannot use SQLDroidDriver.ADDITONAL_DATABASE_FLAGS, simply because there is no constant available, which would negate flag ENABLE_WRITE_AHEAD_LOGGING.
WAL can still be disabled by creating either of these scenarios:
a) set flag OPEN_READONLY (applies to situations where R/O access does suffice).
b) run PRAGMA journal_mode=DELETE as the first query, in order to override PRAGMA journal_mode=WAL.
c) file an issue against SQLDroidConnection.java,
in order to have .enableWriteAheadLogging() and .disableWriteAheadLogging() supported on the driver-level.
#Rockvole please share error that you are facing, that help us to find appropriate solution.
Mean while, i understand that you want to close that WAL in android pie and you are using "SQLDroid" lib to create Sqlite DB.
This lib internally using "SQLiteDatabase" to store data locally, I think you need to call "SQLiteDatabase.disableWriteAheadLogging()" in "SQLiteDatabase" class where DB instance created the package name is "package org.sqldroid;"
or Get internal SQLiteDatabase instance and call disableWriteAheadLogging().
Second solution is create "config.xml" inside values folder and wirte "<bool name="db_compatibility_wal_supported">false</bool>" and run and check its work.
I finally found the answer. It seems that the database errors I was receiving is not directly related to WAL. It is because the widely used code to copy a database from assets has a bug in it where the database is left open during the copy operation. This only started causing a problem in Android P.
The solution is to close the database after you get the database file name.
SQLiteDatabase destinationDatabase = sqLiteOpenHelper.getWritableDatabase();
String dbFileName=destinationDatabase.getPath();
destinationDatabase.close();
// Now write the destinationDatabase using the dbFileName
This is detailed more here :
Android P - 'SQLite: No Such Table Error' after copying database from assets
Related
Android Room has method void clearAllTables() and according to docs it makes the following:
Deletes all rows from all the tables that are registered to this database as entities().
This does NOT reset the auto-increment value generated by autoGenerate().
After deleting the rows, Room will set a WAL checkpoint and run VACUUM. This means that the data is completely erased. The space will be reclaimed by the system if the amount surpasses the threshold of database file size.
I checked it in my project and looks like db has no data after that call, but when I pulled
*.db file from my device and opened it in sqlite viewer I've seen that all the data exists, the tables are filled and nothing has been erased.
How can that be? I consider this as a potential flaw in my app. Please provide a reliable approach of cleaning room database
looks like db has no data after that call
It means the method worked.
when I pulled *.db file from my device and opened it in sqlite viewer
I've seen that all the data exists
Most probably the transactions are not moved to the original database from the WAL file yet.
Solution
You can force a checkpoint using the wal_checkpoint pragma. Query the following statement against the database.
pragma wal_checkpoint(full)
Where does Room store the database and how can I force a rebuild of the DB? I've tried looking for the DB under:
data/data/com.me.myapp/No database directory here
data/user/0/com.me.myapp/No database directory here
I want to see exactly what data is in the database using SQLLite so I followed these directions "Access database in Android Studio" but I only see a cache and codecache directory stored there. No database directory.
The reason for wanting to see the DB is that I changed the model to add a few fields, but I can't figure out how to force Room to recreate and repopulate the DB with data. I added breakpoints inside my data generator class; however they don't seem to ever get hit.
if you change your entities structure (aka add fields) then in order not to loose data and get the fields added when user updates the current version of the app you need to implement migration . See the docs how to do it. So updating the app version will make your db "rebuild" and not lose data. When I just work with test version I delete app and build it on device again it'll implement the changes in db structure but you'll lose the data.
You should definitively see db files in the directories you named.
If you want some other method to debug a database you could use this library git link
I'm new to programing for android and i'm still learning. So I have a question about the location of the SQLite database. Is it stored in the same file system as the application ?
And also i'm not sure can the database be created before the app is installed(can it come with the app) or can the database only be created from inside the app ?
And what if i wanted my app to come with a database that already has tables and records which is local and not on a server. Would that be possible ?
SQLite is available on every Android device. Using an SQLite database in Android does not require any database setup or administration.
You only have to define the SQL statements for creating and updating the database. Afterwards the database is automatically managed for you by the Android platform.
Access to an SQLite database involves accessing the filesystem. This can be slow. Therefore it is recommended to perform database operations asynchronously, for example inside the AsyncTask class.
If your application creates a database, this database is by default saved in the directory DATA/data/APP_NAME/databases/FILENAME.
The parts of the above directory are constructed based on the following rules. DATA is the path which the Environment.getDataDirectory() method returns. APP_NAME is your application name. FILENAME is the name you specify in your application code for the database.
You can also follow this tutorial for further understanding.
http://www.androidhive.info/2011/11/android-sqlite-database-tutorial/
Database is created after the app installation or after the db changes. The db is stored in /data/data/your_package/databases/
My app uses a preloaded and copied from /assets to data/data ( Ship an application with a database) db which is simply a single table of product data and is 'read only' as users do not save to the DB. If I add more products to the DB table I need to get this to existing users. If I update my app with the new DB will the update process delete the old DB that was copied from the assets dir to data/data thereby allowing the 'DBexists' check to fail on first running the updated version thus triggering copying of the new DB from /assets to data/data?
Short answer, yes, if you put the following snippet it the onUpgrade() method:
try {
copyDataBase("database.db");
} catch (IOException e) {
Log.w(TAG, e);
}
It may be worth deleting the db file in copyDataBase() before writing over it, just to make it less likely to corrupt.
NB: this uses the implementation as used in the accepted answer of the question you linked.
Off the top of my head the only thing I can think of is to slightly abuse the onUpgrade() method in your SQLiteOpenHelper implementation, and change the database by making a call to this with a new version number and having it do whatever you need it to do in that method.
Saying that, I don't really know much about the Android app update process, so this could be way off.
I've been going through some tutorials on working with sqlite databases and they all seem to create a new database, tables, etc on the first run of the application. Is this necessary? What if I already have a pre-built database sitting in the assets folder when the application is installed? Can I not simply just open the connection to said database and start using its information or is there a specific reason everyone wants to create it using sql on first launch?
This question comes up frequently. Try this tutorial to use an existing database on Android:
http://www.reigndesign.com/blog/using-your-own-sqlite-database-in-android-applications/
You can't use database file which sits in assets folder directly as SQLite database, since this file would not be usual file located in common filesystem. E.g. you can have only readonly access to it. So the only your option is to copy those database from assets folder to device's filesystem.
To handle database creation for the first time and accessing it there's special helper class SQLiteOpenHelper. Read about it here. Specifically look in SQLiteOpenHelper.onCreate() - where should sit database creation (or copying from assets folder as in your case)
It's been quite sometime since I last worked with SQLite databases (in Android) but I believe that when they write CREATE statements, they always do so with the IF NOT EXISTS condition (i.e., CREATE (DATABASE|TABLE) IF NOT EXISTS...).
I don't know what you'll use SQLite for but I believe they do that in Android "just to make sure". That is, if it's the user's first time to run the app, the DB/Tables must be created first else app goes bonkers. Otherwise, they are (probably) created already and this case will be handled by the IF NOT EXISTS clause and they just go ahead and create a connection with the existing DB. Win-win.
(If, for some reason, it is not the user's first time to use the app and the DB isn't there, it will just be created again. But that's obvious isn't it? ;) )
If you just link with preexisting database it wont bind to your system. So there may be failures. Creating db at first run is the most appropriate way to work with db.