I have been trying to write a lite program version for android that also syncs with a microsoft access datatbase. I first developed the database in access because it was easier and now i want to migrate and synchronize to android with a SQLite backup/synced file.
When i simply replace the SQLite file on an emulated device through the Device Explorer it works fine. When i try to replace the SQLite file on a physical device through the Device Explorer it doesn't work. Every time it replaces the file with a new version and relegates the old database with a .corrupt extension.
I am new to Android Studio and am using Kotlin. I see a lot of answers for Android Studio are in Java which might confuse me. My eventual plan is to synchronize the databases to an SQLite database cloud file and keep a synchronized copy on each device. So the basic problem is between the emulated device and the physical device.
My development environment is Ubuntu. My system is too slow to run in Windows.
One cause may be that you are replacing one file but not another 2 files.
With later versions of Android (9+ see), the default mode changed from journal mode to WAL (Write-Ahead Logging).
In journal mode the journal keeps a record of changes applied to the database, so it is not important as far as the actual database file is concerned.
In WAL mode changes are written to the wal file (database file name suffixed with -wal and -shm (a wal file for the wal file)). If you replace the database file but leave the -wal file and the -wal file is not empty then sqlite will see that the -wal file is not the one for the database file (it belongs to the now overwritten/replaced database) and thus will consider that the database is corrupt, as per :-
If a database file is separated from its WAL file, then transactions that were previously committed to the database might be lost, or the database file might become corrupted. https://sqlite.org/wal.html#the_wal_file
The fix, if this is the issue, would be to delete the -wal file and the -shm file as well as overwriting/replacing the actual database file.
Related
In my android studio in data/data/com.../databases there are 3 files .db, shm.db , wal.db and when i save that .db file it is not showing any table.
While in my another mobile phone files there are only two .db, journal.db and it is showing the table and data when we save it .
i don't know why it is happening ???
If you want to check the database contents, you need to downlaod all those 3 files, store them together in a folder and then you would be able to check the contents of your db.
Your problem:
You are only trying to download the .db extension file. Thats why the issue is coming!
On later versions of Android SQLite Write Ahead logging is enabled by default.
Write ahead logging creates the Wal file, see https://sqlite.org/wal.html for more details.
When the database is closed then the contents of the Wal file will be committed to the main DB file, there is also automatic checkpointing and you can trigger it manually.
It is possible to disable it on a per connection basis, but if data does not get moved to the main db file when you application has been closed then your code is not closing the database properly.
Last time I notice that my app has a problem with SQLite database backup. To do db backup I was copying *.db file from /data/data/com.pkg.app/databases/db_name.db to internal storage. To restore I copying from internal storage to /data/data/com.pkg.app/databases/db_name.db. But in Android P There is something wrong. Those db file exists but its weight is 4kB. When I opened those file by DB Browser I noticed that file is empty - there is no tables and data. It is strange because app is working correctly. The question is where should I look for db file that my app is using? Android 8 and less has no this problem. I am using emulator.
In Android P+ WAL (Write ahead logging) is turned on by default. This results in two additional files that suffix the original file name with -shm (shared memory file) and -wal (the write ahead logging).
These either need to be copied when backing up and restored or you need to backup only after ensuring that the database has been fully checkpointed, you can then delete the -shm and -wal files as part of the restore process.
An alternative is to disable WAL and use the less efficient journal mode.
You may wish to check out Database Import and Export not working in Android Pie.
I have a pre-populated, read only SQLite database file I am shipping as an Asset with my Android App. Changes to the read only data constitute a version release/update in the app store. Thus, the DB file (and currently the journal file) are being stored in version control (TFS).
The question:
Is there any advantage to storing the -journal file in version control? After everything I read about SQLite, my understanding is it is basically just a simple transaction log for a file and there's no database engine in play (like with MSSQL Server). In effect, TFS is keeping my read only data Versioned, so I see no reason to check-in the journal file or have it be part of my project / assets at all. Am I missing anything?
As long as there is not a transaction that has not yet been committed or rolled back, the journal file is empty.
So it should never be required to have it in version control.
(The default journal mode is DELETE, so unless you have changed this setting, you should investigate how this file ended up in TFS in the first place.)
I know about copying the database file solutoin,
How do I backup a database file to the SD card on Android?
Android backup/restore: how to backup an internal database?
Android backup/restore: how to backup an internal database?
in fact I was backing up my databases using this method, and up until a little while ago it was working perfectly.
until some multi-thread stuff made me use of enableWriteAheadLogging
now I have two more files near the database file with .db-wal and .db-shm extensions.
copying just the datbase file .db is not working as most of the times the file does not contain the latest database commits (which are available throw app itself) however when I copy three files together it seems to work fine (not quit sure, albeit)
as Sqlite people recommend, the best practice for backing up a sqlite database is to use backup api but can someone guid me on how I can use this api from inside an application, or even use sqlite .dump (How do I dump the data of some SQLite3 tables?) method from inside an app ?
So Which one is the best practice to back up a sqlite database from an android app?
1- Copy all the database-related files from the sandbox
2- Use sqlite Backup Api
3- Use sqlite .dump
4- any other method
The SQLite backup API is the only mechanism that works correctly with concurrent write accesses from other threads/processes. Which of course means that the Android database framework does not give you access to it.
If you are sure that there are not any active connections, you can just copy all files. (Leaving out journal or WAL files would lead to data corruption.) In the case of WAL, the -shm file
does not contain any permanent data and could be omitted.
The VACUUM INTO command introduced in SQLite version 3.27.0 (2019-02-07) can serve as an alternative to the backup API.
I want to rename database files. The tasks seems to be easy.
1) I close database 2) rename .db file 3) rename .db-journal file 4) I open database
However what will happen if ex. after the step number 2 and before the step number 3 the system will crash (I know the probabiliby is low, but...)?
I noticed that depending on the Android version sometimes after closing the database .db-journal file has 0 size (ex. android 4.0.4) but sometimes it's quite huge (ex. 50% of .db file size) (ex. android 4.2.2). If size is 0, sure I expect there is no problem (loosing .db-file in the mentionned scenario will probably not cause any missing data). But when the .db-jurnal file is quite huge, does it mean that opening .db file without .db-journal (it will have the old name) will cause any data loss?
Is there any way to rename two files in one "transaction" (either both are renamed either none of them)?
And what if I want to share my database (ex. I want to send it to my friend by e-mail)? Is that a must to send two files .db and .db-journal or only sending .db file is enough?
The rollback journal file contains information that is need to roll back a transaction.
If the app crashed while a transaction was active, the database will do the rollback automatically the next time the database is opened.
If you have opened a database (and maybe closed it again), and there is no active transaction, and your app has not crashed and is still running, then you know that the database file is in a consistent state and that you can ignore the journal file.
(If the journal mode is set to DELETE, there will be no -journal file then.)
(Note: in WAL mode, you cannot ignore the -wal file.)