I am seeing some strange behavior with my application and the state of its database file after running some tests that close the database, delete it, and replace it with a test fixture. When I examine the database file with a tool on my debugging PC, it doesn't match what the application itself seems to be reporting. It's possible that this strange behavior is related to this bug.
I noticed that there are two files with the same base name as the database (with the normal .db extension.) The file extensions are .db-shm and .db-wal, and each is newer than the .db file's timestamp.
I assume that these are some type of temporary files. However, I am wondering if the application is terminated, shouldn't they be deleted? More importantly, I assume whatever data is stored in them is updated inside the .db file before the application is terminated by the operating system. Is this correct?
You are correct, these are temporary files created by SQLite. If you are manually deleting the main db you should probably delete these too. From what I can gather the WAL is a replacement for the rollback journal that enables SQLite to rollback changes when a transaction fails. How SQLite uses them and why they are kept around for so long is up to the authors of SQLite but in general SQLite seems pretty rock solid so I wouldn't worry too much about them. For more info take a look here:
http://www.sqlite.org/fileformat2.html#walindexformat
These files are a new feature of SQLite 3.7. I'm not sure if their existence relates to the bug you point out but the bug report suggests a work-around anyway.
UPDATE:
Better documentation about the WAL is here:
https://www.sqlite.org/wal.html
The contents of the WAL are periodically moved to the DB file but this is not guaranteed to occur each time the process exits. Thus when WAL is enabled each SQLite DB consists of two files on disk that must be preserved, both the .db file and the .db-wal file.
The .db-shm file is a shared memory file that contains only temporary data.
I do not yet have enough reputation to just add a comment to satur9nine's answer, so I'll pile on here.
As per the SQLite docs, the DB-SHM file is a Shared Memory file, only present when SQLite it running in WAL (Write-Ahead Log) mode. This is because in WAL mode, db connections sharing the same db file must all update the same memory location used as index for the WAL file, to prevent conflicts.
As for WAL file, as hinted above, it is a write log/journal, useful for commits/rollback purposes.
Make sure that you have closed cursor properly into SELECT operation. Sometimes SQLiteOpenHelper creates .db-shm and .db-wal extensions database due to unclosed Cursor.
Related
(I'm new to android development)
Say android app-1 created a database hello, via roo.
Then could android app-2 created a database hello, on the same phone?
My guess is yes, since room use sqlite, each app may start a new sqlite instance, and may store its data in private space, otherwise it's too easy to get conflict. But I'm not sure.
The questions are:
Will 2 app share the same sqlite instance, or each app will start a new sqlite?
Is the database's name app scoped?
I guess the data of a database is also private to the application?
(I've searched about this, didn't get a confirm.)
By default, Room databases are stored in internal storage, and each app has its own internal storage. You can have two databases named hello, one per app, without a problem.
Really SQLite is a set is a set of file handling routines (more complex than just simple writes and reads though).
Android devices come with the routines built in (albeit different versions of SQLite which are typically backwards compatible).
Room is a wrapper around SQLite and thus the routines and caters for an object orientated approach to accessing the data (file(s)).
Unlike, for example, MySQL, SQLite it is not a server that manages transactions from extrenal sources, so SQLite is not a running instance background app/service. (The command line interface would be though)
A device can have many Apps that use the routines to access and update the underlying file (3 files if using the default WAL (Write-Ahead Logging, the core database file, the wal file and a third file the shm file a WAL file for the WAL), (2 files if using JOURNAL mode).
with WAL changes are applied to the WAL file which are then committed at times to the core/master file. Likewise for the shm file in regard to the WAL file. This allows a rollback to simply effectively delete/empty the WAL file.
JOURNAL mode the journal is a record/log of the changes made to the core database. A rollback is accomplished by reversing the changes according to the JOURNAL.
WAL is typically more efficient.
Typically this is all handled by the SQLite routines.
An App can itself access multiple databases if need be. SQLite even caters for accessing multiple database by ATTACHing databases to another.
By default the database is stored in the App's protected data storage area and thus is unique to the App. What you couldn't do, by default, is have the same database name twice within an App
you could say have two hello databases in different locations e.g. data/data/the_package/databases/hello and data/data/myotherdatabasesfolder/hello
note sure how you would go about this using Room though, certainly feasible using just the SQLite API.
So App1 and App2 could have the same hello database as the files will be in separate App specific locations (data/data/the_apps_package_name/databases by default).
Im creating an app but when i start to create the database clases (sqlite) and it works , now the sqlite files are different in android 9, in anothers versions you go to /data/data/packageapp/databases and you found database.sqlite and database.sqlite-journal, now i found this and this files cant open on sqlite administrator or similars... But the databases works i dont know what happen, i want to create the last files
Check the photo
https://imgur.com/oSiiGqZ
With Android 9 SQLite by default it now uses Write-Ahead Logging mode (WAL) instead of the journal mode due to it's potential for increased performance. Which is why you see the the -shm (Shared Memory file) and the -wal (Write Ahead Logging file) files.
Temporary Files Used By SQLite
However, you should be able to open the actual database file (yonuncafrases.sqlite) elsewhere in tools that support opening SQLite Databases. Compatibility WAL (Write-Ahead Logging) for Apps
i want to create the last files
If need be you can force using journal mode by using the SQLite disableWriteAheadLogging method, noting that this must be done outside a transaction, and then Journal Mode would be used and that the -journal file would then be created as necessary.
I'd suggest overwriting the onConfigure method and calling disableWriteAheadLogging in that overridden method. Alternatively you could use the journal_mode pragma (which I believe is what disableWriteAheadLogging does anyway)
You may wish to read Write-Ahead Logging
I am running a small but critical application on an Android phone. It uses an sqlite database as its main data storage. After over a year of near flawless functioning, it suddenly crashed at a critical point a few days ago as a result of the infamous "database disk image is malformed" problem. This caused some embarassment as I couldn't access the system to fix it for a few hours.
I'm aware that such situations should be very rare. But I'm now coding in some code to more gracefully recover from such a situation, since I can't afford to have it happen again (and it may be due to the phone's hardware, in which case it might now start to happen more frequently - but again I can't afford a crash even once more).
However, to test my recovery code, I need to generate the same error, and due to a stupid mistake I lost the copy of the database that was creating the problem.
My question is - is there something I can do to an sqlite database that would result in the "database disk image is malformed" error? I can then test my recovery code.
The SQLite docs have an article named How To Corrupt an SQLite Database File
SQLite database files are ordinary disk files. That means that any
process can open the file and overwrite it with garbage. There is
nothing that the SQLite library can do to defend against this.
If you write garbage data to the file, I guess you can achieve what you want
You can do it.
Simply copy your working database file in assets.
Then once application runs, copy the database file through code bytes by bytes and discard some bytes while copying, you will get a corrupted database.
You can try something else as well :
Just create a text file. Then rename it to *.db file and try to open the file in android assuming its a database file. I hope you will get the same exception.
Is there any Android Official way of doing sql db backup to SD or Phone? there are several backup helper classes mentioned in the document but all of 'em are pretty useless.
I know that the sql file is a simple db file i can use Java IO class to read and write somewhere. but that is not a wise idea if we have latest version of App which contains several newly added columns in it. we cannot restore back.
Are there anyway to overcome this issue? Any easy way that so called android engineers provided in the document?
Have you seen the Backup Manager?http://developer.android.com/guide/topics/data/backup.html
Here is an article on how to use it with Gson.
https://advancedweb.hu/2015/01/06/efficient-sqlite-backup-on-android/
Not sure if this is the best solution but this will allow you to pick and choose the columns you want to restore.
I use the following approach:
make a copy of db file while back up.
Read that backup db file while restore using normal sqlite APIs.
Perform any operations like inserting default values for new columns if required and insert the data into the current db.You can perform the operations based on the version of db..The version of db can be encoded in the file name-something like backup_version_2.2.db
delete the backup db file
What is the difference between using external db file(from assets) and creating a new db using SQLiteOpenHelper? Performance wise which one will be fast? will accessing db from assets slow down my application.
What is the difference between using external db file(from assets) and creating a new db using SQLiteOpenHelper?
You cannot directly use a database that is pre-packaged in your app as an asset. SQLite needs a file; an asset is not a file, but rather is an entry in the ZIP file that makes up an APK.
Using something like SQLiteAssetHelper, you can a pre-packaged database as the starting point for your user, by copying the database from assets into internal storage for your app.
will accessing db from assets slow down my application.
That depends on what you compare it to, and even then the difference will only be the first time you try to work with the database.
With the database-packaged-as-a-asset approach, the database needs to be copied from assets before it can be used. This takes time. It should not be dramatically slower than creating an empty database and executing SQL statements to populate it. In fact, I would expect it to be faster in many cases. However, it will be slower than starting from an empty database.
Once the database is created and set up -- whether via your own CREATE TABLE statements or by copying a starter database from assets -- performance will be identical, because the databases themselves are identical.
The assets folder is read-only, so if you need to edit your database then it won't work for yourapp. If you have a pre-populated database, one option is to make a copy from the assets folder into somewhere that is writeable when your app is first accessed, and then continue with using the writeable version.
There isn't a performance difference assuming your database is properly configured. See this:
http://www.reigndesign.com/blog/using-your-own-sqlite-database-in-android-applications/
I'm using this procedure for a ~500mb database upon which I have to do fairly complex and substantial queries routinely, and which I zip for distribution and unzip on the first run due to size and this works flawlessly.