I have connected my device and accessed my applications data by going to /data/data/my.package.name/databases.
Here I can see files:
data
data-shm
data-wal
As I can understand these files are specific to android system itself, but they represent SQLite database and yet they are not mountable to SQLite reader?
I have this issue when data is being downloaded and stored to database, after some time data-wal starts to become extremely huge (from maybe 12MB to 7GB) and after sync finished it becomes almost empty again. Am I correct in saying that this is probably the issue with transactions (somewhere transaction is not being closed and is always opened and because of this reason data-wal is being filled with back-up data in case of a rollback)?
The database consists of these three files. When opening data, the other two are automatically used when needed.
The -wal file contains all changes made to the database since the last checkpoint. To prevent the -wal file from becoming too large, use smaller transactions so that SQLite is able to do checkpoints, or just disable WAL mode altogether during bulk writes.
Related
In my application that i wrote using sqlite for data persistence.
I decide to create local db file and to this file i read/write my data.
Because there is an option that the user will do something ( like turn off his device ) that will corrupt this db file - i want to create backup db file that will be override at the end of the writing process to the db file.
( i will always hold 2 db files .. one is the not last update that differently not corrupted and one is 'dirty' db file that hold the latest data - and my app will write to this file and make the swap between those two files after writing )
Is there some other way to avoid a case of corrupted db file ?
Is there some code example or some lib of how to do this backup on android code ?
One option to avoid the risk of a local SQLite database getting corrupted would be to mirror your critical data to the server, ultimately to a more permanent database which is more stable. So as an example here you could periodically sync your local user data with the server. If you detect corrupt data, of if things are so bad that you can't read at all, you would not update with that data, and you might even use the server side database data for doing a restore of some kind.
I recommend you to check out Room - a new library which is an abstraction layer over SQLite that comes to deal with issues programmers face when managing SQLite database manually. Personally, I've migrated to Room and it solved for me some issues (other than yours, but still) that I was struggling with before.
I am making a mobile application which scans the device for audio files and I am putting these files in a sqlite db.
Now, my strategy is I commit first 100 entries to the db and make them available for browsing and play and then when I have scanned the rest of the files , I commit the rest thousands of files at the end, thus completing the db.
But I found that after my first commit of 100 files, sqlite file is not generated till I make a second commit of remaining thousands of files and close the db.
Why the db file is not generated?
Check if you are invoking proper "flush" of your output streams wherever they are needed. Also if you play with transactions all the stuff happens in memory until you do committing.
Close of course does the flushing for you.
My app tracks school grades, calculates averages, etc. and stores all of this in a SQLite database. If a user has to reinstall or gets a new phone, I'd like to be able to restore their data.
It looks like most developers do this either by backing up to SD card or by using Android Backup Service through Google. I'm not sure which is the better method. I'd like restoring to be simple but reliable. I welcome any comments on this.
One thing I'm trying to understand is why Google says to extend BackupAgent instead of BackupAgentHelper if using a database.
If you have an SQLite database that you want to restore when the user re-installs your application, you need to build a custom BackupAgent that reads the appropriate data during a backup operation, then create your table and insert the data during a restore operation.
Why can't I just back up the database as a file and then restore the file? My SQLiteOpenHelper class already handles upgrades if db versions are different. I guess I could just abort on a downgrade.
Why can't I just back up the database as a file and then restore the
file? My SQLiteOpenHelper class already handles upgrades if db
versions are different. I guess I could just abort on a downgrade.
Reason: same database file may not work on different device models(even though most of the cases, it should work, there are cases where it will fail). It depends on parameters like page size etc set at sqlite engine level. Ideal way is to backup the data rather than copying the whole file
It's suggested that you avoid backing up the whole db file all the time mostly because that's a lot of redundant data traffic, especially if you've only changed one record in a large db. Being able to write per-record updates to the backup system is much more efficient (though of course is not nearly as simple to implement).
Main Question
How the cursor retrieves data from SQlite? does it refers to database file addresses dynamically? or loads it fully to the memory? though i know the dalvik virtual machine is address based and the the first assumption is more likely to be true, as the nature of RAM memory and phone storage are almost the same.
So my main question is to know how the data are load? from loading to memory? or just addressing to database file content?
To clarify: (the sample is just for clarifying. you can skip it)
The question is raised from the point that:
I have created an app which loads data from sqlite and displays them in listview. the databsase grows up using user data by time. Now, when the database goes larger, is it required to load data to listview in a manner like using load more or pagination? or its true to load them in one place?
although, pagination would be better for responsiveness but, when trying to export data to xls or pdf format, is it possible to retrieve a cursor to all the database and save data in xls or pdf?
the messaging app of android loads all messages in one place and has causes no problem even when i have 3000 messages in one thread.
Seems that data for Cursor is stored in memory or some kind of cache file (it's implementation details as I've properly mentioned).
There are two possible ways to proof / show why it's not original DB file. I'm sure there should be also some kind of more theoretical explanation.
Take a look at SQLiteCursor source (it's available in your sdk platform installation): it's based on CursorWindow which is
A buffer containing multiple cursor rows.
A CursorWindow is read-write when initially created and used locally.
When sent to a remote process (by writing it to a Parcel), the remote
process receives a read-only view of the cursor window. Typically the
cursor window will be allocated by the producer, filled with data, and
then sent to the consumer for reading.
Also, from the source it looks like that Window contains all data in that buffer.
Create test application with test DB with lot of records. Request all records and show in the list. While list is showing keep constantly changing db content and observe that list content is not changed (I assume eliminate usage of requery() and related deprecated stuff).
I have an app which stores data in a SQLite database, when the app is just installed it uses 8kb of data and when I start inserting data obviously the size increases. The weird thing is when I start cleaning my database tables, I empty completely the database but I will never recover the first 8kb of data but much more, in some cases more than 100kb, where this data come from?
A database is not like a filesystem, so when you delete data from a database, you do not recover the space you previously used. The database driver will tend to prefer to keep the old allocated space.
You can compact your database, which will recover this space by manually executing the SQLite VACUUM command.
It may be possible to enable auto_vacuum mode, but this needs to be done before database creation.