I have an app which stores data in a SQLite database, when the app is just installed it uses 8kb of data and when I start inserting data obviously the size increases. The weird thing is when I start cleaning my database tables, I empty completely the database but I will never recover the first 8kb of data but much more, in some cases more than 100kb, where this data come from?
A database is not like a filesystem, so when you delete data from a database, you do not recover the space you previously used. The database driver will tend to prefer to keep the old allocated space.
You can compact your database, which will recover this space by manually executing the SQLite VACUUM command.
It may be possible to enable auto_vacuum mode, but this needs to be done before database creation.
Related
Android Room has method void clearAllTables() and according to docs it makes the following:
Deletes all rows from all the tables that are registered to this database as entities().
This does NOT reset the auto-increment value generated by autoGenerate().
After deleting the rows, Room will set a WAL checkpoint and run VACUUM. This means that the data is completely erased. The space will be reclaimed by the system if the amount surpasses the threshold of database file size.
I checked it in my project and looks like db has no data after that call, but when I pulled
*.db file from my device and opened it in sqlite viewer I've seen that all the data exists, the tables are filled and nothing has been erased.
How can that be? I consider this as a potential flaw in my app. Please provide a reliable approach of cleaning room database
looks like db has no data after that call
It means the method worked.
when I pulled *.db file from my device and opened it in sqlite viewer
I've seen that all the data exists
Most probably the transactions are not moved to the original database from the WAL file yet.
Solution
You can force a checkpoint using the wal_checkpoint pragma. Query the following statement against the database.
pragma wal_checkpoint(full)
I have connected my device and accessed my applications data by going to /data/data/my.package.name/databases.
Here I can see files:
data
data-shm
data-wal
As I can understand these files are specific to android system itself, but they represent SQLite database and yet they are not mountable to SQLite reader?
I have this issue when data is being downloaded and stored to database, after some time data-wal starts to become extremely huge (from maybe 12MB to 7GB) and after sync finished it becomes almost empty again. Am I correct in saying that this is probably the issue with transactions (somewhere transaction is not being closed and is always opened and because of this reason data-wal is being filled with back-up data in case of a rollback)?
The database consists of these three files. When opening data, the other two are automatically used when needed.
The -wal file contains all changes made to the database since the last checkpoint. To prevent the -wal file from becoming too large, use smaller transactions so that SQLite is able to do checkpoints, or just disable WAL mode altogether during bulk writes.
I am creating a data repository layer in my application which in turn serves data from two sources, either network API calls or getting data from local storage. When there is a request to the data repository layer, I first need to check if the data is present in the local storage or not. If not, I then make a call to my API to get the data. To check if the data is in local storage or not, I was thinking of two ways to implement this:
On app startup, I load all the data in the database into the memory (eg. lists, maps etc.) and then whenever I have to check of existence of data in the local storage, I check from the memory. This has a possible problem that I have faced earlier as well. When the app is in background, Android system, might clear up this allocated memory to serve memory to other apps. This makes things complicated for me.
Whenever I want to check the existence of data in the local storage, I directly make queries to SQL tables and decide based upon that. This is much more leaner and cleaner solution than the above mentioned case. The only worry for me here is the performance hit. I wanted to know if the SQLite database runs after being loaded into the memory or not? If yes, then the memory layer of data that I had created above is useless. If not, is it safe to keep on querying the SQLite database for each and every small data request?
SQLite caches some data, and the Android OS will keep the database file in its file cache, as long as there is enough memory.
So when using SQLite, your app's performance is automatically optimized, depending on available memory.
My app tracks school grades, calculates averages, etc. and stores all of this in a SQLite database. If a user has to reinstall or gets a new phone, I'd like to be able to restore their data.
It looks like most developers do this either by backing up to SD card or by using Android Backup Service through Google. I'm not sure which is the better method. I'd like restoring to be simple but reliable. I welcome any comments on this.
One thing I'm trying to understand is why Google says to extend BackupAgent instead of BackupAgentHelper if using a database.
If you have an SQLite database that you want to restore when the user re-installs your application, you need to build a custom BackupAgent that reads the appropriate data during a backup operation, then create your table and insert the data during a restore operation.
Why can't I just back up the database as a file and then restore the file? My SQLiteOpenHelper class already handles upgrades if db versions are different. I guess I could just abort on a downgrade.
Why can't I just back up the database as a file and then restore the
file? My SQLiteOpenHelper class already handles upgrades if db
versions are different. I guess I could just abort on a downgrade.
Reason: same database file may not work on different device models(even though most of the cases, it should work, there are cases where it will fail). It depends on parameters like page size etc set at sqlite engine level. Ideal way is to backup the data rather than copying the whole file
It's suggested that you avoid backing up the whole db file all the time mostly because that's a lot of redundant data traffic, especially if you've only changed one record in a large db. Being able to write per-record updates to the backup system is much more efficient (though of course is not nearly as simple to implement).
My app uses Fragments, SQLite Database, SharedPreferences and SavedInstanceState to keep the state.
The problem is that the Data size does not stop increasing (about 100-300kB) on each Fragment launch. I can not count on Clear Data as long as it removes all data including SQLite Database of the application.
1- What are data of android application saved in Settings> Apps >App info> STORAGE> Data? OK (tnx to artworkad シ reply)
2- How this problem can be solved? (while SQLite database must be kept)
*****UPDATE**: After disabling every probable source of storage (database & Shared Preferences), I figured out that problem is due to AdView. It means once adview content is shown it adds some data to app storage, when I switch between Fragments/FragmentActivities.
Your help is really appreciated!
Data are application databases and shared preferences. Share preferences are a simple key/value store, you should not abuse it to persist big data sets.
If your sqlite databases grow, the data will take up more space on the phone.
How to solve the problem of growing data size?
You could keep some of this data on a sever and load it on demand. Or you can try compression, e.g. VACUUM.