I want to monitor the size of my Room database in my android app. I have searched around for a while and found little info about exactly how to do that. Is there any Room (SQLite) command to determine the db size ?
After a while, searching around, I found some "old" (aka ancient) pragma calls which I tested, and it worked somehow. Not in the need to get the exact values, but a hint of the usage.
You could do a #Dao #Query in Room like this:
#Dao
interface SystemDao{
#Query("""
SELECT s.*, c.*
FROM pragma_page_size as s
JOIN pragma_page_count as c
""")
suspend fun getDatabaseSizeInfo():List<SystemDatabaseSizeInfo>
}
And in the data class "pojo" you define:
data class SystemDatabaseSizeInfo(
var page_size:Int?=null,
var page_count:Int?=null
)
Since the result from the pragma queries are page_size and page_count. Theese can later be multiplied, or presented separately.
You can read about SQLite pragma calls here:
https://www.sqlite.org/pragma.html
There are bunch of other pragma queries which may useful, but you should use them visely, since some of them may be devastating for the stability of the SQLite DB.
Android IDE/lint may render theese commands not valid (red), but if you have AS4.1Cx (C6+) you may be able to run some of them in the Database Inspector tool to see if they work. I have filed an issuetracker incident on that here: https://issuetracker.google.com/157374656
Thank you for all who answered, you led me to the correct path with intensive googling...
Haven't tried it, but it should be something alike this:
fun getSize(context: Context, fileName: String) : Long {
val file = context.getDatabasePath(fileName)
return file.length()
}
As #Commonsware suggested, unless the write-ahead log had been committed, it may be inaccurate (it's these *.wal files in the same directory). And one can't just add up the sizes, but would have to check if the database file is currently in use and if so, close it first. Another option would be .disableWriteAheadLogging(), which comes at the cost of some performance.
In Android Studio in debug mode in either a device or emulator, you can use the Device File Explorer which is at bottom right of Android Studio, goto data -> data -> your package name (eg. com.myapp) -> databases:
Related
I am using Room on Android and programming in Java and am finding myself jumping through hoops to maintain a count field in a table due to some of the callbacks and background thread requirements and I was hoping somebody might have a simple solution.
I am doing an insert into a table for logging purposes. For each ScenarioRun I need to maintain a count. So it is not the count of records in the whole table, just for a given ScenarioRun.
I believe the best place to do this is in the ViewModel when I am inserting the RunLog entry. Ideally I would calculate the count from the Log table using the RunId, so my Dao would have this query in it:
#Query("SELECT COUNT(*) from ScenaroRunLog WHERE scenarioRunId=:scenarioRunId")
int getScenarioRunLogCount(int scenarioRunId);
Which I would pass through my repository to my ViewModel so on an insert, I would increment it:
public void insert(ScenarioRunLog scenarioRunLog) {
// code to set the count in the object something like:
// int count = scenarioRunLogRepository.getScenarioRunLogCount(scenarioRunLog.getScenarioRunId);
// scenarioRunLog.setCount(count);
scenarioRunLogRepository.insert(scenarioRunLogId, scenarioRunLog);
}
I could do it in the repository but the only way I can see how to do this is by using allowMainThreadQueries(), which doesn't seem like a best practice or using Executors. Basically the suggestions here.
Is there a better way to do things in this case? I have also found simple things to be challenging.
I could also simply load the ScenarioRun in the ViewModel and reset the count at that time, maintaining it in the ViewModel from that point on, but I have the additional challenge of loading the ScenarioRun. I guess I could do this with another AsyncTask for the select, like they do here.
LiveData for RecyclerViews and other UI components are great, but simple queries have seemed to be challenging. Even returning RowId's on inserts has required a lot of work. Looking for some guidance here. I appreciate the help in advance.
In my android app I am using Room with sqlcipher library for encrypt/decrypt. Often I see in Crashlytic the following crash:
java.util.concurrent.TimeoutException:
net.sqlcipher.database.SQLiteCompiledSql.finalize() timed out after 10
seconds at sun.misc.Unsafe.park(Native Method) at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:190) at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:868)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:902)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1227)
at
java.util.concurrent.locks.ReentrantLock$FairSync.lock(ReentrantLock.java:231)
at
java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:294)
at
net.sqlcipher.database.SQLiteDatabase.lock(SQLiteDatabase.java:567)
at
net.sqlcipher.database.SQLiteCompiledSql.releaseSqlStatement(SQLiteCompiledSql.java:104)
at
net.sqlcipher.database.SQLiteCompiledSql.finalize(SQLiteCompiledSql.java:146)
at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:289) at
java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:276) at
java.lang.Daemons$Daemon.run(Daemons.java:137) at
java.lang.Thread.run(Thread.java:929)
The line where it crash is SQLiteDatabase.lock() line 567
Previously it was line 566, but in that method I inserted a check: if the database is not open -> return and don't proceed with locking, but it didn't help and the crash appeared again.
I think that this crash may be because the garbage collecting happens when the app is in the background (our app has a foreground service that works all the time). But not sure how to fix it.
As for Room: I don't close it, it is open all the time, because my app works all the time, so need it often. And close it after every query to the database is the bad practice.
I asked the developers of sqlcipher, but they don't know, what can cause this crash. Maybe anybody knows?
I've been seeing this in Crashlytics for quite a while, and seemingly has been able to fix this in the end completely, I'll post the full research here.
Issue
For queries like this one:
#Query("DELETE FROM table WHERE id NOT IN (:ids)")
abstract fun deleteNotInList(ids: List<String>): Int
Room generates code that doesn’t clean cursor and generated statements (on the image below, compare how the method one doesn’t use cursor, and doesn’t call release() methods, compared to the bottom one which calls cursor.close(); method names and queries are slightly different there, cause I simplified the snippets):
In this scenario statements are kept in memory un-released, and the releasing shifts to GC phase, inside finalize() method. And finalize() in SqlCipher (SQLiteCompiledSql) in turn needs to lock the Database in order to release the statement:
Problem is that the Database might be locked for a long transaction for more than 10 sec (or a batch of shorter transactions, awakening order using such locks isn’t guaranteed and isn’t fair).
GC watchdog crashes the thread when it reaches 10/20 seconds depending on the OS/JVM exact version.
Solution
Is to rewrite all DELETE / UPDATE using IN operator with manual raw queries like this:
#RawQuery
abstract fun deleteNotInListRaw(query: SimpleSQLiteQuery): Int
fun deleteNotInList(
ids: List<String>
) {
deleteNotInListRaw(
SimpleSQLiteQuery(
"DELETE FROM table WHERE id NOT IN (${ids.joinToString(prefix = "'", postfix = "'", separator = "','")})"
)
)
}
In this case the query uses cursor and closes it after deletion is done, still locking database but not on GC phase and on the dedicated thread.
P.S.
There’re more stable long term solutions possible but they needs to be implemented on the Room / SqlCipher sides.
In the given state, SqlCipher can be refactored to not lock the database on GC phase. There’s an open issue for that: https://github.com/sqlcipher/android-database-sqlcipher/issues/537
Room should probably fix the codegen and use the query builder and generate statement closure lines, there’s no open issues there, but I’ll double check this thought later and will raise this as an issue with them.
For us this has seemingly solved the issue completely.
Note that all DELETE / UPDATE queries using IN / NOT IN operators and probably some other operators that prevent Room from precompiling the query (due to runtime params) cause this. You can check the codegen to verify that generated code calls either cursor.close() or statement.release()
So, I wanted to implement a simple database handler class for my Android Studio project with Kotlin. Little did I know when I eye-rolled towards my "what even is kotlin" developer mates, that this hipster choice would lead me to lengthy damnation.
I decided to start out by copy-pasting the code Google suggests for a DB Helper utilizing the SQLiteOpenHelper class.
Naturally, I started getting this unreasonable error
for each of the closing brackets } in 3 different methods containing what looked like perfectly valid syntax.
Of course, I spent over a day checking every single piece of annoying Gradle build property that seems to be there only to increase the chances of something being off, as if it were a perfectly designed developer-Limbo build system from Hell. But every little check was to no avail, as I had no build errors and the "Expecting member" errors kept showing up.
Decided not to quit however, I even attempted to change my implementation to use the more Kotlin-friendly anko.db route, full code below.
package com.example.john.myapplication
import android.content.Context
import android.database.sqlite.SQLiteDatabase
import org.jetbrains.anko.db.*
class DBHandler(context: Context) : ManagedSQLiteOpenHelper(context, "POIEncounters", null, 1) {
override fun onCreate(db: SQLiteDatabase?) {
db?.createTable("Encounter", true, "id" to INTEGER + PRIMARY_KEY + UNIQUE, "Name" to TEXT)
}
override fun onUpgrade(db: SQLiteDatabase?, oldVersion: Int, newVersion: Int) {
db?.dropTable("Encounter", true)
}
override fun onDowngrade(db: SQLiteDatabase?, oldVersion: Int, newVersion: Int) {
onUpgrade(db, oldVersion, newVersion)
}
}
As Divine Order dictates in the Developer Limbo, there was no change at all. Every time I click Build I am fooled by the syntax checker's underhanded reassurances, as the compiler denies my salvation. Even more amazingly. the errors persist even if I comment-out the entire bodies of the overridden methods.
At this point I am so beyond despair that I have fallen to the lowest level of the abyss: rejecting my ego and asking for help.
For those not brave enough to witness the process, TLDR at the end.
The answer is given by my own self, after 3 endless epochs (see: days) in this unbuildable hellish dimension, and written at 3am with the sole company of a bottle of Glenfiddich whisky joined in the dark by the sounds of my demonized keyboard clacking.
The usual suspects of easy-to-miss spelling errors, invalid constructor usage have been discarded over and over as I descend into madness, now fairly certain that even if someone where to copy my exact code, they might not even be getting this error.
I even tried tried downgrading Kotlin itself, in delusional hopes of a bug in its compiler bringing forth all these insistent mutterings of "Expecting declaration"...
In a last insane attempt I even tried importing my code into a different infernal machination of an IDE, just to see if the apocryphal messages would follow me there, in Intellij IDEA...
After one more hour of torment to align the new project's build files and other horrid properties causing minor issues... They did.
I was about to give up and when I spotted this strange, unfamiliar sign on Intellij's syntax checker:
Found byte-order-mark in the middle of a file <...>
SON. OF. A. BENCH.
Now, at this incredibly agonizing part of my journey, much like many other devs living in the blissful heaven-like area of "typing code that is parsed the way you read it", I had no forking idea what a BOM is.
Yet the celestial IDEA even gives us the forbidden, dark knowledge of a wikipedia link, which I followed in foolish innocence, to see a maddening, curious fragment of the cosmos which man was not meant to lay eyes upon...
What the fork.
My fingers trembled and started moving in a feverish climax. I heard myself cackling as I submerged myself in the unreal madness of a last irrational hope...
I deleted the 3 methods. Purged them all an in unholy, functio-cidal frenzy and gazed up the empty class, free of the invisible, sickening infection.
Build. *click* Success.
I rewrote each and every one one of the methods by hand, without copy-pasting or touching any devil-infected piece of vile text, while listening to the fiendish, otherworldly "Dethklok Awaken" calls. And that was it. It worked. I am free, ascending into Elysium, whiskey in one hand and tears dropping to the Flame Below.
Yet, empty and disappointed. For the the cunning shade was not a tangible foe, but an ephemeral force, looming inside the Self...
With this answer, I hope no one has to face the same mistake I did. Not unprepared at least.
TLDR: There was a BOM character that was causing the error. Intellij clarified the cause, while Android Studio didn't. Deleted and re-wrote problematic parts by hand without copy-pasting anything and it worked.
Epilogue: Everyday we stray further from ASCII's light.
I'm looking into using greenDAO for my Android app, but I noticed it doesn't seem to support any kind of data validation other than "not null", "unique", and foreign keys, either on the SQL level (constraints defined when creating tables) or the Java level (validation logic in setter methods). "Keep sections" don't seem like they would be helpful in this case because you can't have them within individual methods. Am I missing something, or would I really need to add yet another layer on top of the generated Java objects if I wanted to validate input data? (I'm somewhat confused how the framework could be useful without providing any place to include validation logic.)
1.
You can write a method
boolean check ();
in KEEP-SECTION of the entity which you call manually before INSERT or UPDATE.
2.
Another possibility is to extend the sourcecode of greendao generator to support checks: In property.java you could add a method to Property.Builder
public Property.Builder check (String expr) {
property.checkConditon = expr;
}
Of course you would have to introduce the String checkCondition = ""; and use it for generating the dao in the dao-template.
Problem:
With new versions of greendao your changes would be lost (but then again new version may already contain such a feature)
3.
A third possibility is to copy the generated CREATE TABLE statement, modify it to fit your needs and call your modified statement instead of the original one or to drop the original table and call your statement.
Problem:
If your table changes you will have to repeat this.
Imagine the following scenario (I allow backup / restore from my app, I'm doing backup / restore white the entire file.db):
The user make backup of the database.
In the future I do an update on my application which has a new version of the database.
what would happen if the user restore the database with the old version?
How can I avoid this kind of problem?
It would be a good idea to use BackupHelper? I did a simulation of the worst scenario of my database and gave 20k, BackupHelper is recommended for less than 1mb, it would be a good idea to use it? I think my app will never exceed 100kb.
You access SQLite databases via a SQLiteOpenHelper, which provides three callbacks: onCreate(), onUpgrade() and onOpen().
The constructor declares a version parameter, and internally it compares the database version (which is stored in the file) with the supplied argument. If the requested version is newer than the file's, your onUpgrade(db, old, new) is called and you get a chance to alter tables, fill rows and so on.
The restore procedure should simply close all open cursors and helpers and copy the new file in.
May be this is not the best approach but you can do it as:
1- Store the DB Version in the database.
2- After restoring the database, check the DB Version and do the required changes accordingly. Like
void afterRestoration()
{
long dbVersion = get from db;
if(dbVersion == 1)
{
alter table1 add column1;
}
else
{
}
}