I made a screen like the current image.
Data such as A, B, C.. are currently being set by getting from the strings.xml resource file.
I am now going to use Room DB instead of strings.xml and I want to get these data from Room.
To do this, we need to pre-populate the Room with data.
In the sample code I found, the method called addCallback() was usually used.
like this :
#Database(entities = arrayOf(Data::class), version = 1)
abstract class DataDatabase : RoomDatabase() {
abstract fun dataDao(): DataDao
companion object {
#Volatile private var INSTANCE: DataDatabase? = null
fun getInstance(context: Context): DataDatabase =
INSTANCE ?: synchronized(this) {
INSTANCE ?: buildDatabase(context).also { INSTANCE = it }
}
private fun buildDatabase(context: Context) =
Room.databaseBuilder(context.applicationContext,
DataDatabase::class.java, "Sample.db")
// prepopulate the database after onCreate was called
.addCallback(object : Callback() {
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
// insert the data on the IO Thread
ioThread {
getInstance(context).dataDao().insertData(PREPOPULATE_DATA)
}
}
})
.build()
val PREPOPULATE_DATA = listOf(Data("1", "val"), Data("2", "val 2"))
}
}
However, as you can see from the code, in the end, data (here, val PREPOPULATE_DATA) is being created again within the code. (In another code, db.execSQL() is used)
In this way, there is no difference from fetching data from resource file in the end.
Is there any good way?
Developer documentation uses assets and files.
However, it is said that it is not supported within In-memory Room databases.
In this case, I do not know what In-memory means, so I am not using it.
In this case, I do not know what In-memory means, so I am not using it.
In-Memory will be a database that is not persistent, that is the database is created using in memory rather than as a file, at some time it will be deleted. You probably do not want an in-memory database.
However, as you can see from the code, in the end, data (here, val PREPOPULATE_DATA) is being created again within the code. (In another code, db.execSQL() is used)
This is a common misconception when writing Apps as the onCreate method of an activity is often repeated when an App is running. With an SQLite database the database is created once in it's lifetime, which would be from the very first time the App is run until the database file is deleted. The database will otherwise remain (even between App version changes).
Is there any good way?
You basically have two options for a pre-populated database. They are
to add the data when/after the database is created, as in your example code (which is not a good example as explained below), or
to utilise a pre-packaged database, that is a database that is created outside of the App (typically using an SQlite tool such as DBeaver, Navicat for SQlite, SQLiteStudio, DB Browser for SQLite).
Option 1 -Adding data
If the data should only be added once then using the overridden onCreate method via the CallBack can be used. However, using functions/methods from the #Dao annotated class(es) should not be used. Instead only SupportSQLiteDatabase functions/methods should be used e.g. execSQL (hence why the SupportSQLiteDatabase is passed to onCreate).
This is because at that stage the database has just been created and all the underlying processing has not been completed.
You could protect against duplicating data quite easily by using INSERT OR IGNORE .... rather than INSERT ..... This will skip insertion if there is an applicable constraint violation (rule being broken). As such it relies upon such rules being in force.
The two most commonly used constraints are NOT NULL and UNIQUE, the latter implicitly for a primary key.
In your case if a Data object has just the 2 fields (columns in Database terminology) then, as Room requires a primary key, an implicit UNIQUE constraint applies (could be either column or a composite primary key across both). As such adding Data(1,"val") a second time would result in a constraint violation which would result in either
The row being deleted and another inserted (if INSERT OR REPLACE)
This further complicated by the value of autogenerate.
An exception due to the violation.
The insert being skipped if INSERT OR IGNORE were used.
This option could be suitable for a small amount of data but if over used can start to bloat the code and result in it's maintainability being compromised.
If INSERT or IGNORE were utilised (or alternative checks) then this could, at some additional overhead, even be undertaken in the Callback's onOpen method. This being called every time the database is opened.
Pre-packaged Database
If you have lots of initial data, then creating the database externally, including it as an asset (so it is part of the package that is deployed) and then using Room's .createFromAsset (or the rarer used .createFromFile) would be the way to go.
However, the downfall with this, is that Room expects such a database to comply with the schema that it determines and those expectations are very strict. As such just putting together a database without understanding the nuances of Room then it can be a nightmare.
e.g. SQLite's flexibility allows column types to be virtually anything (see How flexible/restricive are SQLite column types?). Room only allows column types of INTEGER, TEXT, REAL or BLOB. Anything else and the result is an exception with the Expected .... Found ... message.
However, the easy way around this is to let Room tell you what the schema it expects is. To do so you create the #Entity annotated classes (the tables), create the #Database annotated class, including the respective entities in the entities parameter and then compile. In Android Studio's Android View java(generated) will then be visible in the explorer. Within that there will be a class that is the same name as the #Database annotated class but suffixed with _Impl. Within this class there is a function/method createAllTables and it includes execSQL statements for all the tables (the room_master_table should be ignored as Room will always create that itself).
The database, once created and saved, should be copied into the assets folder and using .createFromAsset(????) will then result in the pre-packaged data being from the package to the appropriate local storage location.
Related
I now finished the android codelab about room db with MVVM arch. But there is one part that i didn't exactly understand. This is a sentence from the codelab:
To delete all content and repopulate the database
whenever the app is created, you'll create a RoomDatabase.Callback and override onCreate().
and this is the code they provide:
private class WordDatabaseCallback(
private val scope: CoroutineScope
) : RoomDatabase.Callback() {
override fun onCreate(db: SupportSQLiteDatabase) {
super.onCreate(db)
INSTANCE?.let { database ->
scope.launch {
populateDatabase(database.wordDao())
}
}
}
suspend fun populateDatabase(wordDao: WordDao) {
// Delete all content here.
wordDao.deleteAll()
}
}
the part that I don't understand is "deleting all content". why do I need to delete all content when the app is created? and what do they mean by "whenever the app is created"? is it for the first time the app installed or everytime app is opened?
when I don't use this code, the app works fine too. can someone explain the purpose of deleting everything?
why do I need to delete all content when the app is created?
You don't need to, this is a specific scenario where for whatever reason you want the App to delete all content stored in the database. However there should not be any anyway as onCreate is only called when the database doesn't actually exist (unless manually invoked), so there will be no content to delete.
If you used a pre-packaged database using the .createFromAsset then onCreate isn't called.
I believe that they have just included a simple, unlikely to fail, introduction to using a Callback.
and what do they mean by "whenever the app is created"? is it for the first time the app installed or everytime app is opened?
The former. That is the whole purpose of a Database is to store data long term. As such the database is stored in the App's data space. When an App is installed then the data space will not have the database. So the App and thus Room has to know to create the database before it can be used.
So when you attempt to use the database, the processing of preparing to use the database checks to see if the database exists.
If the database does exist then it carries on without calling onCreate.
If the database does not exist then Room will try to create the database and the tables therein via the onCreate method. The Callback allows intervention at this stage by overriding the onCreate method.
If the App is stopped and rerun the database will still exist, and onCreate is not called.
If the App is uninstalled and then re-installed the the database will be deleted, so onCreate is called.
If a new version of the App is installed, then the database will still exist and onCreate is not called.
If a new version of the App is installed and it includes a new version for the database, onCreate is not called. Instead whatever migration path is specified will be taken.
In the app I'm working on, we had a complex manual migration that required data parsing, manual SQL commands, etc. This was to convert a List<X> column into a new linked table of X. I've previously written about the approach, but the specific commands are not especially relevant for this question.
The issue I'm encountering is ~1% of users are experiencing a crash as part of this migration. This cannot be reproduced in testing, and due to our table's size, Crashlytics cannot show any useful error:
Losing customer data isn't catastrophic in this context, but being stuck in the current "try migrate, crash, reopen app and repeat" loop is. As such, I want to just give up on the migration and fall back to a destructive migration if we encounter an exception.
Any ideas how this can be done? My current solution is rerunning the DB changes (but not the presumably failing data migration) inside the catch, but this feels very hacky.
Our database is defined as:
Room.databaseBuilder(
context.applicationContext,
CreationDatabase::class.java,
"creation_database"
)
.addMigrations(MIGRATION_11_12, MIGRATION_14_15)
.fallbackToDestructiveMigration()
.build()
where MIGRATION_14_15 is:
private val MIGRATION_14_15 = object : Migration(14, 15) {
override fun migrate(database: SupportSQLiteDatabase) {
try {
// database.execSQL create table etc
} catch (e: Exception) {
e.printStackTrace()
// Here is where I want to give up, and start the DB from scratch
}
}
}
The problem you have is that you cannot (at least easily) invoke the fall-back as that is only invoked when there is no migration.
What you could do is to mimic what fall back does (well close to what it does). That is the fall-back will delete (I think) the database file and create the database from scratch and then invoke the databases _Impl (generated java) createAllTables method.
However, you would likely have issues if you deleted the file as the database connection has been passed to the migration.
So instead you could DROP all the app's tables using the code copied from the dropAllTables method from the generated java. You could then follow this with the code from the createAllTables method.
These methods are in the generated java as the class that is the same as the class that is annotated with #Database suffixed with _Impl.
The gotcha, is that the exception
(Expected .... Found ....) that you have shown is NOT within the migration but after the migration when Room is trying to build the database, so you have no control/place to do the above fall-back mimic unless this was done for all 14-15 migrations.
Perhaps what you could do is to trap the exception, present a dialog requesting the user to uninstall the app and to then re-install. This would then bypass the migration as it would be a fresh install.
I am converting my application to room database and try to follow the google architecture best practices based on "Room with a View".
I am having trouble to understand the repository in terms of clean architecture.
The Words database example contains only one table and one view using it, making it a simple HelloWorld example. But lets start with that.
There is a view which displays a list of words. Thus all words need to be read from the database and displayed.
So we have a MainActivity and a Database to connect.
Entity Word
WordDao to access DB
WordViewModel: To separate the activity lifecycle from the data lifecycle a ViewModel is used.
WordRepository: Since the data maybe kept in a database or the cloud or whatever the repository is introduced to handle decision, where data comes from.
Activity with the View
It would be nice if the view is updated when the data changes, so LiveData is used.
This in turn means, the repository is providing the LiveData for the full table:
// LiveData gives us updated words when they change.
val allWords: LiveData<List<Word>>
This is all fine for a single view.
Now to my questions on expanding this concept.
Let us assume, the word table has two columns "word" and "last_updated" as time string.
For easier comparison the time string needs to be converted to milliseconds, so I have a function.
Question: Where to put the fun queryMaxServerDateMS() to get the max(last_updated)?
/**
* #return Highest server date in table in milliseconds or 1 on empty/error.
*/
fun queryMaxServerDateMS(): Long {
val maxDateTime = wordDao.queryMaxServerDate()
var timeMS: Long = 0
if (maxDateTime != null) {
timeMS = parseDateToMillisOrZero_UTC(maxDateTime)
}
return if (timeMS <= 0) 1 else timeMS
}
For me it would be natural to put this into the WordRepository.
Second requirement: Background job to update the word list in the database.
Suppose I now want a Background Job scheduled on a regular basis which checks the server, if new entries were made and downloads them to the database. The app may not be open.
This question just relays to the question of the above queryMaxServerDateMS.
The job will basically check first, if a new entry was made by asking the server if an entry exists which is newer then the max known entry.
So I would need to get a new class WordRepository, do my query, get max last_update and ask the server.
BUT: I do not need the LiveData in the background job and when val repositoy = WordRepository the full table is read, which is needless and time-, memory and batteryconsuming.
I also can think of a number of different fragments that would require some data of the word table, but never the full data, think of a product detail screen which lists one product.
So I can move it out to another Repository or DbHelper however you want to call it.
But in the end I wonder, if I use LiveData, which requires the View, ViewModel and Repository to be closely coupled together:
Question: Do I need a repository for every activity/fragment instead of having a repository for every table which would be much more logical?
Yes, with your current architecture you should put it in the Repository.
No, you don't need a repository for every activity/fragment. Preferably, 1 repository should be created for 1 entity. You can have a UseCase for every ViewModel.
In Clean architecture there's a concept of UseCase / Interactor, that can contain business logic, and in Android it can act as an additional layer between ViewModel and Repository, you can create some UseCase class for your function queryMaxServerDateMS(), put it there and call it from any ViewModel you need.
Also you can get your LiveData value synchronously, by calling getValue().
You do not need repository for each activity or fragment. To answer your question about getting max server time - when you load words from db you pretty much have access to entire table. That means you can either do that computation yourself to decide which is the latest word that's added or you can delegate that work to room by adding another query in dao and access it in your repo. I'd prefer latter just for the simplicity of it.
To answer your question about using repo across different activities or fragment - room caches your computations so that they are available for use across different users of your repo (and eventually dao). This means if you have already computed the max server time in one activity and used it there, other lifecycle owners can use that computed result as far as the table has not been altered (there might be other conditions as well)
To summarize you're right about having repository for tables as opposed to activities or fragments
Having Room used in Android app, I created entity objects as required. Also, having decoupled layered architecture, I use Room entities only in data layer, converting them further to domain objects to be used in use case and UI layers.
Now, the problem is following:
Room provides us with very convinient way to to CRUD operations with simple anotating methods and providing Entity object.
But, how can we achieve that when Entities (and DB tables) have some meta data fields that are used only in db layer, and I don't map them to domain objects.
Here is an example of two simple classes from different layers (without annotations, for convenience):
data class CarDbEntity(id: Int, brand: String, color: String, _createdAt: Long, _someOtherMeta: Int)
data class CarDomainObject(id: Int, brand: String, color: String)
Now, I have convertors from DbEntity to DomainObject classes. But it cannot be done vice versa, since I am not mapping _metaFields to DomainObjects. Which means, since I cannot properly recreate DbEntity class, I cannot use convinient ways of crudding like this:
#Update
fun updateCar(car: CarDbEntity)
So, if I want to update car brand name or color, I need to write manually custom #Query with SQL for each case I need it.
I even tried creating CarWithNoMetaDbEntity trying to attach it to the same table as CarDbEntity, hoping that it would update just fields that exist in that class (and leave _meta as it is) but Room doesn't allow multiple entity classes bound to the same table.
Is there any technique where I can achieve updating such objects easily, without writing bunch of update SQL queries?
I'm having a problem with a simple #insert operation in a Room database. These are my classes:
The model class
#Entity(tableName = "my_model")
data class MyModel(
#PrimaryKey #ColumnInfo(name = "id_model") var uid: Int,
#ColumnInfo(name = "name") var firstName: String,
#ColumnInfo(name = "last_name") var lastName: String
)
The DAO interface
interface MyModelDAO {
#Insert
fun createMyModel(myModel: MyModel)
}
The database
#Database(
entities = [(MyModel::class)],
version = 1,
exportSchema = false
)
abstract class MyDb : RoomDatabase() {
companion object {
private var INSTANCE: MyDb? = null
fun getInstance(context: Context): MyDb? {
if (INSTANCE == null) {
synchronized(MyDb::class) {
INSTANCE = Room.databaseBuilder(context.applicationContext,
MyDb::class.java, "mydb.db")
.allowMainThreadQueries()//for testing purposes only
.build()
}
}
return INSTANCE
}
fun destroyInstance() {
INSTANCE = null
}
}
abstract fun getMyModelDao(): MyModelDAO
}
And this is how I'm trying to insert an object.
val db = MinhaDb.getInstance(this)
db?.getMyModelDao()?.createMyModel(MyModel(111, "john", "doe"))
The thing is, the operation is not persisted in the db file. If I go into the databases folder, there is a mydb file, a wal and a shm file, and no tables are created in mydb.
However, if i call db?.close() after the insert operation, the operation happens as its supposed to (the table is created and populated) and the wal and shm files are not there.
What am I missing here? I'm pretty sure I shouldn't have to call close() on the database. I've tried surrounding the insert call with a beginTransaction() and a endTransaction() calls to see if it changed anything, but it didn't.
UPDATE:
As #musooff explained in the comments, apparently that's how sqlite dbs work. I queried the database after the insert calls and, indeed, the records where returned, even though the file itself seems empty.
TL;DR
Your code seems to be working fine. Don't get confused by the temporary files SQLite creates to operate.
The WAL and SHM files are temporary internal files that you shouldn't worry about.
If you are checking if the data is present examining the db file directly, the data might not be there yet. Wait until you close the connection.
Use a SQLiteBrowser to see if the data is present or not. You can check SQLiteBrowser or Android Debug Database
Instead of using a SQLiteBrowser, you can simple check if the data is present from your Android app using a SELECT query.
WAL and SHM files
As you have noticed, in your db directory you can find three generated files:
your-database-name
your-database-name-shm
your-database-name-wal
However, the only important for you, where the data really is, is your-database-name.
The wal file is used as a replacement of the Rollback Journal.
Beginning with version 3.7.0 (2010-07-21), SQLite supports a new transaction control mechanism called "write-ahead log" or "WAL". When a database is in WAL mode, all connections to that database must use the WAL. A particular database will use either a rollback journal or a WAL, but not both at the same time. The WAL is always located in the same directory as the database file and has the same name as the database file but with the string "-wal" appended.
What you mention about not being able to see the data in your database file while the wal file is still there, and then you close the connection and the wal file is gone and the data is finally persisted in the database, is the proper behavior while using the wal mechanism.
WAL dissapears
The WAL file exists for as long as any database connection has the database open. Usually, the WAL file is deleted automatically when the last connection to the database closes. (More here)
Transaction not being written immediately to database file
The traditional rollback journal works by writing a copy of the original unchanged database content into a separate rollback journal file and then writing changes directly into the database file. In the event of a crash or ROLLBACK, the original content contained in the rollback journal is played back into the database file to revert the database file to its original state. The COMMIT occurs when the rollback journal is deleted.
The WAL approach inverts this. The original content is preserved in
the database file and the changes are appended into a separate WAL
file. A COMMIT occurs when a special record indicating a commit is
appended to the WAL. Thus a COMMIT can happen without ever writing to
the original database, which allows readers to continue operating from
the original unaltered database while changes are simultaneously being
committed into the WAL. Multiple transactions can be appended to the
end of a single WAL file.
Of course, one wants to eventually transfer all the transactions that are appended in the WAL file back into the original database.
Moving the WAL file transactions back into the database is called a
"checkpoint".
By default, SQLite does a checkpoint automatically when the WAL file
reaches a threshold size of 1000 pages. (The
SQLITE_DEFAULT_WAL_AUTOCHECKPOINT compile-time option can be used to
specify a different default.) Applications using WAL do not have to do
anything in order to for these checkpoints to occur. But if they want
to, applications can adjust the automatic checkpoint threshold. Or
they can turn off the automatic checkpoints and run checkpoints during
idle moments or in a separate thread or process. (More here)
The SHM is just a temporary shared memory file, related to the WAL mechanism, whose only purpose is:
The shared-memory file contains no persistent content. The only purpose of the shared-memory file is to provide a block of shared memory for use by multiple processes all accessing the same database in WAL mode. (More here)