I would like to perform an asynchonous operation on each record in a large Room table.
I thought I could add a method returning Flow in my DAO like this:
#Query("SELECT * FROM events")
fun getEvents(): Flow<EventEntity>
But according to this blog post and this documentation page returning a Flow is making an "observable read" so the Flow never completes and it watches for database changes.
My goal is to iterate over all the entities only once. I don't want the "observability" behavior. Also, since the table is very large, I don't want to load all the records into a List at once in order to avoid consuming too much memory.
Could you recommend some solution, please?
Create a new method that does not use Flow.
#Query("SELECT id FROM events")
fun getAllIds(): List<Int> // If your primary key is Integer.
#Query("SELECT * FROM events WHERE id = :id")
fun getById(id: Int): EventEntity?
Use Kotlin coroutines to call this method on IO thread.
There could be several strategies to load one row at a time. This is the simplest - get all ids and load each item one at a time.
suspend fun getEvents() {
withContext(Dispatchers.IO) {
// Get entities from database on IO thread.
val ids = dao.getAllIds()
ids.forEach { id ->
val event = dao.getById(id)
}
}
}
Pagination based approach
This approach assumes that you have a column that stores timestamp (eg. created_at).
#Query("SELECT * from events WHERE created_at > :timestamp ORDER BY created_at LIMIT 10")
fun getAfter(timestamp: Long): List<EventEntity>
You can use this method to paginate.
suspend fun getEvents() {
withContext(Dispatchers.IO) {
var timestamp: Long = 0
while (true) {
// Get entities from database on IO thread.
val events = dao.getAfter(timestamp)
// Process this batch of events
// Get timestamp for pagination offset.
timestamp = events.maxBy { it.createAt }?.createAt ?: -1
if (timestamp == -1) {
// break the loop. This will be -1 only if the list of events are empty.
}
}
}
}
Related
Is it possible to check if Flow sends back a value and to act on it if it doesn't?
override suspend fun insertUserResponse(userResponse: UserResponse) {
val userResponseFromBDD: Flow<UserResponse>? = userResponseDAO.searchUserByToken(userResponse.profilePOJO.uniqueID)
userResponseFromBDD?.collect {
userResponseDAO.updateUser(userResponse)
} ?: {
userResponseDAO.insertUser(userResponse)
}
}
Several remarks:
There are two types of queries - one-shot and streams. For your use-case you need one-shot query (to ge response or null once with searchUserByToken and once - to insert or update value). For one-shot queries in Room you can use suspend function with no Flow:
#Query("SELECT * FROM userResponse where id = :id")
suspend fun searchUserByToken(id: Int):UserResponse?
And your code with checking value would be:
override suspend fun insertUserResponse(userResponse: UserResponse) {
val userResponseFromBDD = userResponseDAO.searchUserByToken(userResponse.profilePOJO.uniqueID)
userResponseFromBDD?.let { userResponseDAO.updateUser(userResponse)}
?: userResponseDAO.insertUser(userResponse)
}
With Flow you get stream, that issues values whenever data is updated in DB. That's why in your use-case you can get values in loop: you get value value from searchUserByToken -> you update value -> you get new value since Room uses Flow and invokes searchUserByToken again -> you update value -> ...
If you use Room's #Insert(onConflict = OnConflictStrategy.REPLACE) you can not to check if userResponse is in DB and use just insertUser method (since if user is in db insert would cause it's update)
I have an app which has a session end routine. I want to update the session with the end date/time and potentially other information when the End Session button is clicked. I have a dao, a repository, and a ViewModel.
I thought the best way to do this would be to get the record, update the fields in the object, and save the object back to Room.
I don't exactly know the best way to go about this. Here is the code I am working with:
In the Dao:
#Query("SELECT * FROM Session WHERE id=:id")
Single<Session> getSessionById(int id);
In the repository:
public Single<Session> getSessionById(int sessionId) {
return sessionDao.getSessionById(sessionId);
}
In the ViewModel:
public void endSession () {
Single<Session> session = sessionRepository.getSessionById(sessionId);
//????? session.doOnSuccess()
//get current date/time
Date date = Calendar.getInstance().getTime();
//set the end date
session.setEndTime(date);
//update the session
sessionRepository.update(session);
}
I would love any advice on the range of options. I had started using a plain object, but got errors related to running the query on the main thread and wanted to avoid that. I don't need an observable/flowable object. I understand Async is to be deprecated in Android 11. So I thought Single would work.
Any advice on my choice to use Single or code would be really helpful.
UPDATE:
I just need a simple example in Java of pulling a single record from and the main part is the functionality in the ViewModel (what does the doOnSuccess look like and optionally on error as well).
If you just want to update without retrieving the whole record, writing a custom query is the option that I go with:
#Query("UPDATE table_name SET column_name = :value WHERE id = :id")
fun update(id: Int, value: String): Completable
In repository:
fun update(id: Int, value: String): Completable
In ViewModel or UseCase:
update().subscribeOn(...).observeOn(...).subscribe()
If you want to avoid using RxJava:
#Query("UPDATE table_name SET column_name = value WHERE id = :id")
fun update(id: Int, value: String): Boolean
And use Executors to run transactions on a background thread.
Executors.newSingleThreadExecutor().execute {
repository.update()
}
If you want to perform both retrieving and updating you can use #Update and #Query to retrieve the recorded inside your ViewModel or UseCase (interactor) and push the update toward Room.
RxJava Single Example:
#Query("SELECT * FROM table_name")
fun getAll(): Single<List<Model>>
Repository:
fun getAll(): Single<List<Model>> = modelDao.getAll()
ViewModel or UseCase:
val statusLiveData = MutableLive<Strig>()
modelRepository.getAll()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({ statusLiveData.value = "Success" }, { statusLiveData.value = "Failed" })
I am using Room Database to make a database to store information in a table. I want to access one entry from the table and delete the same entry without the need to call two functions.
#Query("SELECT * FROM history_packet_table ORDER BY timestamp ASC LIMIT 1")
fun get(): HistoryPacket?
#Query("DELETE FROM history_packet_table ORDER BY timestamp ASC LIMIT 1")
fun delete()
I want these two operations to happen only by calling get. Is there a way?
I believe that you can add the following to the Dao :-
#Transaction
fun getAndDelete() {
get()
delete()
}
Obviously you can call the function what you wish. However, the get seems to be useless as it is.
So you may want something like :-
#Query("SELECT * FROM history_packet_table WHERE timestamp = (SELECT min(timestamp) FROM history_packet_table)")
fun get() :HistoryPacketTable
#Query("DELETE FROM history_packet_table WHERE timestamp = (SELECT min(timestamp) FROM history_packet_table)")
fun delete() :Int
#Transaction
fun getAndDelete() :HistoryPacketTable {
// Anything inside this method runs in a single transaction.
var rv: HistoryPacketTable = get()
val rowsDeleted: Int = delete()
if (rowsDeleted < 1) {
rv = HistoryPacketTable();
//....... set values of rv to indicate not deleted if needed
}
return rv
}
Note as LIMIT on delete is turned off by default, the queries can be as above, this assumes that timestamp is unique otherwise multiple rows may be deleted, in which case the Dao could be something like
:-
#Delete
fun delete(historyPacketTable: HistoryPacketTable) :Int
#Transaction
fun getAndDelete() :HistoryPacketTable {
// Anything inside this method runs in a single transaction.
var rv: HistoryPacketTable = get()
val rowsDeleted: Int = delete(rv)
if (rowsDeleted < 1) {
rv = HistoryPacketTable();
//....... set values to indicate not deleted
}
return rv
}
I am using room database to store comments and RxJava as a listener to do some stuff when the database is changed.
I want to not call the callback when delete is called on the table, only when insert is called.
What i found out so far is that Room library has triggers that are called on delete, insert and update of the table that in turn call RxJava's methods.
Is there any way to drop the delete trigger and get callbacks only for the insert and update methods?
Here is my CommentDAO:
#Query("SELECT * FROM comments" )
fun getAll(): Flowable<List<Comment>>
#Insert(onConflict = OnConflictStrategy.REPLACE)
fun insert(comment: Comment)
#Delete
fun delete(comment: Comment)
And my RxJava callback functions:
/**
* Inserts comment into comment database
*
* #param object that's going to be inserted to the database
*/
fun saveComment(comment: Comment) {
Observable.just(comment).subscribeOn(Schedulers.io()).map({ comment1 -> commentdb.commentDao().insert(comment1) }).subscribe()
}
/**
* Removes comment from the database
*
* #param comment object that's going to be removed
*/
fun removeComment(comment: Comment){
Observable.just(comment).subscribeOn(Schedulers.io()).map({ comment1 -> commentdb.commentDao().delete(comment1) }).subscribe()
}
fun createCommentObservable(uploader: CommentUploader) {
commentdb.commentDao().getAll().subscribeOn(Schedulers.io()).observeOn(AndroidSchedulers.mainThread()).subscribe(
{
success -> uploader.queue(success)
}
)
}
You can get a Flowable<List<Comment>> that only emits on insertions and not on deletions by filtering the original getAll() Flowable so that only those List<Comment> items are passed through that contain more Comments than the previous List<Comment>.
You can implement this filtering with the following transformations:
Prepend the flowable with an empty list so that we have a baseline for insertions.
Get RxJava window()s of size 2, so that we will be able to compare adjacent items.
window() returns Flowable<Flowable<Comment>>. Convert it to Flowable<List<Comment>> with flatMap() and toList() on the inner Flowable.
Filter those 2-element windows that represent an insertion (the size of the first element is less than the size of the second).
Emit only the 2nd element of the filtered windows.
In Kotlin:
fun getAllAfterInsertions() {
getAll()
.startWith(emptyList<String>()) // (1)
.window(2, 1) // (2)
.flatMap({ w -> w.toList().toFlowable() }) // (3)
.filter({ w -> w.size == 2 && w[0].size < w[1].size }) // (4)
.map({ window -> window[1] }) // (5)
}
To delete without notification I simply replace
MyDao().delete()
with one executing a #Query
MyDao().deleteLast()
then thew Flowable doesn't emit a new event. The #Dao looks like this
#Dao
abstract class MyDao : BaseDao<Data> {
#Query("DELETE FROM Data WHERE id = (select min(id) from Data)") // or something else
abstract fun deleteLast()
#Delete
fun delete(data: Data)
}
I need to request from local storage and if it's empty I should request to Service and store in DB ,but if the DB isn't empty I should ignore the second part.
I'm using Room persistence DB
( Flowable), I mean if I store in DB , I could listen to changes .
I'm using concat but neighter of the parts are working
val item1 = itemDao.loadItem(id)
val item2 = apiIcaSeResource
.fetchItem(offerId)
.toFlowable()
.doOnNext { item ->itemDao.saveItem(Item(...) }
Flowable.concat(item1, item2)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
// items ->
}, {
Timber.e(it, "Error reading items")
})
#Query("SELECT * FROM offer WHERE id = :offerId")
fun loadItem( offerId: String): Flowable<Item>
#Insert(onConflict = OnConflictStrategy.REPLACE)
fun saveItem(items: Item)
#GET("item/{itemId}")
Single<Item> fetchItem(#Path("itemId") Long itemId);
#Query("SELECT * FROM offer WHERE id = :offerId")
fun loadItem( offerId: String): Flowable<Item>
With the Room DAO defined like this, if an item does not exist the Flowable will emit nothing. There will be no onNext events. Also defined as Flowable Room never emits onComplete.
Flowable.concat waits for the first Flowable to emit onComplete then subscribes to the second Flowable and emits all items.
Since alpha5 you can specify the return type in Room as Maybe<Item> so you can detect if there is an existing item in the db. Another option is to define the return type as Flowable<List<Item>> and Room will emit an empty list if the item does not exist in the DB.
If you switch Room to:
#Query("SELECT * FROM offer WHERE id = :offerId")
fun loadItem( offerId: String): Flowable<List<Item>>
you could do something like this (calls the API only if there are 0 items in the DB):
item1.flatMap { if(it.size == 0) item2 else Flowable.just(it) }
Note that this will end the stream if there is an error in the API call. Check the onErrorXXX operators to work around that.