I am using LiveData to observe a Room query that filters a table on a boolean in the table. The boolean is whether the row has been uploaded to an external database or not.
Here is the query I'm observing:
memodao.kt
#Query("SELECT * FROM memos WHERE uploaded = 0 ORDER BY id ASC")
fun getUnsyncedMemos(): LiveData<List<Memo>>
//including this statement because it's used later
#Update
suspend fun update(memo: Memo): Int
When the LiveData fires, I grab that row, upload it to the external database using Retrofit, and then when Retrofit returns I mark that row as uploaded to exclude it from the above query.
Writing to the table I am observing causes the above query to trigger again, as expected.
The trouble is, from time to time the query returns the same row again, even though it NO LONGER meets the where clause.
It actually returns a stale version of the row, that still meets the where clause.
It only happens from time to time. To me it feels like a race condition: the beginning of the Room #Update statement triggers the #Query to execute again, and sometimes the row hasn't been rewritten in time, so the #Query returns a stale result.
Here is the rest of the relevant code.
CloudSyncService.kt
mMemoRepository.getUnsyncedMemos().observeForever{ memoList ->
memoList.forEach { memo ->
if (!memo.uploaded) uploadMemo(memo) //an unnecessary if, I know.
}
}
private fun uploadMemo(memo: Memo) {
mMemoRepository.uploadMemo(memo).observeOnce(this, Observer {
mCloudOK = it != CLOUD_FAILED
})
}
MemoRepository.kt
override fun getUnsyncedMemos() = memoDao.getUnsyncedMemos()
override fun uploadMemo(memo: Memo): LiveData<Int> {
val mutableLiveData = MutableLiveData<Int>()
CoroutineScope(IO).launch{
memoWebService.uploadMemo(memo).enqueue(object : Callback<Memo> {
override fun onResponse(call: Call<Memo>, response: Response<Memo>) {
if (response.isSuccessful) {
memo.uploaded = true
updateMemo(memo)
mutableLiveData.value = CLOUD_OK
}
else {
mutableLiveData.value = CLOUD_UNKNOWN
}
}
override fun onFailure(call: Call<Memo>, t: Throwable) {
mutableLiveData.value = CLOUD_FAILED
mMutableErrorMessage.value = t.message
}
})
}
return mutableLiveData
}
I'm wondering if it's an issue with the co-routines throwing things out of sync and creating a race? I've moved calls in and out of co-routines to try to fix it, but I still get the problem.
I have read about the issue where LiveData returns stale data: Room LiveData fires twice with one stale emission
I believe my question differs, because I am not adding new observers that receive stale data in the first firing. I have one observer, observing forever, that is occasionally getting stale data after a db #Update.
Many thanks!
Related
I'm trying to understand the logic of switchIfEmpty operator. I will be very thankful for every explanation.
I have a local database (Room) and remote server. My goal is to implement logic with switchIfEmpty to check if there is data in local DB to take it and if local DB is empty to call from remote. The process starts in activity where I subscribe to Observable:
private fun subscribeOnDataChanges() = with(viewModel) {
requestNextPage()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
filmsAdapter.addItems(it)
}, {
errorEvent.value = ERROR_MSG
it.printStackTrace()
}).addTo(autoDisposable)
}
Then the methods in the View Model:
fun requestNextPage(): Observable<List<Film>> {
return requestPageOfFilms()
}
private fun requestPageOfFilms(): Observable<List<Film>> =
interactor.requestPageOfFilmsFromDataSource()
And, finally the method with switchIfEmpty in the Interactor:
fun requestPageOfFilmsFromDataSource(): Observable<List<Film>> {
return repo.getPageOfFilmsInCategory(category).filter { it.isNotEmpty() }.switchIfEmpty(
getFromRemote(category)
)
}
private fun getFromRemote(category: String): Observable<List<Film>> {
return convertSingleApiToObservableDtoList(
retrofitService.getFilms(
category, API.KEY, "ru-RU", NEXT_PAGE
)
)
}
I cannot understand the next things:
Why, when local db (repo) is NOT empty, getFromRemote() is called?
If local db is empty, why network call in the method getFromRemote() is not performed? No matters, that I subscribed in the activity? Because if I add the subscription inside the switchIfEmpty(), the network call is performed.
To answer your questions :
Why, when local db (repo) is NOT empty, getFromRemote() is called?
Because this function convertSingleApiToObservableDtoList() is being evaluated at Assembly Time and not Subscription Time (https://github.com/ReactiveX/RxJava#assembly-time). The result is that the Observable object is eagerly evaluated.
If local db is empty, why network call in the method getFromRemote() is not performed? No matters, that I subscribed in the activity? Because if I add the subscription inside the switchIfEmpty(), the network call is performed.
Because the upstream has not emitted a value yet or completed. You are conflating an empty List with an empty Observable (https://reactivex.io/documentation/operators/defaultifempty.html)
A better solution to your problem would be using the flatMap or even switchMap operator if you expect a chatty upstream and only care about latest result, something like :
fun requestPageOfFilmsFromDataSource(): Observable<List<Film>> {
return repo.getPageOfFilmsInCategory(category)
.flatMap { films ->
if (films.isNotEmpty()) {
Observable.just(films)
} else getFromRemote(category)
}
}
I have got the following method:
operator fun invoke(query: String): Flow<MutableList<JobDomainModel>> = flow {
val jobDomainModelList = mutableListOf<JobDomainModel>()
jobListingRepository.searchJobs(sanitizeSearchQuery(query))
.collect { jobEntityList: List<JobEntity> ->
for (jobEntity in jobEntityList) {
categoriesRepository.getCategoryById(jobEntity.categoryId)
.collect { categoryEntity ->
if (categoryEntity.categoryId == jobEntity.categoryId) {
jobDomainModelList.add(jobEntity.toDomainModel(categoryEntity))
}
}
}
emit(jobDomainModelList)
}
}
It searches in a repository calling the search method that returns a Flow<List<JobEntity>>. Then for every JobEntity in the flow, I need to fetch from the DB the category to which that job belongs. Once I have that category and the job, I can convert the job to a domain model object (JobDomainModel) and add it to a list, which will be returned in a flow as the return object of the method.
The problem I'm having is that nothing is ever emitted. I'm not sure if I'm missing something from working with flows in Kotlin, but I don't fetch the category by ID (categoriesRepository.getCategoryById(jobEntity.categoryId)) it then works fine and the list is emitted.
Thanks a lot in advance!
I think the problem is that you're collecting infinite length Flows, so collect never returns. You should use .take(1) to get a finite Flow before collecting it, or use first().
The Flows returned by your DAO are infinite length. The first value is the first query made, but the Flow will continue forever until cancelled. Each item in the Flow is a new query made when the contents of the database change.
Something like this:
operator fun invoke(query: String): Flow<MutableList<JobDomainModel>> =
jobListingRepository.searchJobs(sanitizeSearchQuery(query))
.map { jobEntityList: List<JobEntity> ->
jobEntityList.mapNotNull { jobEntity ->
categoriesRepository.getCategoryById(jobEntity.categoryId)
.first()
.takeIf { it.categoryId == jobEntity.categoryId }
}
}
Alternatively, in your DAO you could make a suspend function version of getCategoryById() that simply returns the list.
Get an idea from the code below if your Kotlin coroutine flow gets lost with a continuation approximate peak alloc exception
fun test(obj1: Object,obj2: Object) = flow {
emit(if (obj1 != null) repository.postObj(obj1).first() else IgnoreObjResponse)
}.map { Pair(it, repository.postObj(obj2).first()) }
I am investigating the new Android Room Paging library
implementation "androidx.paging:paging-runtime-ktx:3.0.0-alpha09"
My source database tables approx 10,000 rows and I filter by the first character of the name field as follows:-
DAO
#Query("SELECT * from citation_style WHERE citation_style_name LIKE :startsWith ORDER BY citation_style_name ASC")
fun fetch(startsWith: String): PagingSource<Int, CitationStyleDO>
Repository
fun fetch(startsWith: String): Flow<PagingData<CitationStyleDO>> {
return Pager(
PagingConfig(pageSize = 60, prefetchDistance = 30, enablePlaceholders = false, maxSize = 200)
) { database.citationStyleDao().fetch("$startsWith%") }.flow
}
ViewModel
fun fetch(startsWith: String): Flow<PagingData<CitationStyleDO>> {
return repository.fetch(startsWith).cachedIn(viewModelScope)
}
Fragment
override fun onStartsWithClicked(startsWith: String) {
lifecycleScope.launch {
viewModel.fetch(startsWith).collectLatest { adapter.submitData(it) }
}
}
Is this the correct approach of repeatedly using lifecycleScope.launch {...} each time the Starts With character is changed?
Should I be map{} or switchMap{} triggered by a MutabaleLiveData<String> for StartwWith?
This won't work, because submitData doesn't return until PagingData is invalidated. You might run into race scenarios where you have multiple jobs launched where PagingDataAdapter is trying to collect from multiple PagingData.
The more "Flow" way would be to turn your fetch calls into a flow and combine that with your Flow<PagingData>, which will automatically propagate cancellation every time your query changes.
A couple other things:
It's recommended to let Paging do the filtering for you, as this way you can avoid re-fetching from DB every time your search changes, and also you can lean on Paging to handle configuration changes and restore state.
You should use viewLifecycleOwner instead of lifecycleScope directly, because you don't want paging to do work after the fragment's view is destroyed
e.g.,
ViewModel
val queryFlow = MutableStateFlow("init_query")
val pagingDataFlow = Pager(...) {
dao.pagingSource()
}.flow
// This multicasts, to prevent combine from refetching
.cachedIn(viewModelScope)
.combine(queryFlow) { pagingData, query ->
pagingData.filter { it.startsWith(query)
}
// Optionally call .cachedIn() here a second time to cache the filtered results.
Fragment
override fun onStartsWithClicked(startsWith: String) {
viewModel.queryFlow = startsWith
}
override fun onViewCreated(...) {
viewLifecycleOwner.lifecycleScope.launch {
viewModel.pagingDataFlow.collectLatest { adapter.submitData(it) }
}
Note: You can definitely use Room to do the filtering if you want, probably the right way there is to .flatMapLatest on the queryFlow and return a new Pager each tine, and pass the query term to dao function that returns a PagingSource
ViewModel
queryFlow.flatMapLatest { query ->
Pager(...) { dao.pagingSource(query) }
.cachedIn(...)
}
I have my architecture like so:
Dao methods returning Flow<T>:
#Query("SELECT * FROM table WHERE id = :id")
fun itemById(id: Int): Flow<Item>
Repository layer returning items from DB but also backfilling from network:
(* Need help here -- this is not working as intended **)
fun items(): Flow<Item> = flow {
// Immediately emit values from DB
emitAll(itemDao.itemById(1))
// Backfill DB via network request without blocking coroutine
itemApi.makeRequest()
.also { insert(it) }
}
ViewModel layer taking the flow, applying any transformations, and converting it into a LiveData using .asLiveData():
fun observeItem(): LiveData<Item> = itemRepository.getItemFlow()
.map { // apply transformation to view model }
.asLiveData()
Fragment observing LiveData emissions and updating UI:
viewModel.item().observeNotNull(viewLifecycleOwner) {
renderUI(it)
}
The issue I'm having is at step 2. I can't seem to figure out a way to structure the logic so that I can emit the items from Flow immediately, but also perform the network fetch without waiting.
Since the fetch from network logic is in the same suspend function it'll wait for the network request to finish before emitting the results downstream. But I just want to fire that request independently since I'm not interested in waiting for a result (when it comes back, it'll update Room and I'll get the results naturally).
Any thoughts?
EDIT
Marko's solution works well for me, but I did attempt a similar approach like so:
suspend fun items(): Flow<List<Cryptocurrency>> = coroutineScope {
launch {
itemApi.makeRequest().also { insert(it) }
}
itemDao.itemById(1)
}
It sounds like you're describing a background task that you want to launch. For that you need access to your coroutine scope, so items() should be an extension function on CoroutineScope:
fun CoroutineScope.items(): Flow<Item> {
launch {
itemApi.makeRequest().also { insert(it) }
}
return flow {
emitAll(itemDao.itemById(1))
}
}
On the other hand, if you'd like to start a remote fetch whose result will also become a part of the response, you can do it as follows:
fun items(): Flow<Item> = flow {
coroutineScope {
val lateItem = async { itemApi.makeRequest().also { insert(it) } }
emitAll(itemDao.itemById(1))
emit(lateItem.await())
}
}
So, I've playing with Android's new Paging Library part of the JetPack tools.
I'm doing a very basic demo App where I show a list of random user profiles that I get from the RandomUser API.
The thing is that I have everything set and It's actually kinda working.
In my the list's fragment I'm listening for the DB changes:
...
mainViewModel.userProfiles.observe(this, Observer { flowableList ->
usersAdapter.submitList(flowableList) })
...
(I tried with both, using Flowables and Observables with the RxPagedListBuilder and now I'm using LiveData with the LivePagedListBuilder. I'd like to use the RxPagedListBuilder if it's possible)
In my ViewModel I'm setting the LivePagedListBuilder
...
private val config = PagedList.Config.Builder()
.setEnablePlaceholders(false)
.setPrefetchDistance(UserProfileDataSource.PAGE_SIZE / 2) //Is very important that the page size is the same everywhere!
.setPageSize(UserProfileDataSource.PAGE_SIZE)
.build()
val userProfiles: LiveData<PagedList<UserProfile>> = LivePagedListBuilder(
UserProfileDataSourceFactory(userProfileDao), config)
.setBoundaryCallback(UserProfileBoundaryCallback(randomUserRepository, userProfileDao))
.build()
...
I've set the DataSource to fetch the data form my DB (if there's any):
class UserProfileDataSource(val userDao: UserDao): PageKeyedDataSource<Int, UserProfile>() {
companion object {
const val PAGE_SIZE = 10
}
override fun loadInitial(params: LoadInitialParams<Int>, callback: LoadInitialCallback<Int, UserProfile>) {
callback.onResult(userDao.getUserProfileWithLimitAndOffset(PAGE_SIZE, 0), 0, PAGE_SIZE)
}
override fun loadAfter(params: LoadParams<Int>, callback: LoadCallback<Int, UserProfile>) {
callback.onResult(userDao.getUserProfileWithLimitAndOffset(PAGE_SIZE, params.key), params.key + PAGE_SIZE)
}
override fun loadBefore(params: LoadParams<Int>, callback: LoadCallback<Int, UserProfile>) {
//This one is not used
}
}
And I've set the BoundaryCallback to fetch the data from network if the database is empty:
class UserProfileBoundaryCallback(val userRepository: RandomUserRepository,
val userDao: UserDao) : PagedList.BoundaryCallback<UserProfile>() {
/**
* Database returned 0 items. We should query the backend for more items.
*/
override fun onZeroItemsLoaded() {
userRepository.getUserProfiles(0, UserProfileDataSource.PAGE_SIZE)
.subscribeOn(Schedulers.io()) //We shouldn't write in the db over the UI thread!
.subscribe(object: DataSourceSubscriber<List<UserProfile>>() {
override fun onResultNext(userProfiles: List<UserProfile>) {
userDao.storeUserProfiles(userProfiles)
}
})
}
/**
* User reached to the end of the list.
*/
override fun onItemAtEndLoaded(itemAtEnd: UserProfile) {
userRepository.getUserProfiles(itemAtEnd.indexPageNumber + 1, UserProfileDataSource.PAGE_SIZE)
.subscribeOn(Schedulers.io()) //We shouldn't write in the db over the UI thread!
.subscribe(object: DataSourceSubscriber<List<UserProfile>>() {
override fun onResultNext(userProfiles: List<UserProfile>) {
userDao.storeUserProfiles(userProfiles)
}
})
}
}
The thing is that the Flowable, Observable or LiveData in the UI only gets triggered if the data is coming from the database, when the UserProfileDataSource returns some result from database. If the UserProfileDataSource doesn't return any result, the BoundaryCallback gets called, the request to the API is successfully done, the data gets stored in the database, but the Flowable never gets triggered. If I close the App and open it again, is gonna fetch the data that we just got from the database and then is going to display it properly.
I've tried every possible solution that I saw in all the articles, tutorials and github projects that I searched.
As I said I tried to change the RxPagedListBuilder to a LivePagedListBuilder.
When using RxPagedListBuilder I tried with both, Flowables and Observables.
I tried setting different configurations for the PagedList.Config.Builder using different Schedulers.
I'm really out of options here, I've been all day long struggling with this, any tip will be appreciated.
If you need more info about the code, just leave me a comment and I'll update the post.
Update:
I've submitted the full source code to my Github repo
Okay, I found the error:
I just removed the UserProfileDataSourceFactory and the UserProfileDataSource classes. Instead of that I added this method to my DAO:
#Query("SELECT * FROM user_profile")
fun getUserProfiles(): DataSource.Factory<Int, UserProfile>
And I'm building my RxPagedListBuilder like:
RxPagedListBuilder(userDao.getUserProfiles(), UserProfileDataSource.PAGE_SIZE)
.setBoundaryCallback(UserProfileBoundaryCallback(randomUserApi, userDao))
.buildObservable()
And that's it. I'm not sure why it doesn't work with the DataSource and the DataSourceFactory, if anyone can explain this oddly behaviour I'll mark that answer as accepted since my solution was more like a lucky shot.
One thing that is happening now though, is that when I load the data into the adapter, the adapter scrolls back to the top position.
If you implement the DataSource , you need to call DataSource#invalidate() after operating room database every time.