Develop Reference

returning a list of data to viewModel using Repository pattern from different data sources (network and room)

I'm developing a Kotlin app using MVVM with repository pattern.
I have a main fragment that displays list of asteroids.
Here is the flow I want to achieve.
When user opens main fragment , I will check if there's asteroids data stored in the local database (room).
If yes I will displays the stored data
If no I will call API to get the asteroids then I will store the data from API to local database
So based on my understating of the repository pattern the viewModel should not be concerned about the data source wether it's from API or local database.
So I defined this function in repository that the viewModel will call
override suspend fun getAsteroid(): List<Asteroid> {
var result : List<Asteroid>
try {
var isDataAvailable = getAnyAsteroidFromDb()
if (isDataAvailable == null) {
result = getAsteroidApi().asDomainModel()
} else {
result = getAsteroidFromDb()
}
} catch (e : Exception) {
}
return result
}
getAnyAsteroidFromDb checks the data availability in room
getAsteroidApi gets the data from netwrok call
getAsteroidFromDb gets the data from room
The problem is getAsteroidFromDb returns Livedata from Dao
#Query("SELECT * FROM asteroid_tbl")
fun getAsteroidsFromDb () : LiveData<List<AsteroidEntity>>
And the function itself getAsteroid returns List of Asteroids.
Let's say I changed the return type to Livedata , This will cause another problem because the api doesn't return a Livedata.
I'm kind of stuck here and I think I'm doing something wrong or maybe my understating of the MVVM and repertory pattern still not good enough.
Any thoughts or idea will be appreciated!

There are more than one solutions to this, but since you are using the MVVM pattern. So I would like to suggest you a pattern for such situations.
Whenever there is a situation where have to check our Local DB or call our backend API. The usual structure of calling is to have one point of data source only to avoid ambiguity and such issues.
So while you can easily get the value from LiveData or convert the API response to LiveData. I would suggest your structure to be as follows.
Check Local DB for data.
If data is there in Local DB, fetch it.
If data is not there in Local DB, call the API, fetch the results, store it in Local DB.
On success of API result, you can then query the local DB again and get results.
This ensures your single source of truth remains your DB and you can easily make calls.

You should be able to return the LiveData value:
override suspend fun getAsteroid(): List<Asteroid> {
var result : List<Asteroid>
try {
var isDataAvailable = getAnyAsteroidFromDb()
if (isDataAvailable == null) {
result = getAsteroidApi().asDomainModel()
} else {
result = getAsteroidFromDb().value //<==List<Asteroid>
}
} catch (e : Exception) {
}
return result
}

If you don't want to return Livedata from the repository, you can unwrap the Livedata you get from Room and use the value from it. That should give you the list of Asteroids object. So, use something like this:
override suspend fun getAsteroid(): List<Asteroid> {
var result : List<Asteroid>
try {
var isDataAvailable = getAnyAsteroidFromDb()
if (isDataAvailable == null) {
result = getAsteroidApi().asDomainModel()
} else {
val data = getAsteroidFromDb()
result = data.value
}
} catch (e : Exception) {
}
return result
}

Android: collecting a Kotlin Flow inside another not emitting

I have got the following method:
operator fun invoke(query: String): Flow<MutableList<JobDomainModel>> = flow {
val jobDomainModelList = mutableListOf<JobDomainModel>()
jobListingRepository.searchJobs(sanitizeSearchQuery(query))
.collect { jobEntityList: List<JobEntity> ->
for (jobEntity in jobEntityList) {
categoriesRepository.getCategoryById(jobEntity.categoryId)
.collect { categoryEntity ->
if (categoryEntity.categoryId == jobEntity.categoryId) {
jobDomainModelList.add(jobEntity.toDomainModel(categoryEntity))
}
}
}
emit(jobDomainModelList)
}
}
It searches in a repository calling the search method that returns a Flow<List<JobEntity>>. Then for every JobEntity in the flow, I need to fetch from the DB the category to which that job belongs. Once I have that category and the job, I can convert the job to a domain model object (JobDomainModel) and add it to a list, which will be returned in a flow as the return object of the method.
The problem I'm having is that nothing is ever emitted. I'm not sure if I'm missing something from working with flows in Kotlin, but I don't fetch the category by ID (categoriesRepository.getCategoryById(jobEntity.categoryId)) it then works fine and the list is emitted.
Thanks a lot in advance!

I think the problem is that you're collecting infinite length Flows, so collect never returns. You should use .take(1) to get a finite Flow before collecting it, or use first().
The Flows returned by your DAO are infinite length. The first value is the first query made, but the Flow will continue forever until cancelled. Each item in the Flow is a new query made when the contents of the database change.
Something like this:
operator fun invoke(query: String): Flow<MutableList<JobDomainModel>> =
jobListingRepository.searchJobs(sanitizeSearchQuery(query))
.map { jobEntityList: List<JobEntity> ->
jobEntityList.mapNotNull { jobEntity ->
categoriesRepository.getCategoryById(jobEntity.categoryId)
.first()
.takeIf { it.categoryId == jobEntity.categoryId }
}
}
Alternatively, in your DAO you could make a suspend function version of getCategoryById() that simply returns the list.

Get an idea from the code below if your Kotlin coroutine flow gets lost with a continuation approximate peak alloc exception
fun test(obj1: Object,obj2: Object) = flow {
emit(if (obj1 != null) repository.postObj(obj1).first() else IgnoreObjResponse)
}.map { Pair(it, repository.postObj(obj2).first()) }

Loading data from Database + Network (Room + Retrofit + RxJava2)

I have a sample API request which returns a list of user's watchlist. I want to achieve the following flow when the user loads the watchlist screen:
Load the data from DB cache immediately.(cacheWatchList)
Initiate the RetroFit network call in the background.
i. onSuccess return apiWatchList
ii. onError return cacheWatchList
Diff cacheWatchList vs apiWatchList
i. Same -> all is well since data is already displayed to the user do nothing.
ii. Differs -> Save apiWatchList to a local store and send the apiWatchList to the downstream.
What I have done so far?
Watchlist.kt
data class Watchlist(
val items: List<Repository> = emptyList()
)
LocalStore.kt (Android room)
fun saveUserWatchlist(repositories: List<Repository>): Completable {
return Completable.fromCallable {
watchlistDao.saveAllUserWatchlist(*repositories.toTypedArray())
}
}
RemoteStore.kt (Retrofit api call)
fun getWatchlist(userId: UUID): Single<Watchlist?> {
return api.getWatchlist(userId)
}
DataManager.kt
fun getWatchlist(userId: UUID): Flowable<List<Repository>?> {
val localSource: Single<List<Repository>?> =
localStore.getUserWatchlist()
.subscribeOn(scheduler.computation)
val remoteSource: Single<List<Repository>> = remoteStore.getWatchlist(userId)
.map(Watchlist::items)
.doOnSuccess { items: List<Repository> ->
localStore.saveUserWatchlist(items)
.subscribeOn(scheduler.io)
.subscribe()
}
.onErrorResumeNext { throwable ->
if (throwable is IOException) {
return#onErrorResumeNext localStore.getUserWatchlist()
}
return#onErrorResumeNext Single.error(throwable)
}
.subscribeOn(scheduler.io)
return Single.concat(localSource, remoteSource)
}
The problem with the above flow is, it calls onNext twice for each stream source to the downstream(presenter) even though both the data are same.
I can do the data diff logic in the presenter and update accordingly but I want the DataManager class to handle the logic for me(CleanArchitecture, SOC).
My Questions?
What's the best possible way to implement the above logic?
Am I leaking the inner subscriptions in DataManager (see: doOnSuccess code) ?. I'm disposing of the outer subscription when the presenter is destroyed.

fun getWatchlist(userId: UUID): Observable<List<Repository>?> {
val remoteSource: Single<List<Repository>> =
remoteStore.getWatchlist(userId)
.map(Watchlist::items)
.subscribeOn(scheduler.io)
return localStore.getUserWatchlist()
.flatMapObservable { listFromLocal: List<Repository> ->
remoteSource
.observeOn(scheduler.computation)
.toObservable()
.filter { apiWatchList: List<Repository> ->
apiWatchList != listFromLocal
}
.flatMapSingle { apiWatchList ->
localSource.saveUserWatchlist(apiWatchList)
.andThen(Single.just(apiWatchList))
}
.startWith(listFromLocal)
}
}
Explanation step by step:
Load data from localStore
Use flatMapObservable to subscribe to remoteSource each time the localStore emits data.
As there are more than one emission from inner observable(initial data from local and new data in case of updated data from the remoteSource) transform Single to Observable.
Compare data from remoteSource with data from the localStore and proceed data only in case if newData != localData.
For each emission after the filter initiate the localSource to save data and on a completion of this operation proceed saved data as Single.
As requested, at the beginning of remote request data from localStore should be proceeded and it is simply done be adding startWith at the end of the operators chain.

RxJava Operator for switching method

Im new with Rxjava on Android Project, here my code
class RadioListRepositoryImpl(private val apiServices: ApiServices, private val chlDao: ChannelDao) : RadioListRepository {
private val results: MutableList<DataResponse>
init {
results = ArrayList<DataResponse>()
}
override fun getData(): Observable<DataResponse> {
return dataFromMemory().switchIfEmpty(dataFromNetwork())
}
override fun dataFromMemory(): Observable<DataResponse> {
val cacheDateExp = DateTime().minusHours(6)
if(chlDao.isCacheExpired(cacheDateExp).isNotEmpty()){
Logger.d("Get data from cache SQLITE")
val chList: MutableList<DataResponse> = ArrayList()
val cache = chlDao.loadAll()
repeat(cache.size){ i ->
val ch = DataResponse()
ch.channelId = cache[i].channelId
ch.channelTitle = cache[i].title
chList.add(ch)
}
return Observable.from(chList)
}else{
chlDao.deleteAll()
return Observable.empty<DataResponse>()
}
}
override fun dataFromNetwork(): Observable<DataResponse> {
val dttime = DateTime()
return apiServices.getChannelList()
.concatMap {
dataListResponseModel -> Observable.from(dataListResponseModel.radio)
}
.doOnNext {
channelDataResponse -> results.add(channelDataResponse)
}
.doOnNext { channelDataResponse ->
Logger.d("Put data to cache")
val c: ChannelEntitiy = ChannelEntitiy()
c.channelId = channelDataResponse.channelId
c.title = channelDataResponse.channelTitle
chlDao.insert(c)
}
}
}
My other class access method getData() and I want if data is empty from memory (sqlite) then get data from network.
But what I want is if data is empty from memory then get data from network insert to memory and after that getData() method return dataFromMemory()
Can I handle it using another Rx operator to simplify my code ?

When you want to get data from multi-sources, concat and first should suit for you.
// Our sources (left as an exercise for the reader)
Observable<Data> memory = ...;
Observable<Data> disk = ...;
Observable<Data> network = ...;
// Retrieve the first source with data
Observable<Data> source = Observable
.concat(memory, disk, network)
.first();
concat() takes multiple Observables and concatenates their sequences. first() emits only the first item from a sequence. Therefore, if you use concat().first(), it retrieves the first item emitted by multiple sources.
The key to this pattern is that concat() only subscribes to each child Observable when it needs to. There's no unnecessary querying of slower sources if data is cached, since first() will stop the sequence early. In other words, if memory returns a result, then we won't bother going to disk or network. Conversely, if neither memory nor disk have data, it'll make a new network request.
Note that the order of the source Observables in concat() matters, since it's checking them one-by-one.
Then if you want to save data for each sources, just change a little bit it your source with doOnNext()
Observable<Data> networkWithSave = network.doOnNext(data -> {
saveToDisk(data);
cacheInMemory(data);
});
Observable<Data> diskWithCache = disk.doOnNext(data -> {
cacheInMemory(data);
});

Implementing search that pushes results to list as soon as they become available using rxJava

I need to implement a search on a large data set that can take some time to complete on mobile devices. So I want to display each matching result as soon as it becomes available.
I need to fetch all available data from a data store that decides whether to get them from network or from the device. This call is an Observable. As soon as the data from that Observable becomes available I want to loop over it, apply a search predicate and notify any Observers for any match found.
So far my idea was to use a PublishSubject to subscribe to and call its onNext function every time the search finds a new match. However I can't seem to get the desired behavior to work.
I'm using MVVM + Android Databinding and want to display every matched entry in a RecyclerView so for every onNext event that is received by the observing viewModel I have to call notifyItemRangeInserted on the RecyclerView's adapter.
class MySearch(val dataStore: MyDataStore) {
private val searchSubject = PublishSubject.create<List<MyDto>>()
fun findEntries(query: String): Observable<List<MyDto>> {
return searchSubject.doOnSubscribe {
// dataStore.fetchAll returns an Observable<List<MyDto>>
dataStore.fetchAll.doOnNext {
myDtos -> if (query.isNotBlank()) {
search(query, myDtos)
} else {
searchSubject.onNext(myDtos)
}
}.subscribe(searchSubject)
}
}
private fun(query: String, data: List<MyDto>) {
data.forEach {
if (it.matches(query)) {
// in real life I cache a few results and don't send each single item
searchSubject.onNext(listOf(it))
}
}
}
fun MyDto.matches(query: String): Boolean // stub
}
-
class MyViewModel(val mySearch: MySearch, val viewNotifications: Observer<Pair<Int, Int>>): BaseObservable() {
var displayItems: List<MyItemViewModel> = listOf()
fun loadData(query: String): Subscription {
return mySearch.findEntries(query)
.observeOn(AndroidSchedulers.mainThread())
.doOnNext(this::onSearchResult)
.doOnCompleted(viewNotifications::onCompleted)
.doOnError(viewNotifications::onError)
.subscribe()
}
private fun onSearchResult(List<MyDto> data) {
val lastIndex = displayItems.lastIndex
displayItems = data.map { createItem(it) }
notifyChange()
viewNotifications.onNext(Pair(lastIndex, data.count()))
}
private fun createItem(dto: MyDto): MyItemViewModel // stub
}
The problem I have with the above code is that with an empty query MyViewModel::onSearchResult is called 3 times in a row and when the query is not empty MyViewModel::onSearchResult isn't called at all.
I suspect the problem lies somewhere in the way I have nested the Observables in findEntries or that I'm subscribing wrong / getting data from a wrong thread.
Does anyone have an idea about this?