Loading data from Database + Network (Room + Retrofit + RxJava2) - android

I have a sample API request which returns a list of user's watchlist. I want to achieve the following flow when the user loads the watchlist screen:
Load the data from DB cache immediately.(cacheWatchList)
Initiate the RetroFit network call in the background.
i. onSuccess return apiWatchList
ii. onError return cacheWatchList
Diff cacheWatchList vs apiWatchList
i. Same -> all is well since data is already displayed to the user do nothing.
ii. Differs -> Save apiWatchList to a local store and send the apiWatchList to the downstream.
What I have done so far?
Watchlist.kt
data class Watchlist(
val items: List<Repository> = emptyList()
)
LocalStore.kt (Android room)
fun saveUserWatchlist(repositories: List<Repository>): Completable {
return Completable.fromCallable {
watchlistDao.saveAllUserWatchlist(*repositories.toTypedArray())
}
}
RemoteStore.kt (Retrofit api call)
fun getWatchlist(userId: UUID): Single<Watchlist?> {
return api.getWatchlist(userId)
}
DataManager.kt
fun getWatchlist(userId: UUID): Flowable<List<Repository>?> {
val localSource: Single<List<Repository>?> =
localStore.getUserWatchlist()
.subscribeOn(scheduler.computation)
val remoteSource: Single<List<Repository>> = remoteStore.getWatchlist(userId)
.map(Watchlist::items)
.doOnSuccess { items: List<Repository> ->
localStore.saveUserWatchlist(items)
.subscribeOn(scheduler.io)
.subscribe()
}
.onErrorResumeNext { throwable ->
if (throwable is IOException) {
return#onErrorResumeNext localStore.getUserWatchlist()
}
return#onErrorResumeNext Single.error(throwable)
}
.subscribeOn(scheduler.io)
return Single.concat(localSource, remoteSource)
}
The problem with the above flow is, it calls onNext twice for each stream source to the downstream(presenter) even though both the data are same.
I can do the data diff logic in the presenter and update accordingly but I want the DataManager class to handle the logic for me(CleanArchitecture, SOC).
My Questions?
What's the best possible way to implement the above logic?
Am I leaking the inner subscriptions in DataManager (see: doOnSuccess code) ?. I'm disposing of the outer subscription when the presenter is destroyed.

fun getWatchlist(userId: UUID): Observable<List<Repository>?> {
val remoteSource: Single<List<Repository>> =
remoteStore.getWatchlist(userId)
.map(Watchlist::items)
.subscribeOn(scheduler.io)
return localStore.getUserWatchlist()
.flatMapObservable { listFromLocal: List<Repository> ->
remoteSource
.observeOn(scheduler.computation)
.toObservable()
.filter { apiWatchList: List<Repository> ->
apiWatchList != listFromLocal
}
.flatMapSingle { apiWatchList ->
localSource.saveUserWatchlist(apiWatchList)
.andThen(Single.just(apiWatchList))
}
.startWith(listFromLocal)
}
}
Explanation step by step:
Load data from localStore
Use flatMapObservable to subscribe to remoteSource each time the localStore emits data.
As there are more than one emission from inner observable(initial data from local and new data in case of updated data from the remoteSource) transform Single to Observable.
Compare data from remoteSource with data from the localStore and proceed data only in case if newData != localData.
For each emission after the filter initiate the localSource to save data and on a completion of this operation proceed saved data as Single.
As requested, at the beginning of remote request data from localStore should be proceeded and it is simply done be adding startWith at the end of the operators chain.

Related

Kotlin rxjava switchIfEmpty

I'm trying to understand the logic of switchIfEmpty operator. I will be very thankful for every explanation.
I have a local database (Room) and remote server. My goal is to implement logic with switchIfEmpty to check if there is data in local DB to take it and if local DB is empty to call from remote. The process starts in activity where I subscribe to Observable:
private fun subscribeOnDataChanges() = with(viewModel) {
requestNextPage()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
filmsAdapter.addItems(it)
}, {
errorEvent.value = ERROR_MSG
it.printStackTrace()
}).addTo(autoDisposable)
}
Then the methods in the View Model:
fun requestNextPage(): Observable<List<Film>> {
return requestPageOfFilms()
}
private fun requestPageOfFilms(): Observable<List<Film>> =
interactor.requestPageOfFilmsFromDataSource()
And, finally the method with switchIfEmpty in the Interactor:
fun requestPageOfFilmsFromDataSource(): Observable<List<Film>> {
return repo.getPageOfFilmsInCategory(category).filter { it.isNotEmpty() }.switchIfEmpty(
getFromRemote(category)
)
}
private fun getFromRemote(category: String): Observable<List<Film>> {
return convertSingleApiToObservableDtoList(
retrofitService.getFilms(
category, API.KEY, "ru-RU", NEXT_PAGE
)
)
}
I cannot understand the next things:
Why, when local db (repo) is NOT empty, getFromRemote() is called?
If local db is empty, why network call in the method getFromRemote() is not performed? No matters, that I subscribed in the activity? Because if I add the subscription inside the switchIfEmpty(), the network call is performed.
To answer your questions :
Why, when local db (repo) is NOT empty, getFromRemote() is called?
Because this function convertSingleApiToObservableDtoList() is being evaluated at Assembly Time and not Subscription Time (https://github.com/ReactiveX/RxJava#assembly-time). The result is that the Observable object is eagerly evaluated.
If local db is empty, why network call in the method getFromRemote() is not performed? No matters, that I subscribed in the activity? Because if I add the subscription inside the switchIfEmpty(), the network call is performed.
Because the upstream has not emitted a value yet or completed. You are conflating an empty List with an empty Observable (https://reactivex.io/documentation/operators/defaultifempty.html)
A better solution to your problem would be using the flatMap or even switchMap operator if you expect a chatty upstream and only care about latest result, something like :
fun requestPageOfFilmsFromDataSource(): Observable<List<Film>> {
return repo.getPageOfFilmsInCategory(category)
.flatMap { films ->
if (films.isNotEmpty()) {
Observable.just(films)
} else getFromRemote(category)
}
}

How can I get the data from PagingData<T> object before it is reached to ViewHolder?

I want to get the data of a PagingData<T> object in some in-between class like ViewModel before it is reached to the Adapter and gather its property as a list.
I can access the code by the flatmap but I don't want to apply any changes. Also, this kind of access is useless because the entire block runs after subscribing to the observable is finished in ViewModel so it is unreachable by the ViewModel running.
private fun getAssignedDbList(): Flowable<PagingData<T>> {
var list = mutableListOf<Int>()
return repository.getList()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.flatMap { pagingData ->
Flowable.just(pagingData.map
{ foo ->
list.add(foo.id)
foo })
}
}
Another example, let's assume that we have this scenario:
I'm going to call another API before the Paging data is reached to the view holder. The paging data needs to be filled with a new API call response. How can I wait for the second API call to update the data of paging data?
private fun getListFromAPI(): Flowable<PagingData<T>> {
var list = mutableListOf<Int>()
return repository.getList()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.flatMap { pagingData ->
//call another API calls and waiting for a response to change some of the fields of
//paging data
}
}

android -MutableLiveData doesn't observe on new data

I'm using mvvm and android architecture component , i'm new in this architecture .
in my application , I get some data from web service and show them in recycleView , it works fine .
then I've a button for adding new data , when the user input the data , it goes into web service , then I have to get the data and update my adapter again.
this is my code in activity:
private fun getUserCats() {
vm.getCats().observe(this, Observer {
if(it!=null) {
rc_cats.visibility= View.VISIBLE
pb.visibility=View.GONE
catAdapter.reloadData(it)
}
})
}
this is view model :
class CategoryViewModel(private val model:CategoryModel): ViewModel() {
private lateinit var catsLiveData:MutableLiveData<MutableList<Cat>>
fun getCats():MutableLiveData<MutableList<Cat>>{
if(!::catsLiveData.isInitialized){
catsLiveData=model.getCats()
}
return catsLiveData;
}
fun addCat(catName:String){
model.addCat(catName)
}
}
and this is my model class:
class CategoryModel(
private val netManager: NetManager,
private val sharedPrefManager: SharedPrefManager) {
private lateinit var categoryDao: CategoryDao
private lateinit var dbConnection: DbConnection
private lateinit var lastUpdate: LastUpdate
fun getCats(): MutableLiveData<MutableList<Cat>> {
dbConnection = DbConnection.getInstance(MyApp.INSTANCE)!!
categoryDao = dbConnection.CategoryDao()
lastUpdate = LastUpdate(MyApp.INSTANCE)
if (netManager.isConnected!!) {
return getCatsOnline();
} else {
return getCatsOffline();
}
}
fun addCat(catName: String) {
val Category = ApiConnection.client.create(Category::class.java)
Category.newCategory(catName, sharedPrefManager.getUid())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ success ->
getCatsOnline()
}, { error ->
Log.v("this", "ErrorNewCat " + error.localizedMessage)
}
)
}
private fun getCatsOnline(): MutableLiveData<MutableList<Cat>> {
Log.v("this", "online ");
var list: MutableLiveData<MutableList<Cat>> = MutableLiveData()
list = getCatsOffline()
val getCats = ApiConnection.client.create(Category::class.java)
getCats.getCats(sharedPrefManager.getUid(), lastUpdate.getLastCatDate())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ success ->
list += success.cats
lastUpdate.setLastCatDate()
Observable.just(DbConnection)
.subscribeOn(Schedulers.io())
.subscribe({ db ->
categoryDao.insert(success.cats)
})
}, { error ->
Log.v("this", "ErrorGetCats " + error.localizedMessage);
}
)
return list;
}
I call getCat from activity and it goes into model and send it to my web service , after it was successful I call getCatsOnline method to get the data again from webservice .
as I debugged , it gets the data but it doesn't notify my activity , I mean the observer is not triggered in my activity .
how can I fix this ? what is wrong with my code?
You have made several different mistakes of varying importance in LiveData and RxJava usage, as well as MVVM design itself.
LiveData and RxJava
Note that LiveData and RxJava are streams. They are not one time use, so you need to observe the same LiveData object, and more importantly that same LiveData object needs to get updated.
If you look at getCatsOnline() method, every time the method gets called it's creating a whole new LiveData instance. That instance is different from the previous LiveData object, so whatever that is listening to the previous LiveData object won't get notified to the new change.
And few additional tips:
In getCatsOnline() you are subscribing to an Observable inside of another subscriber. That is common mistake from beginners who treat RxJava as a call back. It is not a call back, and you need to chain these calls.
Do not subscribe in Model layer, because it breaks the stream and you cannot tell when to unsubscribe.
It does not make sense to ever use AndroidSchedulers.mainThread(). There is no need to switch to main thread in Model layer especially since LiveData observers only run on main thread.
Do not expose MutableLiveData to other layer. Just return as LiveData.
One last thing I want to point out is that you are using RxJava and LiveData together. Since you are new to both, I recommend you to stick with just one of them. If you must need to use both, use LiveDataReactiveStreams to bridge these two correctly.
Design
How to fix all this? I am guessing that what you are trying to do is to:
(1) view needs category -> (2) get categories from the server -> (3) create/update an observable list object with the new cats, and independently keep the result in DB -> (4) list instance should notify activity automatically.
It is difficult to pull this off correctly because you have this list instance that you have to manually create and update. You also need to worry about where and how long to keep this list instance.
A better design would be:
(1) view needs category -> (2) get a LiveData from DB and observe -> (3) get new categories from the server and update DB with the server response -> (4) view is notified automatically because it's been observing DB!
This is much easier to implement because it has this one way dependency: View -> DB -> Server
Example CategoryModel:
class CategoryModel(
private val netManager: NetManager,
private val sharedPrefManager: SharedPrefManager) {
private val categoryDao: CategoryDao
private val dbConnection: DbConnection
private var lastUpdate: LastUpdate // Maybe store this value in more persistent place..
fun getInstance(netManager: NetManager, sharedPrefManager: SharedPrefManager) {
// ... singleton
}
fun getCats(): Observable<List<Cat>> {
return getCatsOffline();
}
// Notice this method returns just Completable. Any new data should be observed through `getCats()` method.
fun refreshCats(): Completable {
val getCats = ApiConnection.client.create(Category::class.java)
// getCats method may return a Single
return getCats.getCats(sharedPrefManager.getUid(), lastUpdate.getLastCatDate())
.flatMap { success -> categoryDao.insert(success.cats) } // insert to db
.doOnSuccess { lastUpdate.setLastCatDate() }
.ignoreElement()
.subscribeOn(Schedulers.io())
}
fun addCat(catName: String): Completable {
val Category = ApiConnection.client.create(Category::class.java)
// newCategory may return a Single
return Category.newCategory(catName, sharedPrefManager.getUid())
.ignoreElement()
.andThen(refreshCats())
.subscribeOn(Schedulers.io())
)
}
}
I recommend you to read through Guide to App Architecture and one of these livedata-mvvm example app from Google.

RxJava2 doOnNext returns later than intended

So I'm pretty sure that I'm kind of at a loss here.
The expected behavior is:
Get data from API -> save it in the local DB -> load data from the
local DB and display it
First of all in my Fragment I call a function that does this:
mLiveData = viewModel.fetchAllCategories(getString(R.string.lang_code))
mCategoryLiveData!!.observe(this, Observer<Array<Category>> { it ->
if (it != null) {
this#CategoryListFragment.allCategories = SparseArray(it.size)
for (category in it) {
allCategories[category.id] = category
}
displayedCategory = allCategories[1]
this#CategoryListFragment.mLoadingCircle.visibility = View.GONE
this#CategoryListFragment.displayCategoryChildren()
}
})
fetchAllCategories calls a function in the ViewModel which calls this function:
fun getAllCategoriesFromAPI(language: String): Flowable<Array<Category>> {
return service.getAllCategories(language)
.doOnNext {
Log.e("Repository", "Fetched ${it.size} Categories from the API ")
storeCategoriesInDb(it)
}
}
However the function displayCategoryChildren() fires before the onNext finishes which results in an error since the data the app is supposed to get from the db is not saved there yet.
If it is in any way relevant I can also post the fuction in the ViewModel

How to make request calls to remote service one by one usnig rx and kotlin?

I have remote service to which the app have to send data:
Definition in retrofit2:
interface FooRemoteService {
#POST("/foos")
fun postFoos(#Body foos: List<FooPojo>): Observable<Response<List<String>>
}
but the call has a limits no more than X Foos at once.
Each call can returns 206 code "partially successful" with list of unsuccessful uploaded foos. Also 413 "Request Entity Too Large". And of course 400 and 500 as well.
And the app needs to send unknown count of foo items (defined by user in runtime).
To avoid DDoS of service app is required to send this calls one by one.
So I made such implementation in my FooRepositoryImpl:
This is an idea. I'm not happy with below solution and I'm sure that it can be done much better but I'm run out of ideas. So any proposes?
override fun postFoos(foos: List<Foo>) Completable {
val fooChunks = divideListInToChuncksUnderRequestLimit(foos)
val unuploadedFoos = mutableListOf<UnuploadedFoo>()
fooChunks.fold(unuploadedFoos)
{ accu: MutableList<UnuploadedFoo>, chunk ->
fooRemoteService
.postFoos(chunk)
.subscribeOn(Schedulers.io())
.flatMapCompletable {
if (it.isSuccessful) {
Completable.complete()
} else {
Timber.e("$it")
accu.add(it.body())
}
}.blockingAwait()
responses
}
return Completable.complete()
}
At the end the app should display list of all unsuccessful foos or if any available. So I need pass from that fuction list of unuploaded Foos.
If you are OK with modifying the return type of postFoos a bit, something like this could work:
override fun postFoos(foos: List<Foo>): Observable<List<UnuploadedFoo>> {
val chunks = foos.chunked(CHUNK_SIZE)
val posters = chunks.map { chunk ->
fooRemoteService.postFoos(chunk)
.map { response ->
response.unUploaded.takeIf { !response.isSuccessful } ?: emptyList()
}
.filter { it.isNotEmpty() }
.toObservable()
}
return Observable.concatDelayError(posters)
}
I'm imagining your service to have something like:
data class Response(val isSuccessful: Boolean, val unUploaded: List<UnoploadedFoo>)
fun postFoos(foos: List<Foo>): Single<Response>
The trick here is that Concat:
(...) waits to subscribe to each additional Observable that you pass to it until the previous Observable completes.

Categories

Resources