Develop Reference

RxJava filtering with inside object

For start I must say I am begginer in RxJava.
Data class:
#Entity(tableName = "google_book")
data class GoogleBook (
#PrimaryKey(autoGenerate = true) val id: Int=0,
val items: ArrayList<VolumeInfo>)
data class VolumeInfo(val volumeInfo: BookInfo){
data class BookInfo(val title: String, val publisher: String, val description: String, val imageLinks: ImageLinks?)
data class ImageLinks(val smallThumbnail: String?)
}
Function which helps me save data to database:
fun searchBooks(query: String) {
searchJob?.cancel()
searchJob = viewModelScope.launch {
val text = query.trim()
if (text.isNotEmpty()) {
bookRepository.getBooksFromApi(query)
.map { t ->
t.items.map {
it.volumeInfo.imageLinks?.smallThumbnail?.filter { x -> x != null }
}
t
}
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe { x ->
x?.let { googleBook ->
searchJob?.cancel()
searchJob = viewModelScope.launch {
bookRepository.deleteGoogleBook()
bookRepository.insertGoogleBook(googleBook)
}
} ?: kotlin.run {
Log.d(TAG, "observeTasks: Error")
}
}
}
}
}
As seen I want to filter list within GoogleBook object by image parameter but It doesnt work. I cannot add filtering for data class ImageLinks so I have no Idea how can I make it right
I am asking mostly about this part:
.map { t ->
t.items.map {
it.volumeInfo.imageLinks?.smallThumbnail?.filter { x -> x != null }
}
t
}
Thanks for reading

welcome to RxJava, you gonna love it.
As far as I can tell the issue with your filtering simply relies here:
.map { t ->
t.items.map {
it.volumeInfo.imageLinks?.smallThumbnail?.filter { x -> x != null })
} // this returns you a new list filtered list here, but does not modify the original one
t // but you return the same data object here, it is not modified at all
}
// also consider naming it bookInfo if it is actually a bookInfo
What you should do is make a copy of your object with the filtered elements, something like this:
fun filterGoogleBookBySmallThumbNail(googleBook: GoogleBook): GoogleBook {
val filteredItems = googleBook.items.filter { it.volumeInfo.imageLinks?.smallThumbnail == null }
return googleBook.copy(items = ArrayList(filteredItems)) // now a new googleBook item is created with the filtered elements
}
// snippet to adjust then
bookRepository.getBooksFromApi(query)
.map { googleBook -> filterGoogleBookBySmallThumbNail(googleBook) }
//...
Some additional notes / suggestions I have:
I don't see you actually disposing of the subscription of the Observable.
bookRepository.getBooksFromApi(query) If this line returns an Observable, even if you cancel the job, you will be still observing that Observable. If it returns a Single then you are in luck, because after one element it is disposed.
To properly dispose, in cancellation you would have to do something like this(still i would recommend the other two rather, just wanted to note the not disposing):
searchJob = viewModelScope.launch {
val text = query.trim()
if (text.isNotEmpty()) {
val disposable = bookRepository.getBooksFromApi(query)
//...
.subscribe { x ->
//...
}
try {
awaitCancellation() // this actually suspends the coroutine until it is cancelled
} catch (cancellableException: CancellationException) {
disposable.dispose() // this disposes the observable subscription
// that way the coroutine stays alive as long as it's not cancelled, and at that point it actually cleans up the Rx Subscription
}
Seems wasteful that you start a new coroutine job just to do actions
If you want to go the Rx way, you could make the
bookRepository.deleteGoogleBook() and bookRepository.insertGoogleBook(googleBook) Completable, and setup the observable as:
bookRepository.getBooksFromApi(query)
//..
.flatMap {
bookRepository.deleteGoogleBook().andThen(bookRepository.insertGoogleBook(it)).andThen(Observable.just(it))
}
//..subscribeOn
.subscribe()
Seems weird you are mixing coroutine and RX this way
if you don't want to go full Rx, you may consider converting your Observable into a kotlin coroutine Flow, that would be easier to handle with coroutine cancellations and calling suspend functions.
I hope it's helpful

How to handle database call errors using Flows

Usually I'm returning from my dao suspend function:
#Dao
interface DataDao {
#Query("SELECT * FROM data")
fun getAllData(): List<Data>
}
And handle the call within the repository:
class DataRepository(
private val dataDao: DataDao
) {
fun getAllData(): Flow<DataState> = flow {
val cacheResult = safeDatabaseCall(dispatcher = Dispatchers.IO) { dataDao.getAllData() }
//handle cacheResult, convert to DataState, emit DataState values
}.flowOn(Dispatchers.IO)
}
With generic fun:
suspend fun <T> safeDatabaseCall(
dispatcher: CoroutineDispatcher,
cacheCall: suspend () -> T?
): CacheResult<T?> {
return withContext(dispatcher) {
try {
withTimeout(10000L) {
CacheResult.Success(cacheCall.invoke())
}
} catch (t: Throwable) {
when (t) {
is TimeoutCancellationException -> {
CacheResult.Error("Timeout error")
}
else -> {
CacheResult.Error("Unknown error")
}
}
}
}
}
The problem is that I want return fun getAllData(): Flow<List<Data>> instead of fun getAllData(): List<Data> In order to get immediate updates, But if I'm returning Flow from the Dao, I can't handle the call with safe call and catch errors.
I thought about collecting the data, but if i'm collecting the data the call already done without error handling
Basically I need the cache result return CacheResult<Data> and not CacheResult<Flow<Data>>
How can I solve the problem And make a generic safeDatabaseCall while returning Flow from Dao?

So if I understand correctly you just want to handle the query and return of information safely in a flow. My only question is around the types. I can sorta assume Data DataState and CacheResult are not the same types so I use a "magic" function that converts the intermediary values to the correct one. You will need to adjust accordingly
class DataRepository(
private val dataDao: DataDao
) {
fun getAllData(): Flow<DataState> = flow {
val result = safeDatabaseCall(dispatcher = Dispatchers.IO) {
dataDao.getAllData()
}
// Emit the result
emit(result)
}.catch { t : Throwable ->
// Do our transformation like before
val result = when (t) {
is TimeoutCancellationException -> {
CacheResult.Error("Timeout error")
}
else -> {
CacheResult.Error("Unknown error")
}
}
// And because catch is actually extending a FlowCollector
// We can emit the result in the stream
emit(result)
}.map { cacheResult ->
convertToDataOrDataState(cacheResult)
}
You shouldn't need flowOn with a dispatcher here since the work inside this flow doesn't require thread dispatching
to Dispatcher.IO. The code we are putting in our flow, is purely exception handling and invoking a function. The only place that seems to require any manual dispatch changing is, safeDatabaseCall(). I am not familiar with this function but if it does exist and takes a dispatcher for the result of actualing making the db calls on an IO thread, then all should be good without flowOn. Otherwise you will be switching dispatchers from original dispatcher -> IO and then to IO again. It's not much but the extra no-op context switch doesn't add anything other than confusion later on.
The flow itself traps any upstream issues and you then make them part of the resulting flow

Wait for Room #Insert query completion

I can't figure out how to do a "simple" operation with Room and MVVM pattern.
I’m fetching some data with Retrofit. A “proper” response triggers an observer in the activity and a small part of the response itself is inserted in the database using Room library, wiping all previous values stored and inserting the fresh ones. Otherwise old values are retained on DB.
Just after that, I would like to check for a field in the database, but I’m not able to force this operation to wait until the previous one is completed.
Models
#Entity(tableName = "licence")
data class Licence(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "licence_id")
var licenceId: Int = 0,
#Ignore
var config: List<LicenceConfig>? = null,
.......
//all the others attributes )
#Entity(foreignKeys = [
ForeignKey(
entity = Licence::class,
parentColumns = ["licence_id"],
childColumns = ["licence_reference"],
onDelete = ForeignKey.CASCADE
)],tableName = "licence_configurations")
data class LicenceConfig(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "licence_config_id")
var licenceConfigId: Int,
#ColumnInfo(name="licence_reference")
var licenceReference: Int,
Observer in the activity
loginViewModel.apiResponse.observe(this, Observer { response ->
response?.let {
loginViewModel.insertLicences(response.licence)
}
//here I need to wait for the insertion to end
loginViewModel.methodToCheckForTheFieldOnDatabase()
})
ViewModel
fun insertLicences(licences: List<Licence>) = viewModelScope.launch {
roomRepository.deleteAllLicences()
licences.forEach { licence ->
roomRepository.insertLicence(licence).also { insertedLicenceId ->
licence.config?.forEach { licenceConfiguration ->
roomRepository.insertLicenceConfiguration(
licenceConfiguration.apply { licenceReference = insertedLicenceId.toInt() }
)
}
}
}
}
Room Repository
class RoomRepository(private val roomDao: RoomDao) {
val allLicences: LiveData<List<Licence>> = roomDao.getAllLicences()
suspend fun insertLicence(licence: Licence): Long {
return roomDao.insertLicence(licence)
}
suspend fun insertLicenceConfiguration(licenceConfiguration: LicenceConfig){
return roomDao.insertLicenceConfiguration(LicenceConfig)
}
}
RoomDao
#Dao
interface RoomDao {
#Query("select * from licence")
fun getAllLicences(): LiveData<List<Licence>>
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insertLicence(licence: Licence): Long
#Insert
suspend fun insertLicenceConfiguration(licence: LicenceConfig)
#Query("DELETE FROM licence")
suspend fun deleteAllLicences()
}
Set an observer to the "allLicences" LiveData or directly on that field on DB is not an option because the operations will be performed just after the activity creation and I have to wait until the API response to perform them.
In another project, without Room, I have used async{} and .await() to perform sequential operations while working with coroutines but I can't really make it works here. When I pause the debugger just after the insertion method the value of "allLicences" it's always null but after resuming and exporting the DB the data are properly inserted. I also tried adding .invokeOnCompletion{} after the ViewModel method but with the same result.
Basically I would like to wait for this method to end to do another operation.
Any suggestions?
EDIT
I totally forgot to report the models! Each licence have a list of configurations. When I perform a licence insert I take the autogenerated id, I apply it to the licenceConfig and then I perform the insert for each licenceConfig object (the code in the nested forEach loop of the ViewModel method). The problem seems to be that performing this nested loop breaks the "synchronicity" of the operation

To wait until insertion is completed, you need to move the coroutine creation from insertLicences() to your observer and also make the insertLicences() a suspend function.
loginViewModel.apiResponse.observe(this, Observer { response ->
lifecycleScope.launch {
response?.let {
loginViewModel.insertLicences(response.licence)
}
//here I need to wait for the insertion to end
loginViewModel.methodToCheckForTheFieldOnDatabase()
}
})
and
suspend fun insertLicences(licences: List<Licence>) {
roomRepository.deleteAllLicences()
licences.forEach { licence ->
roomRepository.insertLicence(licence).also { insertedLicenceId ->
licence.config?.forEach { licenceConfiguration ->
roomRepository.insertLicenceConfiguration(
licenceConfiguration.apply { licenceReference = insertedLicenceId.toInt() }
)
}
}
}
}
Alternative Solution
You can shift all of the code present in the observer into ViewModel.
loginViewModel.apiResponse.observe(this, Observer { response ->
loginViewModel.refreshLicenses(response)
})
and in ViewModel
fun refreshLicenses(response:Response?){
viewModelScope.launch{
response?.let {
insertLicences(response.licence)
}
methodToCheckForTheFieldOnDatabase()
}
}
and also make insertLicences as suspend function
suspend fun insertLicences(licences: List<Licence>) {
roomRepository.deleteAllLicences()
licences.forEach { licence ->
roomRepository.insertLicence(licence).also { insertedLicenceId ->
licence.config?.forEach { licenceConfiguration ->
roomRepository.insertLicenceConfiguration(
licenceConfiguration.apply { licenceReference = insertedLicenceId.toInt() }
)
}
}
}
}

Edit: Didn't read your conclusion before I reply but, I still think that your answer lies in coroutines
Using callbacks or promises, won't your function be executed when the insert query is finished?
Callbacks
With callbacks, the idea is to pass one function as a parameter to
another function, and have this one invoked once the process has
completed.
fun postItem(item: Item) {
preparePostAsync { token ->
submitPostAsync(token, item) { post ->
processPost(post)
}
}
}
fun preparePostAsync(callback: (Token) -> Unit) {
// make request and return immediately
// arrange callback to be invoked later
}
I would prefer promises to be honest
Promises
The idea behind futures or promises (there are also other terms these
can be referred to depending on language/platform), is that when we
make a call, we're promised that at some point it will return with an
object called a Promise, which can then be operated on.
fun postItem(item: Item) {
preparePostAsync()
.thenCompose { token ->
submitPostAsync(token, item)
}
.thenAccept { post ->
processPost(post)
}
}
fun preparePostAsync(): Promise<Token> {
// makes request an returns a promise that is completed later
return promise
}
Do your work and when the promise is fullfilled, proceed to data validation.
You can read more about coroutines here

How to make request calls to remote service one by one usnig rx and kotlin?

I have remote service to which the app have to send data:
Definition in retrofit2:
interface FooRemoteService {
#POST("/foos")
fun postFoos(#Body foos: List<FooPojo>): Observable<Response<List<String>>
}
but the call has a limits no more than X Foos at once.
Each call can returns 206 code "partially successful" with list of unsuccessful uploaded foos. Also 413 "Request Entity Too Large". And of course 400 and 500 as well.
And the app needs to send unknown count of foo items (defined by user in runtime).
To avoid DDoS of service app is required to send this calls one by one.
So I made such implementation in my FooRepositoryImpl:
This is an idea. I'm not happy with below solution and I'm sure that it can be done much better but I'm run out of ideas. So any proposes?
override fun postFoos(foos: List<Foo>) Completable {
val fooChunks = divideListInToChuncksUnderRequestLimit(foos)
val unuploadedFoos = mutableListOf<UnuploadedFoo>()
fooChunks.fold(unuploadedFoos)
{ accu: MutableList<UnuploadedFoo>, chunk ->
fooRemoteService
.postFoos(chunk)
.subscribeOn(Schedulers.io())
.flatMapCompletable {
if (it.isSuccessful) {
Completable.complete()
} else {
Timber.e("$it")
accu.add(it.body())
}
}.blockingAwait()
responses
}
return Completable.complete()
}
At the end the app should display list of all unsuccessful foos or if any available. So I need pass from that fuction list of unuploaded Foos.

If you are OK with modifying the return type of postFoos a bit, something like this could work:
override fun postFoos(foos: List<Foo>): Observable<List<UnuploadedFoo>> {
val chunks = foos.chunked(CHUNK_SIZE)
val posters = chunks.map { chunk ->
fooRemoteService.postFoos(chunk)
.map { response ->
response.unUploaded.takeIf { !response.isSuccessful } ?: emptyList()
}
.filter { it.isNotEmpty() }
.toObservable()
}
return Observable.concatDelayError(posters)
}
I'm imagining your service to have something like:
data class Response(val isSuccessful: Boolean, val unUploaded: List<UnoploadedFoo>)
fun postFoos(foos: List<Foo>): Single<Response>
The trick here is that Concat:
(...) waits to subscribe to each additional Observable that you pass to it until the previous Observable completes.

Loading data from Database + Network (Room + Retrofit + RxJava2)

I have a sample API request which returns a list of user's watchlist. I want to achieve the following flow when the user loads the watchlist screen:
Load the data from DB cache immediately.(cacheWatchList)
Initiate the RetroFit network call in the background.
i. onSuccess return apiWatchList
ii. onError return cacheWatchList
Diff cacheWatchList vs apiWatchList
i. Same -> all is well since data is already displayed to the user do nothing.
ii. Differs -> Save apiWatchList to a local store and send the apiWatchList to the downstream.
What I have done so far?
Watchlist.kt
data class Watchlist(
val items: List<Repository> = emptyList()
)
LocalStore.kt (Android room)
fun saveUserWatchlist(repositories: List<Repository>): Completable {
return Completable.fromCallable {
watchlistDao.saveAllUserWatchlist(*repositories.toTypedArray())
}
}
RemoteStore.kt (Retrofit api call)
fun getWatchlist(userId: UUID): Single<Watchlist?> {
return api.getWatchlist(userId)
}
DataManager.kt
fun getWatchlist(userId: UUID): Flowable<List<Repository>?> {
val localSource: Single<List<Repository>?> =
localStore.getUserWatchlist()
.subscribeOn(scheduler.computation)
val remoteSource: Single<List<Repository>> = remoteStore.getWatchlist(userId)
.map(Watchlist::items)
.doOnSuccess { items: List<Repository> ->
localStore.saveUserWatchlist(items)
.subscribeOn(scheduler.io)
.subscribe()
}
.onErrorResumeNext { throwable ->
if (throwable is IOException) {
return#onErrorResumeNext localStore.getUserWatchlist()
}
return#onErrorResumeNext Single.error(throwable)
}
.subscribeOn(scheduler.io)
return Single.concat(localSource, remoteSource)
}
The problem with the above flow is, it calls onNext twice for each stream source to the downstream(presenter) even though both the data are same.
I can do the data diff logic in the presenter and update accordingly but I want the DataManager class to handle the logic for me(CleanArchitecture, SOC).
My Questions?
What's the best possible way to implement the above logic?
Am I leaking the inner subscriptions in DataManager (see: doOnSuccess code) ?. I'm disposing of the outer subscription when the presenter is destroyed.

fun getWatchlist(userId: UUID): Observable<List<Repository>?> {
val remoteSource: Single<List<Repository>> =
remoteStore.getWatchlist(userId)
.map(Watchlist::items)
.subscribeOn(scheduler.io)
return localStore.getUserWatchlist()
.flatMapObservable { listFromLocal: List<Repository> ->
remoteSource
.observeOn(scheduler.computation)
.toObservable()
.filter { apiWatchList: List<Repository> ->
apiWatchList != listFromLocal
}
.flatMapSingle { apiWatchList ->
localSource.saveUserWatchlist(apiWatchList)
.andThen(Single.just(apiWatchList))
}
.startWith(listFromLocal)
}
}
Explanation step by step:
Load data from localStore
Use flatMapObservable to subscribe to remoteSource each time the localStore emits data.
As there are more than one emission from inner observable(initial data from local and new data in case of updated data from the remoteSource) transform Single to Observable.
Compare data from remoteSource with data from the localStore and proceed data only in case if newData != localData.
For each emission after the filter initiate the localSource to save data and on a completion of this operation proceed saved data as Single.
As requested, at the beginning of remote request data from localStore should be proceeded and it is simply done be adding startWith at the end of the operators chain.