TL;DR Looking for recommendations on robust offline support using rx-kotlin.
I've followed a nice guide on offline-support in Android apps.
It works like this:
Load data from the Internet, go to step 3 if error
Store the data in the local database
Load the data from the local database
Display the data in the UI
The code is:
Observable.mergeDelayError(
loadRemoteData()
.doOnNext { writeDataToLocalDatabase(it) }
.subscribeOn(Schedulers.io()),
loadDataFromLocalDatabase()
.subscribeOn(Schedulers.io())
)
Unfortunately, this approach relies on the database code always working. If the database operations for some reason fail, everything fails, even when the data is loaded successfully from the remote server.
Is there a way I can achieve the following using rx-kotlin/rx-java?:
Load data from the Internet, go to step 3 if error
Store the data in the local database
Load the data from the local database
(if steps 2 or 3 failed) Use the data from step 1
Display the data in the UI
I'd like to avoid loading the data from the Internet twice. I'm using room + retrofit, if that matters.
EDIT:
Thanks to #MinseongPark, I ended up with the code below.
The mergeDelayError reports an error only if both the remote and the local source fails, and if the writeDataToLocalDatabase method fails (throws an Exception), then that does not keep the remote data from being reported to the UI. The information about errors in writeDataToLocalDatabase is saved by reporting it remotely.
Now, the code is robust to one of the two sources failing, and to writing new entries to the database failing.
return Observable.mergeDelayError(
loadRemoteData().doOnNext {
try {
writeDataToLocalDatabase(it)
} catch (error: Throwable) {
Timber.d(error)
Crashlytics.logException(error)
}
},
loadDataFromLocalDatabase()
)
.subscribeOn(Schedulers.io())
Try this.
Observable.mergeDelayError(
loadRemoteData()
.doOnNext { runCatching { writeDataToLocalDatabase(it) } }
.subscribeOn(Schedulers.io()),
loadDataFromLocalDatabase()
.onErrorResumeNext(Observable.empty())
.subscribeOn(Schedulers.io())
)
Related
Because database fetches usually happen asynchronously by default, a variable that holds the data from the firebase database fetch will be null when used right after the fetch. To solve this I have seen people use the ".await()" feature in Kotlin coroutines but this goes against the purpose of asynchronous database queries. People also call the succeeding code from within 'addOnSuccessListener{}' but this seems to go against the purpose of MVVM, since 'addOnSuccessListener{}' will be called in the model part of MVVM, and the succeeding code that uses the fetched data will be in the ViewModel. The answer I'm looking for is maybe a listener or observer that is activated when the variable (whose value is filled from the fetched data) is given a value.
Edit:
by "succeeding code" I mean what happens after the database fetch using the fetched data.
As #FrankvanPuffelen already mentioned in his comment, that's what the listener does. When the operation for reading the data completes the listener fires. That means you know if you got the data or the operation was rejected by the Firebase servers due to improper security rules.
To solve this I have seen people use the ".await()" feature in Kotlin coroutines but this goes against the purpose of asynchronous database queries.
It doesn't. Using ".await()" is indeed an asynchronous programming technique that can help us prevent our applications from blocking. When it comes to the MVVM architecture pattern, the operation for reading the data should be done in the repository class. Since reading the data is an asynchronous operation, we need to create a suspend function. Assuming that we want to read documents that exist in a collection called "products", the following function is needed:
suspend fun getProductsFirestore(): List<Product> {
var products = listOf<Product>()
try {
products = productsRef.get().await().documents.mapNotNull { snapShot ->
snapShot.toObject(Product::class.java)
}
} catch (e: Exception) {
Log.d("TAG", e.message!!)
}
return products
}
This method can be called from within the ViewModel class:
val productsLiveData = liveData(Dispatchers.IO) {
emit(repository.getProductsFromFirestore())
}
So it can be observed in activity/fragment class:
private fun getProducts() {
viewModel.producsLiveData.observe(this, {
print(it)
//Do what you need to do with the product list
})
}
I have even written an article in which I have explained four ways in which you can read the data from Cloud Firestore:
How to read data from Cloud Firestore using get()?
When I make a request to the network, if an error occurs, then I will return data from the cache and the error. But sometimes I don’t get data from the cache, but I get only an error. The first time I launch the application, I always get only an error. If I call the getDashboard method once or several times, then everything is fine.
Here is a piece of code.
.onErrorResumeNext(throwable -> {
return Observable.concat(Observable.just(fromCache), Observable.error(throwable));
});
Full code here
https://gist.github.com/githubgist123/7e027675bb4db07fef606e23f39f8a96
Sorry, but the way you're doing your caching is wrong in my opinion. If you request twice consecutively and the cache is empty you'll end up with two network requests running and racing each other to write to the cache. You don't want that.
What you need is:
Observable.concat(
cache(),
network().share().doOnNext(setCache(...)).onErrorReturn(...)
.first()
)
You might think of synchronized your cache too or make it thread-safe by using the proper data structure.
For the first time I want to retrieve data from server cache it and next times show data on UI from local storage and request from server and update local storage and UI as
I have tried
(getCachedData()).concatWith(getRemoteData())
getCachedData returns Single
return apiSeResource.getData()
.doAfterSuccess { response ->
saveData(response.body())
}
}
.onErrorReturn {
return#onErrorReturn emptyList()
}
}```
The problem with `concat` is that the subsequent observable doesn't even start until the first Observable completes. That can be a problem. We want all observables to start simultaneously but produce the results in a way we expect.
I can use `concatEager` : It starts both observables but buffers the result from the latter one until the former Observable finishes.
Sometimes though, I just want to start showing the results immediately.
I don't necessarily want to "wait" on any Observable. In these situations, we could use the `merge` operator.
However the problem with merge is: if for some strange reason an item is emitted by the cache or slower observable after the newer/fresher observable, it will overwrite the newer content.
So none of mentioned above solution is not proper ,what is your solution?
Create 2 data sources one local data source and one remote and use the flatMap for running the Obervables. You can publish the data from the cache and when u get data from remote save data to cache and publish.
Or you can also try Observable.merge(dataRequestOne, dataRequestTwo) . run both the Observables on different threads
I noticed this behavior:
Disconnect a device from the internet
Create a few new documents while still offline
Close the app Then
Go online and reopen App
Documents are automatically synced with firestore (I lose control over completeListener)
Problem is that actions I had in onCompleteListener are not run.
Like in this snippet (do some super important stuff) is not run in the described scenario, also OnFailureListener is not called when a user is offline, so I cannot tell if it went ok or not.
FirebaseFirestore.getInstance().document(...).collection(...)
.set(message).addOnCompleteListener {
//do some super important stuff
}
I would rather do sync in this case on my own so I want it to fail and not repeat again.
How can I disable automatic sync of firestore?, for this one case only
So I wanted to ask this question and somehow Firestore transactions got to me.
So to fix this problem I used transactions (firestore / realtime dbs)
Transactions will fail when the client is offline.
So how it works now.
I try to run the transaction.
A If it fails I store it into a database
B If it succeeds I try to remove it from the database
And on every app start, I run sync service (check unsynced dbs and insert missing)
val OBJECT = ...
val ref = FirebaseFirestore.getInstance().document(...).collection(...)
FirebaseFirestore.getInstance().runTransaction(
object : Transaction.Function<Boolean> {
override fun apply(transaction: Transaction): Boolean? {
transaction.set(ref, OBJECT)
transaction.update(ref, PROPERTY, VALUE)
return true
}
})
.addOnSuccessListener {
println("Inserting $OBJECT ended with success==$it")
//todo remove from dbs (unsynced messages)
}.addOnFailureListener {
it.printStackTrace()
//todo save to dbs (unsynced messages)
}
//They work similarly in realtime database
I am fairly new to rxJava, trying stuff by my own. I would like to get some advice if I'm doing it right.
Usecase: On the first run of my app, after a successful login I have to download and save in a local database several dictionaries for the app to run with. The user has to wait till the downloading process finishes.
Current solution: I am using retrofit 2 with rxjava adapter in order to get the data. I am bundling all Observables into one using the zip operator. After all downloads are done the callback triggers and saving into database begins.
Nothing speaks better than some code:
Observable<List<OrderType>> orderTypesObservable = backendService.getOrderTypes();
Observable<List<OrderStatus>> orderStatusObservable = mockBackendService.getOrderStatuses();
Observable<List<Priority>> prioritiesObservable = backendService.getPriorities();
return Observable.zip(orderTypesObservable,
orderStatusObservable,
prioritiesObservable,
(orderTypes, orderStatuses, priorities) -> {
orderTypeDao.deleteAll();
orderTypeDao.insertInTx(orderTypes);
orderStatusDao.deleteAll();
orderStatusDao.insertInTx(orderStatuses);
priorityDao.deleteAll();
priorityDao.insertInTx(priorities);
return null;
});
Questions:
Should I use the zip operator or is there a better one to fit my cause?
It seems a bit messy doing it this way. This is only a part of the code, I have currently 12 dictionaries to load. Is there a way to refactor it?
I would like to insert a single dictionary data as soon as it finishes downloading and have a retry mechanism it the download fails. How can I achieve that?
I think in your case it's better to use Completable, because for you matter only tasks completion.
Completable getAndStoreOrderTypes = backendService.getOrderTypes()
.doOnNext(types -> *store to db*)
.toCompletable();
Completable getAndStoreOrderStatuses = backendService.getOrderStatuses()
.doOnNext(statuses -> *store to db*)
.toCompletable();
Completable getAndStoreOrderPriorities = backendService.getOrderPriorities()
.doOnNext(priorities -> *store to db*)
.toCompletable();
return Completable.merge(getAndStoreOrderTypes,
getAndStoreOrderStatuses,
getAndStoreOrderPriorities);
If you need serial execution - use Completable.concat() instead of merge()
a retry mechanism if the download fails
Use handy retry() operator
It is not good, to throw null value object into Rx Stream (in zip your return null, it is bad).
Try to not doing that.
In your case, you have 1 api call and 2 actions to save response into the database, so you can create the chain with flatMap.
It will look like:
backendService.getOrderTypes()
.doOnNext(savingToDatabaseLogic)
.flatMap(data -> mockBackendService.getOrderStatuses())
.doOnNext(...)
.flatMap(data -> backendService.getPriorities())
.doOnNext(...)
if you want to react on error situation, in particular, observable, you can add onErrorResumeNext(exception->Observable.empty()) and chain will continue even if something happened
Also, you can create something like BaseDao, which can save any Dao objects.