Some background: I'm new to RxJava and I'm trying to make a feature in the app that will work offline and sync when there is network. I'm trying to chain multiple operations but I'm not well versed in how to chain different types like Completable, Maybe and Observable together.
Here are the list of operations that need to be done in the order after user adds or updates some data:
Update data on local db, just set the status to syncing, using Room here.
Upload the file to Firebase storage.
Get the file url and update the data to Firebase Database.
Update data on local db, set the status to synced.
Here are the methods for each operation:
Update local db:
private Completable setStatus(Entity entity, Entity.Status status){
entity.setStatus(status);
return Completable.fromAction(() -> localDataStore.updatePersonalPlace(personalPlaceEntity));
}
Upload file to FirebaseStorage, using Rx2Firebase
RxFirebaseStorage.putBytes(storageRef, bytes); // returns a Maybe<TaskSnapshot>
Set data in firebase database
RxFirebaseDatabase.setValue(dataRef, model); // returns a Completable
I've tried
setStatus(...)
.toObservable()
.map(o -> uploadFile())
.map(fileUrl -> updateFirebaseDatabase(fileUrl))
.doOnNext(() -> setStatus(..) ) // set status to synced
.subscribe(() -> Timber.d("Data updated",
t -> setStatus(..)); // set status back to what it was on error
But this doesn't work and I think I don't really understand the fundamentals of how to chain these operations. None of the operations after toObservable get called.
I've also tried to convert the maybe to a completable and chain them using Completable.andThen but I'm not sure how to do that correctly and I need the fileUrl returned to update the firebase database.
Could someone please point me in the right direction as to what should I use here. It's a fairly simple task which feels a lot complicated right now, maybe my approach is horribly wrong.
Thanks,
I add some comments to your code:
setStatus(...) // completable => (onError|onComplete)?
.toObservable() // => will call (onError|onComplete)? (beacause of the nature of completable)
.map(o -> uploadFile()) // never call because no item is emitted (completable...)
.map(fileUrl -> updateFirebaseDatabase(fileUrl)) // never call too
.doOnNext(() -> setStatus(..) ) // set status to synced // never call too
.subscribe(..)
You have to change your Completable for a Single and returning something like true.
Set status returns a Completable, which will only ever call onComplete or onError. Your map and doOnNext never get called because it never emits any items. What you probably want use doOnComplete or look into using concatArray, startWith or concatWith that can chain Completables.
Thanks to answers from Kevinrob and cwbowron I was able to figure out what was going wrong.
setStatus now returns a Single:
private Single<Integer> setStatus(Entity entity, Entity.Status status){
entity.setStatus(status);
return Single.fromCallable(() -> localDataStore.updatePersonalPlace(personalPlaceEntity));
}
This returns a completable which:
Sets the entity status as syncing in local db.
Converts a bitmap to byte array.
Uploads the photo on Firebase Storage.
Gets the photo url.
Updates the data on Firebase Database.
Finally updates the entity status as synced in local db.
return setPlaceStatusSingle(entity, Entity.Status.SYNCING)
.subscribeOn(Schedulers.io())
.toObservable()
.map(integer -> BitmapUtils.convertBitmapToByteArray(entity.getPhoto()))
.doOnNext(bytes -> Timber.d("Converted bitmap to bytes"))
.flatMap(bytes -> RxFirebaseStorage.putBytes(fileRef, bytes).toObservable())
// Using flatmap to pass on the observable downstream, using map was a mistake which created a Single<Observable<T>>
.observeOn(Schedulers.io())
.doOnNext(taskSnapshot -> Timber.d("Uploaded to storage"))
.map(taskSnapshot -> taskSnapshot.getDownloadUrl().toString()) // Firebase stuff, getting the photo Url
.flatMapCompletable(photoUrl -> {
Timber.d("Photo url %s", photoUrl);
model.setPhotoUrl(photoUrl);
return RxFirebaseDatabase.setValue(ref, model);
})
// Passes the Completable returned from setValue downstream
.observeOn(Schedulers.io())
.doOnComplete(() -> {
entity.setStatus(Entity.Status.SYNCED);
entity.setPhotoUrl(model.getPhotoUrl());
localDataStore.updateEntity(entity);
})
.doOnError(throwable -> onErrorUpdatingEntity(entity, throwable));
Related
I have an Android app using Room to save my favorites.
Here is my DAO :
#Query("SELECT * FROM favorites ORDER BY lastConsultation DESC")
fun getAll() : Flowable<List<Favorite>>
I want to use Flowable to enable my MainActivity to be notified every time a favorite is added or removed.
On my MainActivity, I want to retrieve all my favorites and make a network request to check some information about my favorite.
In my UseCase, I have the following piece of code to make my call
favoritesRepository.getAll()
.flatMap { Flowable.just(it) }
.concatMapEager { Flowable.fromIterable(it) }
.concatMapEager {
itemRepository.getItem(it.id)
.toFlowable()
}
.toList()
.toFlowable()
The itemRepository returns a Single when getItem is called. I retrieve a Flowable<List<Favorite>> from my favoritesRepository and want to turn each Favorite in an Item after making a network request, my method returning a Flowable<List<Item>>
I thought that adding .flatMap { Flowable.just(it) } would create a new Flowable that would emit onComplete once the item has been emitted (since Room will not emit onComplete).
That piece of code is not working, the onComplete is never called so the .toList() is not called either.
Is there a way I could achieve those calls (with concurrency hence the concatMapEager) while keeping my Flowable implementation (I could use a Single to be rid of the problem but I would lose the "auto notification" on the MainActivity) ?
I'll make an assumption that realtyRepository returns a Single. Try this.
favoritesRepository
.getAll()
.switchMapSingle { Flowable
.fromIterable(it)
.concatMapEager { realtyRepository.getRealty(it.id).toFlowable() }
.toList()
}
This will get what I understand you want, yet you'll still get notified when DB changes. Also, switchMapSingle will make sure that whatever work you have ongoing, will get cancelled if any DB updates occurs in the meantime.
Here is my use case:
I am developing an app that communicates with a server via a REST API and stores the received data in a SQLite database (it's using it as a cache of some sorts).
When the user opens a screen, the following has to occur:
The data is loaded from the DB, if available.
The app call the API to refresh the data.
The result of the API call is persisted to the DB.
The data is reloaded from the DB when the data change notification is intercepted.
This is very similar to the case presented here, but there is a slight difference.
Since I am using SQLBrite, the DB observables don't terminate (because there is a ContentObserver registered there, that pushes new data down the stream), so methods like concat, merge, etc. won't work.
Currently, I have resolved this using the following approach:
Observable.create(subscriber -> {
dbObservable.subscribe(subscriber);
apiObservable
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(
(data) -> {
try {
persistData(data);
} catch (Throwable t) {
Exceptions.throwOrReport(t, subscriber);
}
},
(throwable) -> {
Exceptions.throwOrReport(throwable, subscriber);
})
})
It seems like it's working OK, but it just doesn't seem elegant and "correct".
Can you suggest or point me to a resource that explains what's the best way to handle this situation?
The solution to your problem is actually super easy and clean if you change the way of thinking a bit. I am using the exact same data interaction (Retrofit + Sqlbrite) and this solution works perfectly.
What you have to do is to use two separate observable subscriptions, that take care of completely different processes.
Database -> View: This one is used to attach your View (Activity, Fragment or whatever displays your data) to the persisted data in db. You subscribe to it ONCE for created View.
dbObservable
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(data -> {
displayData(data);
}, throwable -> {
handleError(throwable);
});
API -> Database: The other one to fetch the data from api and persist it in the db. You subscribe to it every time you want to refresh your data in the database.
apiObservable
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(data -> {
storeDataInDatabase(data);
}, throwable -> {
handleError(throwable);
});
EDIT:
You don't want to "transform" both observables into one, purely for the reason you've included in your question. Both observables act completely differently.
The observable from Retrofit acts like a Single. It does what it needs to do, and finishes (with onCompleted).
The observable from Sqlbrite is a typical Observable, it will emit something every time a specific table changes. Theoretically it should finish in the future.
Ofc you can work around that difference, but it would lead you far, far away from having a clean and easily readable code.
If you really, really need to expose a single observable, you can just hide the fact that you're actually subscribing to the observable from retrofit when subscribing to your database.
Wrap the Api subscription in a method:
public void fetchRemoteData() {
apiObservable
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(data -> {
persistData(data);
}, throwable -> {
handleError(throwable);
});
}
fetchRemoteData on subscription
dbObservable
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.doOnSubscribe(() -> fetchRemoteData())
.subscribe(data -> {
displayData(data);
}, throwable -> {
handleError(throwable);
});
I suggest you really think about all that. Because the fact that you're forcing yourself into the position where you need a single observable, might be restricting you quite badly. I believe that this will be the exact thing that will force you to change your concept in the future, instead of protecting you from the change itself.
I want to asynchronously retrieve data via multiple REST APIs. I'm using Retrofit on Android with the rxJava extension, i.e. I execute any GET request by subscribing to an Observable.
As I said, I have multiple source APIs, so when the first source does not yield the desired result I want to try the next on, if that also fails, again try the next and so forth, until all sources have been queried or a result was found.
I'm struggling to translate this approach into proper use of Observables since I don't know which operators can achieve this behaviour and there are also some constraints to honor:
when a result has been found, the remaining APIs, if any, should not be queried
other components depend on the result of the query, I want them to get an Observable when starting the request, so this Observable can notify them of the completion of the request
I need to keep a reference to aforementioned Observable because the same request could possibly be made more than once before it has finished, in that case I only start it the first time it is wanted and subsequent requests only get the Observable which notifies when the request finished
I was starting out with only one API to query and used the following for the request and subsequent notification of dependent components:
private Observable<String> loadData(int jobId) {
final ConnectableObservable<String> result = Async
.fromCallable(() -> getResult(jobId))
.publish();
getRestRequest()
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
dataHolder -> {
if (dataHolder.getData() != null && !dataHolder.getData().isEmpty()) {
saveData(dataHolder.getData());
} else {
markNotFound(dataHolder);
}
},
error -> currentJobs.remove(jobId),
() -> {
currentJobs.remove(jobId);
result.connect();
});
return result;
}
This code was only called for the first request, the returned Observable result would then be saved in currentJobs and subsequent requests would only get the Observable without triggering the request again.
Any help is highly appreciated.
Assuming you have a set of observables that re-connect each time you subscribe:
List<Observable<Result>> suppliers = ...
Then you just need to do the logical thing:
Observable<Result> results = Observable
.from(suppliers)
.concatMap(supplier -> supplier)
.takeFirst(result -> isAcceptable(result))
.cache()
Use .onErrorResumeNext, and assuming that each service observable may return 0 or 1 elements use first to emit an error if no elements are emitted:
Observable<T> a, b, c;
...
a.first().onErrorResumeNext(t -> b.first())
.onErrorResumeNext(t -> c.first())
.onErrorResumeNext(t -> d.first())
...
In my Android app I am using domain level Repository interface, which is backed with local DB implemented using SqlBrite and network api with Retrofit observables. So I have method getDomains(): Observable<List<Domain>> in Repository and two corresponding methods in my Retrofit and SqlBrite.
I don't want to concatenate or merge, or amb these two observables. I want my Repository to take data only from SqlBrite and since SqlBrite returns QueryObservable, which triggers onNext() every time underlying data changed, I can run my network request independently and store results to SqlBrite and have my Observable updated with fetched from network and stored to DB data.
So I tried to implement my Repository's getDomains() method as follow:
fun getDomains(): Observable<List<Domain>> {
return db.getDomains()
.doOnSubscribe {
networkClient.getDomains()
.doOnNext { db.putDomains(it) }
.onErrorReturn{ emptyList() }
.subscribe()
}
}
But in this case every time the client should subscribe, every time it would make network requests, that is not so good. I thought about other do... operators to move requests there, but doOnCompleted() in case of QueryObservable would never be called, until I call toBlocking() somewhere, which I won't, doOnEach() also not good as it makes requests every time item from db extracted.
I also tried to use replay() operator, but though the Observable cached in this case, the subscription happens and results in network requests.
So, how can combine these two Observables in the desired way?
Ok, it depends on the concrete use case you have: i.e. assuming you want to display the latest data from your local database and from time to time update the database by doing a network request in the background.
Maybe there is a better way, but maybe you could do something like this
fun <T> createDataAwareObservable(databaseQuery: Observable<T>): Observable<T> =
stateDeterminer.getState().flatMap {
when (it) {
State.UP_TO_DATE -> databaseQuery // Nothing to do, data is up to date so observable can be returned directly
State.NO_DATA ->
networkClient.getDomains() // no data so first do the network call
.flatMap { db.save(it) } // save network call result in database
.flatMap { databaseQuery } // continue with original observable
State.SYNC_IN_BACKGROUND -> {
// Execute sync in background
networkClient.getDomains()
.flatMap { db.save(it) }
.observeOn(backgroundSyncScheduler)
.subscribeOn(backgroundSyncScheduler)
.subscribe({}, { Timber.e(it, "Error when starting background sync") }, {})
// Continue with original observable in parallel, network call will then update database and thanks to sqlbrite databaseQuery will be update automatically
databaseQuery
}
}
}
So at the end you create your SQLBrite Observable (QueryObservable) and pass it into the createDataAwareObservable() function. Than it will ensure that it loads the data from network if no data is here, otherwise it will check if the data should be updated in background (will save it into database, which then will update the SQLBrite QueryObservable automatically) or if the data is up to date.
Basically you can use it like this:
createDataAwareObservable( db.getAllDomains() ).subscribe(...)
So for you as user of this createDataAwareObservable() you always get the same type Observable<T> back as you pass in as parameter. So essentially it seems that you were always subscribing to db.getAllDomains() ...
if your problem is that you have to subscribe your observer every time that you want to get data you can use relay, which never unsubscribe the observers because does not implement onComplete
/**
* Relay is just an observable which subscribe an observer, but it wont unsubscribe once emit the items. So the pipeline keep open
* It should return 1,2,3,4,5 for first observer and just 3, 4, 5 fot the second observer since default relay emit last emitted item,
* and all the next items passed to the pipeline.
*/
#Test
public void testRelay() throws InterruptedException {
BehaviorRelay<String> relay = BehaviorRelay.create("default");
relay.subscribe(result -> System.out.println("Observer1:" + result));
relay.call("1");
relay.call("2");
relay.call("3");
relay.subscribe(result -> System.out.println("Observer2:" + result));
relay.call("4");
relay.call("5");
}
Another examples here https://github.com/politrons/reactive/blob/master/src/test/java/rx/relay/Relay.java
I am seeking an example of a flow I'm trying to implement with help of RxJava.
Suppose I want to show a list of data. The flow should look something like this:
Read cache. If it contains the data, show it;
Send an API request to the server:
If it returned the data, then cache it and show it.
If it returned and error and there was no cached data, then show an error.
If it returned and error and there was something cached, then do nothing.
Right now I have a method that does something similar (with lots of inspiration from Jake's u2020). The main difference is that it uses in-memory caching, which means there's no need for a separate Observable for reading from cache and it can be done synchronously.
I don't know how to combine two observables (one for reading from cache and the other for API call) and obtain the flow described above.
Any suggestions?
I think I solved my problem. The observable chain looks like so:
apiCall()
.map(data -> dataInMemory = data)
.onErrorResumeNext(t -> data == null ?
Observable.just(Data.empty()) : Observable.empty())
.startWith(readDataFromCache().map(data -> dataInMemory = data))
.subscribeOn(ioScheduler)
.observeOn(uiScheduler)
.subscribe(dataRequest);
The main point is, that if readDataFromCache() throws an error, it will call onCompleted() without calling onError(). So it should be a custom Observable which you can control.
Data.empty() is a stub for my data - the Subscriber should treat it as an error.
dataInMemory is a member in my controller which acts as in-memory cache.
EDIT: the solution doesn't work properly. The completion of one use case (see comment) is not achieved.
EDIT 2: well, the solution does work properly after some tweaking. The fix was returning different types of observables depending on the state of in-memory cache. Kind of dirty.
Here is my solution:
readDataFromCache().defaultIfEmpty(null)
.flatMap(new Func1<Data, Observable<Data>>() {
#Override
public Observable<Data> call(final Data data) {
if (data == null) {
// no cache, show the data from network or throw an error
return apiCall();
} else {
return Observable.concat(
Observable.just(data),
// something cached, show the data from network or do nothing.
apiCall().onErrorResumeNext(Observable.<Data>empty()));
}
}
});
I don't add the subscribeOn and observeOn because I'm not sure readDataFromCache() should use ioScheduler or uiScheduler.