I want to asynchronously retrieve data via multiple REST APIs. I'm using Retrofit on Android with the rxJava extension, i.e. I execute any GET request by subscribing to an Observable.
As I said, I have multiple source APIs, so when the first source does not yield the desired result I want to try the next on, if that also fails, again try the next and so forth, until all sources have been queried or a result was found.
I'm struggling to translate this approach into proper use of Observables since I don't know which operators can achieve this behaviour and there are also some constraints to honor:
when a result has been found, the remaining APIs, if any, should not be queried
other components depend on the result of the query, I want them to get an Observable when starting the request, so this Observable can notify them of the completion of the request
I need to keep a reference to aforementioned Observable because the same request could possibly be made more than once before it has finished, in that case I only start it the first time it is wanted and subsequent requests only get the Observable which notifies when the request finished
I was starting out with only one API to query and used the following for the request and subsequent notification of dependent components:
private Observable<String> loadData(int jobId) {
final ConnectableObservable<String> result = Async
.fromCallable(() -> getResult(jobId))
.publish();
getRestRequest()
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
dataHolder -> {
if (dataHolder.getData() != null && !dataHolder.getData().isEmpty()) {
saveData(dataHolder.getData());
} else {
markNotFound(dataHolder);
}
},
error -> currentJobs.remove(jobId),
() -> {
currentJobs.remove(jobId);
result.connect();
});
return result;
}
This code was only called for the first request, the returned Observable result would then be saved in currentJobs and subsequent requests would only get the Observable without triggering the request again.
Any help is highly appreciated.
Assuming you have a set of observables that re-connect each time you subscribe:
List<Observable<Result>> suppliers = ...
Then you just need to do the logical thing:
Observable<Result> results = Observable
.from(suppliers)
.concatMap(supplier -> supplier)
.takeFirst(result -> isAcceptable(result))
.cache()
Use .onErrorResumeNext, and assuming that each service observable may return 0 or 1 elements use first to emit an error if no elements are emitted:
Observable<T> a, b, c;
...
a.first().onErrorResumeNext(t -> b.first())
.onErrorResumeNext(t -> c.first())
.onErrorResumeNext(t -> d.first())
...
Related
I'm doing an API call to get the descriptions of a program podcast and based on the type of podcast, I may or may not have to do another API call to get more information. I'm new to RxJava and I'm not sure the best way to make such calls. Here's what I have so far:
public void fetchFeaturedItems() {
Timber.i("Fetching Featured...");
disposables.add(episodeService.getFeatured()
.subscribeOn(Schedulers.io())
.doOnNext(featured -> { //make second call
final Episode episode = featured.getEpisode();
Observable<Timing> timingObservable = episodeService.getTimingForEpisodeActs(episode);
if (timingObservable != null) {
timingObservable
.subscribeOn(Schedulers.io())
.doOnError(Timber::e)
.subscribe(timing -> {episodeManager.saveTiming(timing);}); //save to database
}
})
.observeOn(Schedulers.io())
.subscribe(featured -> {
saveFeatured(featured);
final Episode episode = featured.getEpisode();
notificationManager.handleNewEpisodeNotification(episode);
}, Timber::e));
}
This all works, but I'm getting a "result of subscribe is not used" lint warning on the second subscribe. I'm not combining results of the two calls. I could really use some guidance.
Use flatMap() instead of onNext(). You get warning about "result of subscribe is not used" cause of second subscribtion. flatMap() should help.
read this first and other RxJava documentation
.doOnNext is a side-effect operator. What you're doing:
timingObservable
.subscribeOn(Schedulers.io())
.doOnError(Timber::e)
.subscribe(timing -> {episodeManager.saveTiming(timing);});
Will just create a disposable. This disposable won't be a part of the stream. Also note that the timingObservable stream will now run totally independently, because as I just said, doOnNext is a side-effect operator. What you're doing is a fire-and-forget call. To make the response as a part of the stream, use a .flatMap in place of .doOnNext. It will merge your responses as they come, and push them to the downstream operators.
Ideally, a reactive stream should be subscribed to only once - you are doing it twice. This is an immediate code smell.
Basically I need to create a List of Observables without initial values. I subscribe to that list of Observables and will provide required results based on the responses from all Observables. I use zip operator.
The problem is that I need to create Observables initially, add them to the list and use zip operator. Only later I do network request with Retrofit and I need to update the value of observable in the list so my whole zip operator will be working.
However, I did not find a way to force update an observable in the list with the response from Retrofit. It seems very easy but I did not found any solutions.. only idea is to use tons of subjects and add them to the list instead...
List<Observable<Object>> listObservables = new ArrayList<>();
//Adding initial request
Observable<Object> testObservable = RetrofitFactory.create().startProcess();
listObservables.add(testObservable);
Observable.concatDelayError(listObservables).subscribe(response ->
{
//This is where all results should be managed
Log.d("response", String.valueOf(response));
},
error ->
{
Log.d("response", String.valueOf(error));
});
//Actual request occurs much later in application
listObservables.get(0).subscribeOn(Schedulers.io()).
observeOn(AndroidSchedulers.mainThread()).subscribe(response ->
{
// the response of this, should notify concatDelayError
Log.d("respoonse", String.valueOf(response));
});
If I understand correctly, you want to implement sub requests model. For this task you can break chain of operators to different execution flows and combine it back, for example, with zip operator. With this approach you can create completely independent data flow with sole trigger.
Subject<Event> eventSubject = PublishSubject.create();
Observable<TriggerObject> mainRequest = eventSubject.flatMap((event) ->
RetrofitFactory.create().startProcess());
Observable<FirstSubResult> firstSubRequest = mainRequest.flatMap(tigger -> {
// make first sub request
});
Observable<SecondSubResult> secondSubRequest = mainRequest.flatMap(tigger -> {
// make second sub request
});
Observable<CompleteResult> resultObservable = Observable.zip(firstSubRequest, secondSubRequest,
// zip function
(first, second) -> {
// combine result of sub requests to complete result object
});
Now you can start request flow by your event:
// post your event. On button clicked for evxample
eventSubject.doOnNext(yourEvent);
NOTE: this answer show main idea of chaining data flow sequences. This applicable to to other types of requests, and you can use this approach without retrofit
I recently learned to use merge where I combined 4 API requests into one output. (Reference: https://stackoverflow.com/q/51262421/1083093)
Restapi.class
/************/
#GET("app/dashboard")
Observable<CategoryHomeModel[]> getCategories(#HeaderMap Map<String, String> headers);
#GET("app/wallet/balance")
Observable<WalletBalance> getWalletBalance(#HeaderMap Map<String, String> headers);
#GET("/app/swap/myrateswaps")
Observable<SwapSettings> rateMySwap(#HeaderMap Map<String, String> headers);
#GET("/app/getsettings")
Observable<Settings> getSettings(#HeaderMap Map<String, String> headers);
/************/
I have Four observables
Observable<CategoryHomeModel[]> categoriesObservable = retrofit
.create(Restapi.class)
.getCategories(prepareHeaders())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Observable<WalletBalance> walletObservable = retrofit
.create(Restapi.class)
.getWalletBalance(prepareHeaders())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Observable<Settings> settingsObservable = retrofit
.create(Restapi.class)
.getSettings(prepareHeaders())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Observable<SwapSettings> ratingsObservable = retrofit
.create(Restapi.class)
.rateMySwap(prepareHeaders())
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
Answer I found was useful using Merge function
Extending the Question above:
I have a similar question: How can I wait for first Observable to
complete moving to second to third to fourth. Since I am using a
variable obtained from first observable to pass it into second and so
on till fourth observable
CONCAT - I searched is a good way to achieve this i think.
How to use CONCAT as i have used MERGE above ?
What you need is FLATMAP operator. It maps items emitted by the observable into new observables. It's convenient to use for Retrofit observables, where only one result item may be emitted by the observable. So we can transform observable item (request result) into observable representing next request in the sequence.
Referring to original question observable samples the code could look like this (assuming lamda syntax as in Java 8/RetroLambda):
categoriesObservable.flatMap(categoryHomeModel -> {
/* Work on categoryHomeModel result; you can use it to configure next request */
return walletObservable.flatMap(walletBalance -> {
/* Work on walletBalance result; you can use it to configure next request */
return settingsObservable.flatMap(settings -> {
/* Work on settings result; you can use it to configure next request */
return ratingsObservable;
});
});
}).subscribe(swapSettings -> { /* Work on swapSettings; sequence is done */ });
This will make sure your requests are performed in sequence. Even if you use CONCAT, the results will be emitted to subscriber in sequence, but the actual requests may start in parallel, which sometimes is not expected. CONCAT/ZIP are good options if you have truly independent requests (e.g. querying different kinds of data), and it's ok (and even desirable) to run requests in parallel, when order of actual HTTP requests does not matter.
My overall workflow for the Rx calls should work as follows (regardless of the current Rx code):
Get a list of motion sensor readings from a Room Dao (with the purpose of uploading them to a REST API). I'm using a Single<List<Reading>> for this
If that readings list is empty, then perform a jobFinished() callback and execute nothing after this
If readings is not empty, then chain a network call to this Single. The network call returns a Completable
The Single never logically throws an error, since it either fetches an empty or a non-empty readings list
When the entire Rx call chain is terminated, perform the jobFinished() callback
On the success of the entire Rx call chain, delete those readings from the Dao
On success of the Single, but error of the Completable, update the readings in the Dao
My current code is as follows:
Single.create<List<Reading>> {
readings = readingDao.getNextUploadBatch()
if (readings.isEmpty()) {
jobFinished(job, false)
return#create
}
it.onSuccess(readings)
}
.flatMapCompletable { api.uploadSensorReadings(it) }
.doOnTerminate {
jobFinished(job, !readingDao.isEmpty())
}
.subscribeOn(rxSchedulers.network)
.observeOn(rxSchedulers.database)
.subscribe(
{
readingDao.delete(*readings.toTypedArray())
},
{
markCurrentReadingsAsNotUploading()
}
)
The logical problem with the above code is (haven't tested it in runtime, but it compiles) that:
I want to cut off the code starting from the flatMapCompletable if readings list is empty
I do not want doOnTerminate to execute if readings is empty
I do not want the onComplete part (the first {} block) of subscribe to execute unless readings was non-empty, and the Completable returned a success as well
I do not want the onError part (the second {} block) of subscribe to execute unless readings was non-empty, and the Completable failed
I'm not sure how to implement my workflow as an efficient and neat Rx call chain. Any suggestions would be dearly welcome!
If you want to perform something different depending on a value, think of flatMap:
Single.fromCallable(() -> readingDao.getNextUploadBatch())
.subscribeOn(rxSchedulers.network)
.flatMapCompletable(readings -> {
if (readings.isEmpty()) {
jobFinished(job, false);
return Completable.complete();
}
return api.uploadSensorReadings(readings)
.doFinally(() -> jobFinished(job, !readingDao.isEmpty()))
.observeOn(rxSchedulers.database)
.doOnComplete(() -> readingDao.delete(readings.toTypedArray()))
})
.subscribe(() -> /* ignored */, error -> markCurrentReadingsAsNotUploading());
In my Android app I am using domain level Repository interface, which is backed with local DB implemented using SqlBrite and network api with Retrofit observables. So I have method getDomains(): Observable<List<Domain>> in Repository and two corresponding methods in my Retrofit and SqlBrite.
I don't want to concatenate or merge, or amb these two observables. I want my Repository to take data only from SqlBrite and since SqlBrite returns QueryObservable, which triggers onNext() every time underlying data changed, I can run my network request independently and store results to SqlBrite and have my Observable updated with fetched from network and stored to DB data.
So I tried to implement my Repository's getDomains() method as follow:
fun getDomains(): Observable<List<Domain>> {
return db.getDomains()
.doOnSubscribe {
networkClient.getDomains()
.doOnNext { db.putDomains(it) }
.onErrorReturn{ emptyList() }
.subscribe()
}
}
But in this case every time the client should subscribe, every time it would make network requests, that is not so good. I thought about other do... operators to move requests there, but doOnCompleted() in case of QueryObservable would never be called, until I call toBlocking() somewhere, which I won't, doOnEach() also not good as it makes requests every time item from db extracted.
I also tried to use replay() operator, but though the Observable cached in this case, the subscription happens and results in network requests.
So, how can combine these two Observables in the desired way?
Ok, it depends on the concrete use case you have: i.e. assuming you want to display the latest data from your local database and from time to time update the database by doing a network request in the background.
Maybe there is a better way, but maybe you could do something like this
fun <T> createDataAwareObservable(databaseQuery: Observable<T>): Observable<T> =
stateDeterminer.getState().flatMap {
when (it) {
State.UP_TO_DATE -> databaseQuery // Nothing to do, data is up to date so observable can be returned directly
State.NO_DATA ->
networkClient.getDomains() // no data so first do the network call
.flatMap { db.save(it) } // save network call result in database
.flatMap { databaseQuery } // continue with original observable
State.SYNC_IN_BACKGROUND -> {
// Execute sync in background
networkClient.getDomains()
.flatMap { db.save(it) }
.observeOn(backgroundSyncScheduler)
.subscribeOn(backgroundSyncScheduler)
.subscribe({}, { Timber.e(it, "Error when starting background sync") }, {})
// Continue with original observable in parallel, network call will then update database and thanks to sqlbrite databaseQuery will be update automatically
databaseQuery
}
}
}
So at the end you create your SQLBrite Observable (QueryObservable) and pass it into the createDataAwareObservable() function. Than it will ensure that it loads the data from network if no data is here, otherwise it will check if the data should be updated in background (will save it into database, which then will update the SQLBrite QueryObservable automatically) or if the data is up to date.
Basically you can use it like this:
createDataAwareObservable( db.getAllDomains() ).subscribe(...)
So for you as user of this createDataAwareObservable() you always get the same type Observable<T> back as you pass in as parameter. So essentially it seems that you were always subscribing to db.getAllDomains() ...
if your problem is that you have to subscribe your observer every time that you want to get data you can use relay, which never unsubscribe the observers because does not implement onComplete
/**
* Relay is just an observable which subscribe an observer, but it wont unsubscribe once emit the items. So the pipeline keep open
* It should return 1,2,3,4,5 for first observer and just 3, 4, 5 fot the second observer since default relay emit last emitted item,
* and all the next items passed to the pipeline.
*/
#Test
public void testRelay() throws InterruptedException {
BehaviorRelay<String> relay = BehaviorRelay.create("default");
relay.subscribe(result -> System.out.println("Observer1:" + result));
relay.call("1");
relay.call("2");
relay.call("3");
relay.subscribe(result -> System.out.println("Observer2:" + result));
relay.call("4");
relay.call("5");
}
Another examples here https://github.com/politrons/reactive/blob/master/src/test/java/rx/relay/Relay.java