I have multiple modules that return Observables: O1, O2, O3... On
The result of all modules should be combined into one observable Ocomb so that individual tasks can fail but the combination is not terminated or influenced by individual issues.
With my current solution I'm encountering various problems as in the following example:
This code combines the output of my modules:
public Observable<Data> getModuleData(){
List<Observable<Data>> tasks = new ArrayList<>();
for(MyModule module : modules){
tasks.add(module.getData());
}
return Observable
.mergeDelayError(Observable.from(tasks))
.onBackpressureBuffer(MAX_BUFFER)
.observeOn(AndroidSchedulers.mainThread());
}
Now, I want to display attribute X e.g. 'name' of all emitted data objects:
public List<String> getNames() {
return getModuleData()
.map(new Func1<Data, String>() {
#Override
public String call(Data data) {
return data.getName();
}
})
.timeout(600, TimeUnit.MILLISECONDS)
.toList()
.toBlocking()
.firstOrDefault(new ArrayList<String>());
}
The getNames() method should return a list and therefore block the execution.
Problem 1
It seems there is an issue in RxJava that if I call observeOn() and make it blocking it will not return no matter what timeout etc are saying.
Problem 2
If onObserve() is removed, the code will work but in a different place of the app I'm rendering the results of the non-blocking observable in the UI. Data will be displayed but afterwards my UI does crazy stuff. I have to touch my UI list component to refresh the screen every time data changes.
Problem 3
Some of the modules might create internal errors or will not call onCompleted(). I thought that a combination of mergeDelayError() and timeout() could handle these cases and call onCompleted() for unresponsive modules. However, if one of the modules does not call onCompleted() and the timeout() statement is removed the blocking call will never return.
Questions:
What is the best way to combine multiple observable so that individual observables can fail but it's handled as onCompleted() / ignored and does not affect the combined observable?
What is the best solution to make the combined observable blocking and handle the timeout without stopping the execution or ending up in a loop?
What is the best way to combine multiple observable so that individual observables can fail but it's handled as onCompleted() / ignored and does not affect the combined observable?
Observable.from(modules)
.flatMap(MyModule::getData)
.onErrorResumeNext(Observable.empty())
.timeout(600,TimeUnit.MILLISECONDS, Observable.empty())
.toList()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(System.out::println);
Above stream could be converted to blocking by adding toBlocking before subscribe(), but does not make much sense beyond testing
Related
I have some expensive operations that only need to be performed once (e.g. load/ download large files, load large ML models, or calculate optimized data structure based on some other data). I want to use this for every value the Observable/ Flowable generates:
The following code works, but it runs heavyProcessing() and heavyProcessing2() on the caller's thread. In my case, I can't choose what my callers thread (its the main thread because I am using WorkManager's RxWorker, which calls createWork from main). Therefore, start blocks the main thread. How do I get heavyProcessing to be performed in the background with RxJava and also available to the subsequent RxJava chain?
fun start(): Observable<Unit> {
val heavy = heavyProcessing() // the heavy value i want to use everywhere!
val anotherHeavyObject = heavyProcessing2()
val items = Observable.fromIterable(listOfHundredsOfItems)
.map { doSomeWork(it, heavy) }
.map { doSomeWork(it, anotherHeavyObject) }
}
My attempts has so far not worked:
Create a wrapper around the existing function: The issue with this code is the Observable returned by start() does not get observed, so the doSomeWork doesn't actually get done. I only know this because I put breakpoints in at doSomeWork, and it never gets called.
fun startInBackground(): Single<Unit> {
return Single.fromCallable {
start()
}
}
I've been trying to find ways of 'unnesting' the inner Observable (inside the Single), as that's probably the issue here. The inner Observable is not being observed.
This RxJava stuff is very unintuitive even after reading the guide
Yes, it was related to Deferred-dependent. The example in the docs state:
Sometimes, there is an implicit data dependency between the previous sequence and the new sequence that, for some reason, was not flowing through the "regular channels". One would be inclined to write such continuations as follows:
AtomicInteger count = new AtomicInteger();
Observable.range(1, 10)
.doOnNext(ignored -> count.incrementAndGet())
.ignoreElements()
.andThen(Single.defer(() -> Single.just(count.get())))
.subscribe(System.out::println);
Actually, all I needed the caller to do is:
Single.defer { start() }.map { doMoreWork() }
instead of
start().map { doMoreWork() }
How do I convert a future returning void into an RxJava Flowable or Observable?
I am using RxJava and AndroidX WorkManager which provides an API which returns a Future<void>. I understand RxJava does not handle null values and will NullPointerException immediately. I am using Flowable.fromFuture(resultFuture), where resultFuture is Future<Void> (specifically ListenableFuture<Void>), but since it returns Null, it crashes the app.
Motivation: I want to turn this future into a Rx observable/ flowable so that I can do processing after this future completes.
return Flowable.fromFuture(futureReturningVoid)
.flatMap { Flowable.fromIterable(files) }
...more processing here...
I need to return a Single at the end, so I cannot move the work into a listener, with Future.addListener.
I need flowable because I am dealing with processing of multiple input files, and want the backpressure to prevent opening too many files at once. I included Observable in case people have less complex requirements.
So why not :
return Completable.fromFuture(futureReturningVoid)
.andThen(Flowable.fromIterable(files))
Which is basically :
return Completable.fromAction { futureReturningVoid.get() }
.andThen(Flowable.fromIterable(files))
I have a number of Observables that are used for network requests in my app. Since so much is the same, I apply an Observable transformation to them:
/**
* Creates a transformer that applies the schedulers and error handling for all of the observables in this ViewModel.
*/
private fun applyTransformations(): Observable.Transformer<NetworkState, NetworkState> {
return Observable.Transformer { observable ->
observable
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.onErrorReturn { NetworkState.Error(it) }
.doOnNext { publishState(it) }
.startWith(NetworkState.Loading())
}
}
The goals I am trying to accomplish with the above:
Apply consistent schedulers
Handle any error by returning an instance of my sealed class.
Handle any onNext by publishing the state returned by the observable.
Start off by sending a Loading state.
This works mostly fine, but what I've noticed is that while I call startWith and a loading state, it is never actually handled by doOnNext(). In other words, publishState() is never called for my loading state.
Where I set up the observables, I don't bother to add a subscriber, because the doOnNext() above is all that I'll need:
val subscription = repository.getInstagramPhotos(count)
.map { mapIGPhotoResponse(it) }
.compose(applyTransformations())
.subscribe()
If I were to supply a subscriber above, though, it would handle the loading state. It would also handle two onNext() calls - one for the subscriber supplied, and one for the doOnNext in the transform.
Is there a way to modify this startWith call to emit to whatever I've specified in doOnNext? I'm using RxJava 1.
Edit: Just to clarify some more, if I track what's emitted I expect to see two things. Loading -> Success. What I actually see is just Success. If I supply a subscriber to the observable I see Loading -> Success -> Success.
startWith should be before doOnNext.
Rxjava methods, though they look like they use the builder pattern, actually don't. They return a new observable each time an operator is applied. In your case, your doOnNext observable completes before your start with observable, so it's consumer isn't called with what you supply in startWith.
Ideally, you should go with:
observable
.startWith(NetworkState.Loading())
.doOnNext { publishState(it) }
.onErrorReturn { NetworkState.Error(it) }
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
Also, be careful with subscribing with no Consumer for onEror should it happen. Since you have nothing to consume the onError, RxJava will crash your app since it has nothing to notify for the error. Consider replacing the doOnNext with a Success Consumer in subscribe, and an empty Consumer for the error if you want to ignore it.
Also doOnNext is typically used for side effects, such as logging and the sort, they're more of a convenience than true functional operators.
I have a case when I have multiple observables, each observable has its own implementation, they may be with the same type, or different I didn't decide know yet, but let's assume it they're the same type.
Observable<String> source1;
Observable<String> source2;
Observable<String> source3;
Observable<String> source4;
what I need to do now is to execute only one of them, so stream only move to the next observable if the previous one failed.
Some potential solutions:
the onErrorResumeNext() which it may be good if they're only two
observables, but in my case here, if I need to change the order of execution it will hard to update each observable.
there is the combineLatest but I don't know if it behaves the way I
described, or what modification to make work as I need.
how to achieve something like this and if they're with different types, what I need to do?
I don't know if there is better way to do it, but I would just use onErrorResumeNext() with the help of some methods for making it flexible:
Observable<String> buildObservable(Observable<String> obs, Observable<String>... subsequentObservables) {
Observable<String> observable = obs;
for (int i = 0; i < subsequentObservables.length; i++) {
observable = concatErrorObservable(observable, subsequentObservables[i]);
}
return observable;
}
where concatErrorObservable is:
Observable<String> concatErrorObservable(Observable<String> observable, Observable<String> observable2) {
return observable.onErrorResumeNext(observable2);
}
So you just need to provide the list of Observable to the buildObservable method. For example:
buildObservable(Observable.error(new Throwable("error!!")),
Observable.just("observable2"),
Observable.just("observable3"))
.subscribe(s -> Log.d(TAG, "result: " + s));
will print observable2 (in the logcat) because the first observable throws an error.
About the different types, you probably need a different map for each Observable, because I think your consumer (observer) will just expect one type of emitted data.
You can get a combined observable using onErrorResumeNext and reduce like this:
Observable<String> buildObservable(List<Observable<String>> observables) {
return Observable.fromIterable(observables)
.reduce(Observable::onErrorResumeNext)
.flatMapObservable(obs -> obs);
}
UPDATE:
To explain further, if you call the method with a list [o1, o2, o3], then
the fromIterable will return a higher-level observable equivalent to just(o1, o2, o3)
the reduce will combine the elements of this observable, sequentially calling onErrorResumeNext() with each element, like this:
o1 -> o1.onErrorResumeNext(o2) -> o1.onErrorResumeNext(o2).onErrorResumeNext(o3),
resulting in a still "higher level" 1-element observable that is equivalent to just(o1.onErrorResumeNext(o2).onErrorResumeNext(o3)).
the flatMapObservable() line will replace this 1-element observable with its one and only element itself, which is o1.onErrorResumeNext(o2).onErrorResumeNext(o3) (without the just()).
This result implements the fallback mechanism you need.
I'm new to RxJava, often got confused by flatMap function. According to the doc, flatmap transform the items emitted by an Observable into Observables, then flatten the emissions from those into a single Observable
Can someone give a good use case for it? Why transform the original Observable into Observables (plural) then turn them into a single Observable.
Why don't you just use 'map'?
If you give an example in Android that's awesome, otherwise plain Java is good enough. Thanks
I see tag Android on your question. So, probably you should be familiar with Retrofit.
Let's image that you have 2 methods:
public interface FoxreyRestApi {
#POST("/signin")
Observable<SignInResponse> signin(#Body SignInRequest request);
#GET("/user")
Observable<User> getUser(String accessToken);
}
You want to get user data, but you need accessToken, which return is SignInResponse.
You can do this:
1). Create your RestAdapter.
2). Do queries one - after - another:
restAdapter.signin(request)
.flatMap(r -> restAdapter.getUser(r.getAccessToken()))
.subscribe(user -> {/*User your user*/});
Let's say you have an
Observable<Foo> fooObservable;
And you want to call another method which takes a Foo and emits an Observable<Bar>
Something like:
public Observable<Bar> getBars(Foo foo);
If you did:
fooObservable.map(foo -> getBars(foo));
You'd end up with an Observable<Observable<Bar>> because you've transformed your Foo -> Observable<Bar> which is probably not what you want.
Instead you can use flatMap which "flattens the observable":
Observable<Bar> barObservable = fooObservable.flatMap(foo -> getBars(foo));
Very often I use it to transform some of the UI events to observable background tasks:
ViewObservable.clicks(calculateBtn)
.flatMap(new Func1<OnClickEvent, Observable<Integer>>() {
#Override
public Observable<Integer> call(OnClickEvent onClickEvent) {
return observeBackgroundOperation()
.observeOn(AndroidSchedulers.mainThread())//interaction with UI must be performed on main thread
.doOnError(new Action1<Throwable>() {//handle error before it will be suppressed
#Override
public void call(Throwable throwable) {
progress.setVisibility(View.GONE);
calculateBtn.setEnabled(true);
Toast.makeText(IOCombineSampleActivity.this, R.string.mix_error_message, Toast.LENGTH_SHORT).show();
}
})
.onErrorResumeNext(Observable.<Integer>empty());//prevent observable from breaking
}
})
.subscribe(new Action1<Integer>() {...});
Because it's easy to define background operations using observable, I used flatMap to transform button click events to 'something done in background events' (for example network request finished with Retrofit) and then observe them.
Note, that observable in flatMap can emit single value, which is done in sample.
This way I have declaratively defined interaction between UI and background processes.
I handle errors with doOnError and then use onErrorResumeNext(Observable.<Integer>empty()) to prevent observable from terminating with onError. Because I use flatMap, my observable is not completed (while inner flatMap was) and is waiting for next click events.
Full sample of code you can find in my article.