I want to achieve the following with RxJava and as I may not have enough knowledge in this area would like to have some help :)
I need to create a PublishSubject which would emit events with the following sequence:
Emit 1, 2, 3
Buffer 4 in subscribe's completion if a certain condition is not satisfied (may be a network connection for example or some other condition).
For 5, 6 ... buffer after 4 if the condition is not satisfied yet.
Repeat to emit 4 after some time when the condition is satisfied.
If trying to emit 5,6 and the condition is satisfied, then instead of buffering 5, 6 ... after 4, just emit 4 and then 5, 6, 7 , 8 ...
The last 2 points are necessary because the sequence of emitting is really important, which makes difficulties for me to achieve to this.
I hope I could describe what I want to achieve :)
Findings: After asking this question I've done some findings and achieved the following:
private Observable observable = publishSubject
.observeOn(Schedulers.io())
.map(Manager::callNew)
.doOnError(throwable -> Logger.e(throwable, "Error occurred"))
.retryWhen(throwableObservable -> throwableObservable
.zipWith(Observable.range(1, 10), (n, i) -> i)
.flatMap(retryCount -> {
long retrySeconds = (long) Math.pow(2, retryCount);
Logger.d("The call has been failed retrying in %s seconds. Retry count %s", retrySeconds, retryCount);
return Observable.timer(retrySeconds, TimeUnit.SECONDS)
.doOnNext(aLong -> {
C24Logger.d("Timer was completed. %s", aLong);
})
.doOnComplete(() -> Logger.d("Timer was completed."));
}));
The problem is here with PublishSubject. Because it already has emitted all the items, it emits only new ones for retryWhen. If I use ReplaySubject them it emits also the old completed items too for the new retryWhen re-subscribe, which I do not need anymore.
Is there a way to use the ReplaySubject to remove the completed items from the buffer?
You want to be able to turn buffering on and off, depending upon an external condition. Perhaps the simplest way to do it is use the buffer() operator to continually buffer items based on the condition.
(I have removed stuff from the observer chain)
private Observable observable = publishSubject
.publish( obs -> obs.buffer( obs.filter( v -> externalCondition( v ) ) ) )
.flatMapIterable( bufferedList -> bufferedList )
.subscribe( ... );
The publish() operator allows multiple observer chains to subscribe to the incoming observer chain. The buffer() operator monitors the observable that emits a value only when the external condition is true.
When the external condition is true, buffer() will emit a series of lists with only a single element. When the condition goes false, buffer() starts buffering up the results, and when the condition goes true again, all the buffered items are emitted as a list. The flatMapIterable() step will take each item out of the buffer and emit it separately.
Related
I'm trying to add a delay in between each batch write and I managed to get it working by modifying this example but I'm not sure this is the correct way to achieve this?
rxBleConnection.createNewLongWriteBuilder()
.setCharacteristicUuid(characteristic)
.setBytes(data)
.setWriteOperationAckStrategy(booleanObservable -> {
return Observable.zip(
Observable.timer(delayInMillis, MILLISECONDS).repeat()
,booleanObservable, (callback0, aBoolean) -> aBoolean);
})
.build()
Your approach would delay next (sub)writes if Observable.timer().repeat() would emit after the booleanObservable. Unfortunately that would work only for the second (sub)write as after that .repeat() would start to emit very quickly as it does not resubscribe to the upstream Observable. From .repeat() Javadoc:
Returns an Observable that repeats the sequence of items emitted by the source ObservableSource indefinitely.
If you would use Observable.timer(delayInMillis, MILLISECONDS).repeatWhen(completions -> completions) or Observable.interval(delayInMillis, MILLISECONDS) then these would make writes happen not more frequent than delayInMillis, MILLISECONDS apart.
If you would like to give the peripheral delayInMillis, MILLISECONDS time before issuing next write then there seems to be a simpler approach:
rxBleConnection.createNewLongWriteBuilder()
.setCharacteristicUuid(characteristic)
.setBytes(data)
.setWriteOperationAckStrategy(booleanObservable -> booleanObservable.delay(delayInMillis, MILLISECONDS))
.build()
I'm looking to implement a batching mechanism before an api post for some simple event collection and logging.
Since this is Android, I also would like to handle lifecycle events for if this service is stopped, so what is the way to manually flush the buffered window if the service is stopped but the count or time has not been hit yet.
For example, I have a PublishSubject (subject), create a flowable and the perform a window operation on it like so:
subject.toFlowable(BackpressureStrategy.BUFFER)
.window(30,
TimeUnit.SECONDS,
20,
true)
.flatMapSingle { it.toList() }
.subscribe (this::send)
If my service/app is paused or killed, I'd like to just send what is in the buffer.
The problem you face is to stop observing when necessary and flush current items in window. Documentation for Flowable.window() operator say this:
When the source Publisher completes or encounters an error, the resulting Publisher emits the current window and propagates the notification from the source Publisher.
So you need to make your Subject emit error or complete. In most of the cases, this is not a correct way how to work with subjects. Let's replace Subject with something what can be easily completed:
private val stopObserver = BehaviorSubject.create<Unit>() // (1)
private fun emitStop() { // (2)
stopObserver.onNext(Unit)
}
private fun sourceSubject(): Flowable<Long> { // (3)
return Flowable.interval(1, TimeUnit.SECONDS)
.takeUntil(stopObserver.toFlowable(BackpressureStrategy.BUFFER)) // (4)
}
private fun runObservation() { // (5)
sourceSubject()
.window(10)
.flatMapSingle { it.toList() }
.doOnNext { Log.d("onNext", "${it.count()} items") }
.subscribe()
}
Explanation of important parts:
Create new Subject which emits everytime you realize app to being stopped or paused.
You can simply emit onNext event to Subject when needed with function emitStop()
sourceSubject() function imitates your source Subject. This one emits item every second.
takeUntil() operator completes stream when passed Publisher (stopObserver) emits an item. This ensures, our overall source Publisher (sourceSubject) completes.
I have used simpler version of window() operator, but all of them use the same principle regarding to source publisher.
Possible output:
2019-11-30 10:48:54.527 D/onNext: 10 items
2019-11-30 10:49:04.524 D/onNext: 10 items
2019-11-30 10:49:14.525 D/onNext: 10 items
2019-11-30 10:49:19.056 D/onNext: 4 items
Is there any way to signal an Observable to produce more data?
I have a buffered Observable that produces 10 items and i want it to continue producing more items only when i want to.
To be more specific, i have an infinite scrolling Recycler View, i want to produce more items only when i continue scrolling to the end.
Assuming you have already some sort of a back-pressured data source.
You can use request(n) to let your data source know that you are ready for more items.
What you can do is have an Observer that emits page scrool events, let's call it paginator.
On each emission you can switch to another observer that brings more data,
paginator
.onBackpressureDrop()
.map { lastVisibleIndex -> (lastVisibleIndex + 2) / PAGE_SIZE + 1 }
.distinct()
.concatMap { page -> getCollectionsObservable(page, pageSize)
}
.subscribeWith(GetCollectionObserver())
)
}
What is the difference between
ObservableTransformer {
Observable.merge(
it.ofType(x).compose(transformerherex),
it.ofType(y).compose(transformerherey)
)
}
and
ObservableTransformer {
it.publish{ shared ->
Observable.merge(
shared.ofType(x).compose(transformerherex),
shared.ofType(y).compose(transformerherey)
)
}
}
when I run my code using this two, I got the same results. What does publish do here.
The difference is that the top transformer will subscribe to the upstream twice for a single subscription from the downstream, duplicating any side effects of the upstream which is usually not wanted:
Observable<Object> mixedSource = Observable.<Object>just("a", 1, "b", 2, "c", 3)
.doOnSubscribe(s -> System.out.println("Subscribed!"));
mixedSource.compose(f ->
Observable.merge(
f.ofType(Integer.class).compose(g -> g.map(v -> v + 1)),
f.ofType(String.class).compose(g -> g.map(v -> v.toUpperCase()))
)
)
.subscribe(System.out::println);
will print
Subscribed!
2
3
4
Subscribed!
A
B
C
The side-effect represented here is the printout Subscribed! Depending on the actual work in a real source, that could mean sending an email twice, retrieving the rows of a table twice. With this particular example, you can see that even if the source values are interleaved in their type, the output contains them separately.
In contrast, publish(Function) will establish one subscription to the source per one end subscriber, thus any side-effects at the source only happen once.
mixedSource.publish(f ->
Observable.merge(
f.ofType(Integer.class).compose(g -> g.map(v -> v + 1)),
f.ofType(String.class).compose(g -> g.map(v -> v.toUpperCase()))
)
)
.subscribe(System.out::println);
which prints
Subscribed!
A
2
B
3
C
4
because the source is subscribed once and each item is multicast to the two "arms" of the .ofType().compose().
publish operator converts your Observable to Connectable Observable.
Lets see what does Connectable Observable mean: Suppose you want to subscribe an observable multiple time and want to serve same items to each subscriber. You need to use Connectable Observable.
Example:
var period = TimeSpan.FromSeconds(1);
var observable = Observable.Interval(period).Publish();
observable.Connect();
observable.Subscribe(i => Console.WriteLine("first subscription : {0}", i));
Thread.Sleep(period);
observable.Subscribe(i => Console.WriteLine("second subscription : {0}", i));
output:
first subscription : 0
first subscription : 1
second subscription : 1
first subscription : 2
second subscription : 2
In this case, we are quick enough to subscribe before the first item is published, but only on the first subscription. The second subscription subscribes late and misses the first publication.
We could move the invocation of the Connect() method until after all subscriptions have been made. That way, even with the call to Thread.Sleep we will not really subscribe to the underlying until after both subscriptions are made. This would be done as follows:
var period = TimeSpan.FromSeconds(1);
var observable = Observable.Interval(period).Publish();
observable.Subscribe(i => Console.WriteLine("first subscription : {0}", i));
Thread.Sleep(period);
observable.Subscribe(i => Console.WriteLine("second subscription : {0}", i));
observable.Connect();
output:
first subscription : 0
second subscription : 0
first subscription : 1
second subscription : 1
first subscription : 2
second subscription : 2
So using Completable Observable, we have a way to control when to let Observable emit items.
Example taken from : http://www.introtorx.com/Content/v1.0.10621.0/14_HotAndColdObservables.html#PublishAndConnect
EDIT
According to 180th slide in this link:
Another nature of publish is that if any observer start observing after 10 seconds of observable started emitting items, observer gets only items those were emitted after 10 seconds(at the time of subscription) not all the items. So in sides, as i could understood that publish is being used for UI events. And it totally makes sense that any observer should only receive those events that has been performed after it has subscribed NOT all the events happened before.
Hope it helps.
I have to schedule same observable repeatedly for N number of times with M seconds delay between each observable:
O1____1sec____O2____1sec____O3____1sec____O4(completed)
Notice no delay between start and ending observable,
Observable<Precious> result = <~> // hidden for brevity (it's just a long time consuming observable that can take suppose up to 10 seconds or more)
Observable.timer(M,TimeUnit.SECONDS).compose(x -> result).take(N).subscribe();
Problem Here is result observable that is doing expensive network calls, will it timout itself after timer expires , or we have to tell it to do so , if so how?
You can use the combination of concatMap to concat the observables, and Delay to delay the emission of every one
/**
* Another elegant solution it would be to create an observable with the list of items, and then use
* concatMap to pass all items from the first observable to the second, then this second observable
* can be created used delay operator afterwards.
*/
#Test
public void delayObservablesWithConcatMap() {
Observable.from(Arrays.asList(Observable.just(1), Observable.just(2),Observable.just(3)))
.concatMap(s -> s.delay(100, TimeUnit.MILLISECONDS))
.subscribe(n -> System.out.println(n + " just emitted..."),
e -> {
},
() -> System.out.println("Everybody emitt!"));
new TestSubscriber().awaitTerminalEvent(1000, TimeUnit.MILLISECONDS);
}
You can see more examples of delay here https://github.com/politrons/reactive/blob/master/src/test/java/rx/observables/utils/ObservableDelay.java
To space out items from obsevable source so that they're emitted every N seconds, use the following pattern:
source.zipWith(Observable.interval(1, TimeUnit.SECONDS), (src, dummy) -> src)
Caveat here is that if your source observable takes more time than the interval, then the items get queued up.
Now that I've re-read your question and clarifications, I think what you need is this:
Observable.interval(1, TimeUnit.SECONDS)
.switchMap(dummy -> result)
This will unsubscribe and resubscribe to the result observable every 1 second. It will only workif your result observable cancels network calls on unsubscription.