Is there any way to signal an Observable to produce more data?
I have a buffered Observable that produces 10 items and i want it to continue producing more items only when i want to.
To be more specific, i have an infinite scrolling Recycler View, i want to produce more items only when i continue scrolling to the end.
Assuming you have already some sort of a back-pressured data source.
You can use request(n) to let your data source know that you are ready for more items.
What you can do is have an Observer that emits page scrool events, let's call it paginator.
On each emission you can switch to another observer that brings more data,
paginator
.onBackpressureDrop()
.map { lastVisibleIndex -> (lastVisibleIndex + 2) / PAGE_SIZE + 1 }
.distinct()
.concatMap { page -> getCollectionsObservable(page, pageSize)
}
.subscribeWith(GetCollectionObserver())
)
}
Related
Background:
I have an observable that returns mock items and actual items. Mock items are used in ui while actual items are still on the way. Mock items should always arrive first to prevent accidental override of actual items in ui. The premise is that ui layer is dumb, and it does not know which items it is showing nor it's managing them in any way. So the single source of items is this observable. Simplified it looks like this.
val one = Observable.fromCallable { "Mock/cached items" }
val two = Observable.fromCallable { "Live items" }
val result = Observable.concat(listOf(one, two))
.subscribe {
println(it)
}
Now I want to replay this observable, because further downstream it's just one observable of many that are combined together via combineLatest() operator, hence I can't resubscribe to it individually. So, I've added replayWhen(...) after concat() and everything worked properly.
Problem:
Things got tricky when I needed to replay only mock items in some cases vs replaying all items in other cases. Attempt that failed:
val subject1 = PublishSubject.create<Unit>()
val subject2 = PublishSubject.create<Unit>()
val one = Observable.fromCallable { "Mock/cached items" }.repeatWhen { subject1 }
val two = Observable.fromCallable { "Live items" }.repeatWhen { subject2 }
val result = Observable.concat(listOf(one, two))
.subscribe {
println(it)
}
subject1.onNext(Unit)
subject2.onNext(Unit)
Obviously, it prints only mock items while live items aren't called (anytime soon:-)). This is because concat waits for onComplete from first observable, that doesn't happen because of replay operator.
Solutions, and their drawbacks:
Replace concat with merge. This way both observable would work, but there's no guarantee that observable one would fire before the observable two. Is there any way to achieve ordering between observables without concat?
Have replayWhen() after concat, to replay this result observable as a whole, but have something inside .fromCallable{} of observable two to know when to skip it. Not sure what could it be, as a simple flag would be unreliable and dependant on external state that may change sporadically.
Thank you for any other ideas.
All the examples I have seen using the PagedList use it in conjunction with a RecyclerView. I have the scenario where I would like to use the Paging library but without any need for the UI. For example, I initially have a Recyclerview that does retrieve a list of data through the use of the PagedList that is bound to a PagedListAdapter. The user can then remove items on the list they don't want and save the list.
The problem is, that the list can be large and the user may not necessarily scroll through the entire list. When the user goes to save their choices, I need to run the query again a second time that generated the RecyclerView items, only this time, the items are handled either by my ViewModel or some other business object layer. The user could hit the back button and leave the screen where they initiated the saving. They could do this while the saving is not even finished. So the saving of this data must not be bound to the UI because once the activity or fragment is destroyed, the paging data will no longer be received.
All the examples I've seen use LiveData with an observer. Because the saving can take a long time depending on the number of items to be saved, retrieving all the records must be done in batches. According to the PagedList documentation, the load around method is supposed to load the next batch of records. I wasn't able to get this to work. So I have two problems. How do I avoid using LiveData to obtain the batched records and how do I get load around to load the next batch. Here is the code in my ViewModel:
fun getConnectionsFromDB(forSearchResults: Boolean): LiveData<PagedList<Connection>> {
return LivePagedListBuilder(connections.getConnectionsFromDB(forSearchResults), App.context.repository.LIST_PAGE_SIZE).build()
}
fun storeGroupConnections(groupId: String, connections: PagedList<Connection>) {
val groupConnections = mutableListOf<GroupConnection>()
connections.forEach { connection ->
if (connection != null)
groupConnections.add(GroupConnection(groupId = groupId, connectionId = connection.rowid))
}
val disposable = Observable.fromCallable { groups.storeGroupConnections(groupConnections) }
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ },
{ ex ->
App.context.displayErrorMessage(R.string.problem_storing_connections)
},
{
connections.loadAround(groupConnections.size - 1)
}
)
disposables.add(disposable)
}
And in my fragment:
fun storeGroupConnections(groupId: String) {
connectionsViewModel.getConnectionsFromDB(args.forSearchResults).observe(this, Observer { connections ->
groupsViewModel.storeGroupConnections(groupId, connections)
})
}
My observable looks like following:
obs
.doOnNext { logger("Items changed (${it.size})") }
.distinctUntilChanged()
.doOnNext { logger("Items changed (${it.size})- EMIITTED") }
Log looks like following:
Items changed (7)
Items changed (7)- EMIITTED
Items changed (8)
// => missing EMIITTED message although it.size has changed => WHY?
Using the default comparator with a list of comparable items seems to fail here. Why? If the observables emitted list item size changed, the data is different, so distinctUntilChanged should not filter out the new list. But it seems like this happens here. Why?
Do I really need to provide my own comparator for distinctUntilChanged if I emit a list of items that compares the list size and the items one by one?
Edit
My obs basically looks like following:
obs = Observable.combineLatest(
RxDBDataManager.appsManager.observeList(),
RxDBDataManager.widgetsManager.observeList(),
RxDBDataManager.shortcutsManager.observeList(),
RxDBDataManager.customItemsManager.observeList(),
RxDBDataManager.foldersManager.observeList(),
Function5<List<IDBApp>, List<IDBWidget>, List<IDBShortcut>, List<IDBCustomItem>, List<IDBFolder>, List<IFolderOrSidebarItem>> { t1, t2, t3, t4, t5 ->
val list = ArrayList<IFolderOrSidebarItem>()
list.addAll(t1)
list.addAll(t2)
list.addAll(t3)
list.addAll(t4)
list.addAll(t5)
list
}
.flatMapSingle {
Observable.fromIterable(it)
.filter { it.parentType == parentType && it.parentId == parentId }
.cast(T::class.java)
.toList()
}
.flatMapSingle {
Observable.fromIterable(it)
.sorted(comparator)
.toList()
}
Additionally I apply some sorting and filtering on this data with
Based on the exchange in the comments:
RxJava users are encouraged to use immutable data types in its flows which prevents concurrency issues such as modifying the same object at different stages from different threads resulting in broken operator behavior and seemingly impossible business logic failures.
In this case, distinctUntilChanged didn't work as expected because mutable items were changed in a way that two subsequent onNext signals basically had the same content and the operator filtered them out as being non-distinct.
A way to detect if the items involved are in fact the same unintentionally is to use the bi-predicate version of the operator and then placing a breakpoint in the custom lambda. This lets one inspect the previous and current values and see if they are truly equal even if they shouldn't be:
source.distinctUntilChanged((prev, curr) -> {
// breakpoint the next line
return prev.equals(curr);
});
As in this case, broken behavior was due to a mutable item changed somewhere and thus evaluating as the same as the current/previous. With Lists, it is often not practical to breakpoint all mutation methods (such as add, addAll, set, remove etc.) but one can turn a mutable list into an immutable one and send it along the sequence. The built-in way is to convert it via the Collections::unmodifiableList:
source
.toList()
.map(Collections::unmodifiableList)
;
This will crash whenever a mutation is attempted on the now unmodifiable list instance, pointing to the logic that should be investigated further.
I want to achieve the following with RxJava and as I may not have enough knowledge in this area would like to have some help :)
I need to create a PublishSubject which would emit events with the following sequence:
Emit 1, 2, 3
Buffer 4 in subscribe's completion if a certain condition is not satisfied (may be a network connection for example or some other condition).
For 5, 6 ... buffer after 4 if the condition is not satisfied yet.
Repeat to emit 4 after some time when the condition is satisfied.
If trying to emit 5,6 and the condition is satisfied, then instead of buffering 5, 6 ... after 4, just emit 4 and then 5, 6, 7 , 8 ...
The last 2 points are necessary because the sequence of emitting is really important, which makes difficulties for me to achieve to this.
I hope I could describe what I want to achieve :)
Findings: After asking this question I've done some findings and achieved the following:
private Observable observable = publishSubject
.observeOn(Schedulers.io())
.map(Manager::callNew)
.doOnError(throwable -> Logger.e(throwable, "Error occurred"))
.retryWhen(throwableObservable -> throwableObservable
.zipWith(Observable.range(1, 10), (n, i) -> i)
.flatMap(retryCount -> {
long retrySeconds = (long) Math.pow(2, retryCount);
Logger.d("The call has been failed retrying in %s seconds. Retry count %s", retrySeconds, retryCount);
return Observable.timer(retrySeconds, TimeUnit.SECONDS)
.doOnNext(aLong -> {
C24Logger.d("Timer was completed. %s", aLong);
})
.doOnComplete(() -> Logger.d("Timer was completed."));
}));
The problem is here with PublishSubject. Because it already has emitted all the items, it emits only new ones for retryWhen. If I use ReplaySubject them it emits also the old completed items too for the new retryWhen re-subscribe, which I do not need anymore.
Is there a way to use the ReplaySubject to remove the completed items from the buffer?
You want to be able to turn buffering on and off, depending upon an external condition. Perhaps the simplest way to do it is use the buffer() operator to continually buffer items based on the condition.
(I have removed stuff from the observer chain)
private Observable observable = publishSubject
.publish( obs -> obs.buffer( obs.filter( v -> externalCondition( v ) ) ) )
.flatMapIterable( bufferedList -> bufferedList )
.subscribe( ... );
The publish() operator allows multiple observer chains to subscribe to the incoming observer chain. The buffer() operator monitors the observable that emits a value only when the external condition is true.
When the external condition is true, buffer() will emit a series of lists with only a single element. When the condition goes false, buffer() starts buffering up the results, and when the condition goes true again, all the buffered items are emitted as a list. The flatMapIterable() step will take each item out of the buffer and emit it separately.
I have the following code to fetch a list of items from the internet.
Observable<RealmList<Artist>> popArtists = restInterface.getArtists();
compositeSubscription.add(popArtists.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread()).subscribe(artistsObserver));
The trouble is the list has over 80 items and i only want to get the first 5 items. What is the best way to achieve this?
takeis the operator you looking for. (see documentation here : http://reactivex.io/documentation/operators/take.html )
The flatMapIterable transform your RealmList (which implement Iterable, that's why flatMapIterable can be used) to an Observable which emiting all items of your list
Subscription subscription = restInterface.getArtists()
.flatMapIterable(l -> l)
.take(5)
.subscribeOn(Schedulers.io())
.observeOn(androidSchedulers.mainThread())
.subscribe(artistsObserver);
compositeSubscription.add(subscription);
I guess you have no control over server side, so solution is to take first 5 items from received result:
Observable<RealmList<Artist>> popArtists = restInterface.getArtists();
compositeSubscription.add(
popArtists.flatMap(list-> Observable.from(list).limit(5)).subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(artistsObserver));