I'm almost sold to RxJava, which is a perfect companion to Retrofit, but I'm struggling into a common pattern while migrating my code: to save bandwidth, I'd like to lazily fetch (paginated) objects from my webservice as needed, while my listview (or recyclerview) is scrolling using reactive programming.
My previous code was doing the job perfectly, but reactive programming seems worth the try.
Listening to listview/recyclerview scrolling (and other boring stuffs) isn't the concern and getting an Observable is easy using Retrofit:
#GET("/api/messages")
Observable<List<Message>> getMessages(#Path("offset") int offset, #Path("limit") int limit);
I just can't figure out the pattern to use in reactive programming.
The Concat operator seems a good starting point, along with ConnectableObservable at some point to defer emission and maybe flatMap, but how ?
EDIT:
Here's my current (naive) solution:
public interface Paged<T> {
boolean isLoading();
void cancel();
void next(int count);
void next(int count, Scheduler scheduler);
Observable<List<T>> asObservable();
boolean hasCompleted();
int position();
}
And my implementation using a subject:
public abstract class SimplePaged<T> implements Paged<T> {
final PublishSubject<List<T>> subject = PublishSubject.create();
private volatile boolean loading;
private volatile int offset;
private Subscription subscription;
#Override
public boolean isLoading() {
return loading;
}
#Override
public synchronized void cancel() {
if(subscription != null && !subscription.isUnsubscribed())
subscription.unsubscribe();
if(!hasCompleted())
subject.onCompleted();
subscription = null;
loading = false;
}
#Override
public void next(int count) {
next(count, null);
}
#Override
public synchronized void next(int count, Scheduler scheduler) {
if (isLoading())
throw new IllegalStateException("you can't call next() before onNext()");
if(hasCompleted())
throw new IllegalStateException("you can't call next() after onCompleted()");
loading = true;
Observable<List<T>> obs = onNextPage(offset, count).single();
if(scheduler != null)
obs = obs.subscribeOn(scheduler); // BEWARE! onNext/onError/onComplete will happens on that scheduler!
subscription = obs.subscribe(this::onNext, this::onError, this::onComplete);
}
#Override
public Observable<List<T>> asObservable() {
return subject.asObservable();
}
#Override
public boolean hasCompleted() {
return subject.hasCompleted();
}
#Override
public int position() {
return offset;
}
/* Warning: functions below may be called from another thread */
protected synchronized void onNext(List<T> items) {
if (items != null)
offset += items.size();
loading = false;
if (items == null || items.size() == 0)
subject.onCompleted();
else
subject.onNext(items);
}
protected synchronized void onError(Throwable t) {
loading = false;
subject.onError(t);
}
protected synchronized void onComplete() {
loading = false;
}
abstract protected Observable<List<T>> onNextPage(int offset, int count);
}
Here's one out of a few potential ways to handle reactive paging. Let's assume we have a method getNextPageTrigger which returns an Observable emits some event object when the scroll listener (or whatever input) wants a new page to be loaded. In real life it would probably have the debounce operator, but in addition to that we'll make sure we only trigger it after the latest page has loaded.
We also define a method to unwrap the messages from their list:
Observable<Message> getPage(final int page) {
return service.getMessages(page * PAGE_SIZE, PAGE_SIZE)
.flatMap(messageList -> Observable.from(messageList));
}
Then we can make the actual fetching logic:
// Start with the first page.
getPage(0)
// Add on each incremental future page.
.concatWith(Observable.range(1, Integer.MAX_VALUE)
// Uses a little trick to get the next page to wait for a signal to load.
// By ignoring all actual elements emitted and casting, the trigger must
// complete before the actual page request will be made.
.concatMap(page -> getNextPageTrigger().limit(1)
.ignoreElements()
.cast(Message.class)
.concatWith(getPage(page))) // Then subscribe, etc..
This is still missing a couple potentially important things:
1 - This obviously doesn't know when to stop fetching additional pages, which means once it hits the end, depending on what the server returns, it could either keep hitting errors or empty results every time scroll is triggered. Approaches to solving this depend on how you signal to the client that there are no more pages to load.
2 - If you need error retries, I would suggest looking into the retryWhen operator. Otherwise, common network errors could cause an error in a page load to propagate.
Related
Apologies in advance if I lack a basic understanding of how to use RxJava2, because this seems to me something that should be quite fundamental. I've wracked my brains with unsuccessful Google searches, so welcome any resource recommendations. I've opted to use a 'sanitized' representation of my workaround code for the sake of clarity.
Problem description
I have an RxJava2 function asyncCallForList() that returns a Maybe<Arraylist<CustomClass>>. Each CustomClass object in this list only has a few basic fields populated (e.g. the source database only contains a unique identifier and a title string for each item).
The full data required for each item is in another database location, which is retrieved using another function asyncCallForItem(uid), which returns a Maybe<CustomClass> based on the unique identifier, where the encapsulated CustomClass has all the required data. This function is to be called for each item in the list returned by asyncCallForList().
The desired functionality is to update my UI once all the objects in the list have been populated.
Workaround #1
It is easy enough to loop through the resulting array list in the doOnSuccess() attached to the initial Maybe<Arraylist<CustomClass>>, then update my UI in the doOnSuccess() on the Maybe<CustomClass> returned by the subsequent asynchronous calls. This is not an acceptable workaround as there will be an unknown number of UI updates being made (the initial list returned could have any amount of items) and will hurt performance.
Workaround #2
This gets the desired outcome but feels like the wrong way to go about it - I suspect there is a more elegant RxJava2 solution. Basically, I create a custom Observable in which loop through the items in the list and get the full data for each. However, rather than update the UI each time I populate a CustomClass item, I increase a counter, then check if the counter exceeds or equals the initial list size. When this condition is met I call the onComplete() method for the observable's emitter and update the UI there.
private void fetchRemoteDataAndUpdateUi() {
//Counter reset to zero before any asynchronous calls are made.
int count = 0;
Maybe<ArrayList<CustomClass>> itemList = asyncCallForList();
Consumer<ArrayList<CustomClass>> onListReturnedSuccess;
onListReturnedSuccess = new Consumer<ArrayList<CustomClass >>() {
#Override
public void accept(ArrayList<CustomClass> list) throws Exception {
//Custom observable created here, in which the resulting array list is processed.
listObservable = Observable.create(new ObservableOnSubscribe<CustomClass>() {
#Override
public void subscribe(final ObservableEmitter<CustomClass> e) throws Exception {
for (CustomClass customClass : list) {
final CustomClass thisCustomClass = customClass;
//Call to get full data on list item called here.
asyncCallForItem(customClass.getUid())
.doOnSuccess(new Consumer<CustomClass>() {
#Override
public void accept(CustomClass customClass) throws Exception {
thisCustomClass.update(customClass);
e.onNext(thisCustomClass);
count++;
if (count >= list.size()) {
e.onComplete();
}
}
}).subscribe();
}
}
});
listObservable
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe(new Observer<CustomClass>() {
#Override
public void onSubscribe(Disposable d) {
}
#Override
public void onNext(CustomClass customClass) {
//Here I add the populated CustomClass object to an ArrayList field that is utilised by the UI.
listForUi.add(customClass);
}
#Override
public void onError(Throwable e) {
}
#Override
public void onComplete() {
//Here the UI is finally updated once all CustomClass objects have been populated.
updateUi();
}
});
}
};
//Kick everything off.
itemList.doOnSuccess(onListReturnedSuccess).subscribe();
}
flatMap it!
asyncCallForList()
.subscribeOn(Schedulers.io())
.flatMapSingle(list ->
Flowable.fromIterable(list)
.flatMapMaybe(item ->
asyncCallForItem(item.id)
.subscribeOn(Schedulers.io())
.doOnSuccess(response -> {
// copy state from the original item
response.text = item.text;
})
, 1) // number of concurrent item calls
.toList()
)
.observeOn(AndroidSchedulers.mainThread())
.subscribe(successList -> { /* update UI */ }, error -> { /* report error */ });
just like this sample
Observable.fromCallable(() -> okHttp.getDataFromNet(page))
.subscribeOn(Schedulers.io())
.doOnSubscribe(()->{adapter.loadmore})
.subscribeOn(AndroidSchedulers.mainThread())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Subscriber<String>() {
#Override
public void onCompleted() {
adapter.stopLoad();
}
#Override
public void onError(Throwable throwable) {
}
#Override
public void onNext(String s) {
Log.e("fail","fail");
adapter.add();
}
});
I find onNextwill be called two times in this case.If i add Log.e() in
onCompleted(),it also will called two times.What is wrong about my code.What is the reason that if a method will be called in Subscriber ,then I add doOnSubscribe() it will be called two times。
//progressbarAdapter extends Recyclerview.Adapter
static final int LOADING_TYPE = -1;
private boolean isLoading = false;
public void loadingMore(){
isLoading = true;
notifyItemInserted(getItemCount());
// size = index +1
}
public void loadingEnd(){
notifyItemRemoved(getItemCount());
isLoading = false;
}
#Override
public int getItemCount() {
return getDataCount() + (isLoading ? 1: 0);
}
#Override
public int getItemViewType(int position) {
if (position == getDataCount() && !isError){
return LOADING_TYPE;
}else {
return getExtItemViewType(position);
}
}
the real reason
when adapter.notify it will trigger scroll methods,so it will be called two times. I am sorry for that I asked a bad question.
please have a look how to use observeOn and subscribeOn. There can only be one subscribeOn for an observable. If you use multiple, the last one wins. Use observeOn to switch threads between pipelines and subscribeOn to specify the thread which will create the observable.
I transformed your given code in order to use it in a test. I only get one onNext-call.
Problem: getting called two times.
This may be because of wrong usage of the observable somewhere else in the code by flatMap or multiple subscriptions to observable. Please provide more context.
Testenvironment: IntelliJ Idea 2017 EAP, io.reactivex:rxjava:1.2.1
#Test
public void name() throws Exception {
Observable<String> doOnSub = Observable.just("1")
.subscribeOn(Schedulers.io())
.doOnSubscribe(() -> {
System.out.println("doOnSub");
})
.subscribeOn(Schedulers.computation())
.observeOn(Schedulers.computation());
TestSubscriber<String> objectTestSubscriber = TestSubscriber.create();
doOnSub.subscribe(objectTestSubscriber);
objectTestSubscriber.awaitTerminalEvent();
objectTestSubscriber.assertValues("1");
}
I am just getting started with RxJava2 and wonder how I could correctly implement a UDP observable.
I already have some working code, but I think there may be some issues: see the 4 questions in the comments of the source-code below.
I've also published the code on GitHub RxJava2_Udp: comments, issues and pull requests welcome.
class UdpObservable {
private static class UdpThread extends Thread {
private final int portNo;
private final int bufferSizeInBytes;
private final ObservableEmitter<DatagramPacket> emitter;
private DatagramSocket udpSocket;
private UdpThread(#NonNull ObservableEmitter<DatagramPacket> emitter
, int portNo, int bufferSizeInBytes) {
this.emitter = emitter;
this.portNo = portNo;
this.bufferSizeInBytes = bufferSizeInBytes;
}
#Override
public void run() {
try {
// we don't want to create the DatagramSocket in the constructor, because this
// might raise an Exception that the observer wants to handle
udpSocket = new DatagramSocket(portNo);
try {
/* QUESTION 1:
Do I really need to check isInterrupted() and emitter.isDisposed()?
When the thread is interrupted an interrupted exception will
be raised anyway and the emitter is being disposed (this is what
caused the interruption)
*/
while (!isInterrupted() && !emitter.isDisposed()) {
byte[] rcvBuffer = new byte[bufferSizeInBytes];
DatagramPacket datagramPacket = new DatagramPacket(rcvBuffer, rcvBuffer.length);
udpSocket.receive(datagramPacket);
// QUESTION 1a: same as QUESTION 1 above
if (!isInterrupted() && !emitter.isDisposed()) {
emitter.onNext(datagramPacket);
}
}
} finally {
closeUdpSocket();
}
} catch (Throwable th) {
// the thread will only be interrupted when the observer has unsubscribed:
// so we need not report it
if (!isInterrupted()) {
if (!emitter.isDisposed()) {
emitter.onError(th);
} else {
// QUESTION 2: is this the correct way to handle errors, when the emitter
// is already disposed?
RxJavaPlugins.onError(th);
}
}
}
}
private void closeUdpSocket() {
if (!udpSocket.isClosed()) {
udpSocket.close();
}
}
#Override
public void interrupt() {
super.interrupt();
// QUESTION 3: this is called from an external thread, right, so
// how can we correctly synchronize the access to udpSocket?
closeUdpSocket();
}
}
/**
* creates an Observable that will emit all UDP datagrams of a UDP port.
* <p>
* This will be an infinite stream that ends when the observer unsubscribes, or when an error
* occurs. The observer does not handle backpressure.
* </p>
*/
public static Observable<DatagramPacket> create(final int portNo, final int bufferSizeInBytes) {
return Observable.create(
new ObservableOnSubscribe<DatagramPacket>() {
#Override
public void subscribe(ObservableEmitter<DatagramPacket> emitter) throws Exception {
final UdpThread udpThread = new UdpThread(emitter, portNo, bufferSizeInBytes);
/* QUESTION 4: Is this the right way to handle unsubscription?
*/
emitter.setCancellable(new Cancellable() {
#Override
public void cancel() throws Exception {
udpThread.interrupt();
}
});
udpThread.start();
}
}
);
}
}
Generally speaking, I think it is not the right way of creating it, you should not create thread yourself, as RxJava and it's Schedulers should do it for you.
Consider that the code that code executed at the ObservableOnSubscribe will run at a thread per your Scheduler strategy, so you don't need to construct it yourself. just do the ude while-loop inside the create.
You don't need to call Thread.interrupt() method, RxJava will do that for you when you're dispose (unsubscribe) the Observable. (set the cancelable before the while loop of course)
As for your questions:
You don't need to check for the interrupted as the exception will
be raise if you'r waiting for io operation, you also don't need to
check for the disposal because onNext() will do it for you and will
not emit of unsubscribed.
Again you can call onError and the emitter will take care of checking if the Observable was unsubscribed.
As said before, there should be no Thread, but for resource cleanup, you can use the emitter.setCancellable method. (close the stream), this is happen on the same thread your code runs.
Answered before, Thread.interrput() will be raised with dispose/unsubscribe by RxJava, resource clean up should go to the emitter.setCancellable method
I want to chain up three network calls with RxJavaand Retrofit. The first call (retrieves the session token) has to be the first, the other two depend on this call and if the first call isn't finished before, the other two calls will result in an error.
For the other two calls, they should retrieve some information and update the UI. What would be the best way to proceed?
I first thought about using the zip Operator, but I'm not sure if it respects the order of the requests and as it returns a value, it felt like abusing it to just use it to bundle up the requests without any further processing.
My second approach would be to flatmap the requests and use doOnNext to update the UI once, but I'm not certain if this is the correct way.
private void setUpInitialUIState() {
restClient.requestSessionToken()
.subscribeOn(Schedulers.io())
.unsubscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread());
.flatMap(new Func1<SessionTokenResponse, Observable<CurrentPlmnResponse>>() {
#Override
public Observable<CurrentPlmnResponse> call(SessionTokenResponse sessionTokenResponse) {
return restClient.requestCurrentPlmn();
}
})
.doOnNext(new Action1<CurrentPlmnResponse>() {
#Override
public void call(CurrentPlmnResponse currentPlmnResponse) {
if (!currentPlmnResponse.isError()) {
tvProvider.setText(currentPlmnResponse.getData().getFullName());
}
}
})
.flatMap(new Func1<CurrentPlmnResponse, Observable<MonitoringStatusResponse>>() {
#Override
public Observable<MonitoringStatusResponse> call(CurrentPlmnResponse currentPlmnResponse) {
return restClient.requestMonitoringStatus();
}
})
.subscribe(new Subscriber<MonitoringStatusResponse>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable throwable) {
Log.d("onError", throwable.toString());
}
#Override
public void onNext(MonitoringStatusResponse monitoringStatusResponse) {
if (monitoringStatusResponse != null && !monitoringStatusResponse.isError() && monitoringStatusResponse.getData().getSignalIcon() >= 0 && monitoringStatusResponse.getData().getSignalIcon() <= 5) {
ivSignalStrength.setImageResource(getResources().getIdentifier("ic_signal_" + monitoringStatusResponse.getData().getSignalIcon(), "drawable", getPackageName()));
tvNetworkType.setText(getNetworkTypeTitle(monitoringStatusResponse.getData().getCurrentNetworkType()));
}
}
});
}
Depends if you want your 2nd and 3rd calls to be executed in parallel or one after another. If in parallel go for the .zip and don't feel bad about it :)
3 tips on your (current) code (maybe you are aware already or slightly different in your app, so apologies):
Catch the subscription returned from the .subscribe and kill (unsubscribe) at onDestroy the latest. If the app closes the network calls will continue to live.
If .requestCurrentPlmn() is in a thread then the .setText will complain from a touching view from not ui thread exception.
You miss a .onError in your .subscribe. If a request fails, the app will crash.
Here is a use case I am trying to resolve with rxJava and Dagger2 in my android app.
Load recording details
Check backend server if HLS transcode exists (REST Call)
If exists, monitor until process is 100% (REST Call every n seconds until 100%)
If does not exist, don't call monitor process
The REST Calls are injected through a dagger component. I am struggling with setting up rxJava to create a monitor that will refresh the REST Call until the process is 100% and stops, or the user just backs out the screen.
I am not sure I am asking this question in the correct way, so if an update is required, please let me know.
Here is a link to my presenter on github repo. This loads the data and needs to trigger the updates back to the fragment that is responsible for displaying data.
UPDATE: 2015-10-26 PM
I know this is probably a hack, but this is how I implemented the repeating delayed calls:
#Override
protected Observable buildUseCaseObservable() {
Action1<List<LiveStreamInfo>> onNextAction = new Action1<List<LiveStreamInfo>>() {
#Override
public void call( List<LiveStreamInfo> liveStreamInfos ) {
try {
Thread.sleep( 5000 );
} catch( InterruptedException e ) { }
}
};
return this.contentRepository.liveStreamInfos( this.filename )
.repeat( Schedulers.io() )
.doOnNext( onNextAction );
}
Then, in the call method that establishes a subsriber:
private void getProgramDetails() {
this.getProgramDetailsUseCase.execute(new ProgramDetailsSubscriber());
}
And the subscriber:
private final class LiveStreamInfosListSubscriber extends DefaultSubscriber<List<LiveStreamInfo>> {
#Override
public void onCompleted() {
...
}
#Override
public void onError( Throwable e ) {
...
}
#Override
public void onNext( List<LiveStreamInfo> liveStreamInfos ) {
if( null != liveStreamInfos && !liveStreamInfos.isEmpty() ) {
ProgramDetailsPresenter.this.showLiveStreamDetailsInView( liveStreamInfos.get( 0 ) );
if( liveStreamInfos.get( 0 ).getPercentComplete() == 100 ) {
ProgramDetailsPresenter.this.getLiveStreamsListUseCase.unsubscribe();
}
}
}
}
The subscriber will unsubscribe from the observable once the percent complete reaches 100%, cancelling all future call. The benefit here is that this subscriber fires when a user initiates the transcode, creating the live stream, from within the app, or it picks it up from the backend is it is initiated from the backend web interface.
How about adding .retry() with how often you want to retry and a large value for the number of retries to your rx observer. Then just unsubscribe from your source observable when exiting your fragment to stop the polling.