i am using Retrofit with RxJAva for an app that gets Rss Feeds, but the rss doesn't contain all the informations so i use jsoup to parse every item link, to retrieve the image and the article's description. now i am using it this way:
public Observable<Rss> getDumpData() {
return newsAppService.getDumpData()
.flatMap(rss -> Observable.from(rss.channel.items)
.observeOn(Schedulers.io())
.flatMap(Checked.f1(item -> Observable.just(Jsoup.connect(item.link).get())
.observeOn(Schedulers.io())
.map(document -> document.select("div[itemprop=image] > img").first())
.doOnNext(element -> item.image = element.attr("src"))
)))
.defaultIfEmpty(rss)
.ignoreElements()
.observeOn(Schedulers.io())
.subscribeOn(AndroidSchedulers.mainThread());
}
and i am getting an error on this line: defaultIfEmpty(rss)
it doesn't recognize rss of the flatmap. and when i move the defaultIfEmpty(rss) in flatmap brackets i have another error saying that the return type must be changed to Element. is their any solution ?
first of all you need to get rid of all the concurrency with observeOn and use subscribeOn.
.observeOn(Schedulers.io())
Please consider using observeOn with AndroidScheduler if want to sync back data from another thread back to the event-loop. Normally you would use observeOn before subscribing to a observable in order to sync back to ui-loop and change ui-information.
.observeOn(AndroidSchedulers.mainThread())
Secondly it is not recommended to mutate objects in the pipeline. You should return a new object very time.
.doOnNext(element -> item.image = element.attr("src"))
I tried to refactor your solution under consideration of the first two points. I am using RxJava2-RC5
The flatMap operator has many overloades. One of them provides a function to zip together the incoming value and the created value.
Observable<Rss> rssItemObservable = newsService.getDumpData()
.flatMap(rss -> getRssItemInformation(rss).subscribeOn(Schedulers.io()),
(r, rItemList) -> {
Rss rInterim = new Rss();
rInterim.items = rItemList;
return rInterim;
});
Helping-method for retrieving information for each item in Rss. Please consider using the overload with maxConcurrency, because on default it will subscribe to every stream at once. Therefore flatMap would create many http-requests.
private Observable<List<RssItem>> getRssItemInformation(Rss rss) {
return Observable.fromIterable(rss.items)
.flatMap(rssItem -> getImageUrl(rssItem).subscribeOn(Schedulers.io()), (rItem, img) -> {
RssItem item = new RssItem();
printCurrentThread("merge1");
item.image = img;
item.link = rItem.link;
return item;
}).toList().toObservable();
}
Helping-method for retrieving the image url. Returning observable is not opinionated about concurrency. If an error occurs, an empty string will be returned as default value.
private Observable<String> getImageUrl(String link) {
return Observable.fromCallable(() -> Jsoup.connect(link).get())
.map(document -> document.select("div[itemprop=image] > img").first())
.map(element -> element.attr("src"))
.onErrorResumeNext(throwable -> {
return Observable.just("");
});
}
You may look at the full example at github.gist: https://gist.github.com/anonymous/a8e36205fc2430517c66c802f6eef38e
You can't mix internal parameter of one RxJava parameter (flatMap lambda parameter) with another operator parameter (defaultIfEmpty).
First of all, create a helper function to keep main reactive stream cleaner:
private Observable<List<Item>> getDetails(List<Item> items) {
return Observable.from(items)
.observeOn(Schedulers.io())
.flatMap(Checked.f1(item ->
Observable.zip(
Observable.just(item),
Observable.just(Jsoup.connect(item.link).get())
.observeOn(Schedulers.io())
.map(document -> document.select("div[itemprop=image] > img").first()),
(itemInner, element) -> {
itemInner.image = element.attr("src");
return itemInner;
}
)
))
.toList();
}
Then reformat main function:
newsAppService.getDumpData()
.flatMap(rss ->
Observable.zip(
Observable.<Rss>just(rss),
getDetails(rss.channel.items),
(rssInner, items) -> {
rssInner.channel.items = items;
return rss;
}).onErrorResumeNext((throwable -> Observable.just(rss))
)
)
.observeOn(Schedulers.io())
.subscribeOn(AndroidSchedulers.mainThread());
Hope I got your aim properly. It may not work, as I am unable to test it, however I hope you get the idea. The reason I used .zip functions it that you can't loose the reference to the currently parsed item or rss
Related
I need make two parallel requests with RxJava. For this I use zip operator. Here is my code:
public Disposable getBooksAndAuthors(String id, ReuqestCallback requestCallback) {
return singleRequest(Single.zip(
getBooks(id).subscribeOn(Schedulers.newThread()).observeOn(AndroidSchedulers.mainThread()),
getAuthors(id).subscribeOn(Schedulers.newThread()).observeOn(AndroidSchedulers.mainThread()),
(book, author) -> new ZipResponseWrapper(book, author).getResponse()), requestCallback);
}
private <T extends NetworkResponse> Disposable singleRequest(Single<T> single, RequestCallback requestCallback) {
return single.doOnSubscribe(d -> requestCallback.onStartRequest())
.doOnSuccess(s -> requestCallback.onSuccess(s))
.doOnError(ErrorConsumer.consume((t) -> requestCallback.onError(t)))
.doFinally(() -> requestCallback.onFinish())
.subscribe();
}
But I don’t understand how to receive response separately for each request. That is, I need to, if the answer came to the first request, immediately display the data received from this request and not wait for a response to the second request. And after the answer to the second request arrives, display the data received on the second request.This is necessary due to the fact that the second request fulfills a long time. Please help me.
Here is an example of how you can handle it with the responses for each function:
val disposable = Observable.zip(
firstNetworkCall().subscribeOn(Schedulers.io()),
secondNetworkCall().subscribeOn(Schedulers.io()),
BiFunction{
firstResonse: ResponseOneType,
secondResponse: ResponseTwoType ->
combineResult(firstResponse, secondResponse) }))
.observeOn(AndroidSchedulers.mainThread())
.subscribe { it -> doSomethingWithIndividualResponse(it) }
My suggestion (in Kotlin though):
val id = 0L
Observables.combineLatest(
getBooks(id).startWith(emptyList<Book>()).subscribeOn(Schedulers.io()).observeOn(Schedulers.computation()),
getAuthor(id).startWith(emptyList<Author>()).subscribeOn(Schedulers.io()).observeOn(Schedulers.computation())
) { book: List<Book>, author: List<Author> ->
Pair(book, author)
}.skip(1)
.observeOn(AndroidSchedulers.mainThread())
.subscribe { (books: List<Book>, authors: List<Author>) ->
view.show(books)
view.show(authors)
}
I have a method that does a call to Firebase. This method accepts a date and returns an observable.
Then I have an array of dates, that will be used as parameter of this firebase call.
I need to call the method once per item in the array, and finally concatenate to a one list.
But I don't know how to achieve it.
I'm trying to do something like:
for (dateToRetrieve in listOfDatesToRetrieve) {
val subscription = FireBaseUtils.getEventsForMap(dateToRetrieve)
.subscribeOn(Schedulers.io())
.subscribe { retrievedEventsForMap ->
val eventsList: MutableList<Event> = retrievedEventsForMap
eventListWithNoDuplicatesTotal.addAll(eventsList)
var eventListWithNoDuplicates = eventListWithNoDuplicatesTotal.distinctBy { it -> it.eventID }
this.presenter.onEventsRetrieved(eventListWithNoDuplicates as MutableList<Event>)
}
this.presenter.addSubscription(subscription)
}
But I know that is not the best solution because I'm sending the calls one by one, and adding to the list.
Is there any possibility to do it and return 1 result with the combination of all the calls?
Thanks
Try this:
val subscription = Observable.fromIterable(listOfDatesToRetrieve)
.flatMap(dateToRetrieve -> FireBaseUtils.getEventsForMap(dateToRetrieve))
.flatMapIterable { item -> item }
.toList()
.distinct { it -> it.eventID }
.subscribeOn(Schedulers.io())
.subscribe { list ->
this.presenter.onEventsRetrieved(list as MutableList<Event>)
}
this.presenter.addSubscription(subscription)
I've a TestService, where I do an async task to get my data. I would like to wait for the response before I continue.
public List<Data> getData() {
List<Data> data = new ArrayList<>();
Disposable disposable = repository.getDataFromApi(false)
.observeOn(AndroidSchedulers.mainThread())
.subscribe( newData -> {
data.addAll(newData);
}, __ -> { });
mCompositeDisposable.add(disposable);
//Here I want to stop till "Data" arraylist is filled with data
... do something with data
}
In Volley I could just call req.executeSynchronously(); to make it happen. As getData() have to return data already, I've to somehow make it wait till I get response. How to do it? I'm using Single.
My approach using getBlocking();
public List<Data> getData() {
List<Data> data = new ArrayList<>();
Disposable disposable = repository.getDataFromApi(false)
.observeOn(AndroidSchedulers.mainThread())
.blockingGet();
.subscribe( newData -> {
data.addAll(newData);
}, __ -> { });
mCompositeDisposable.add(disposable);
//Here I want to stop till "Data" arraylist is filled with data
... do something with data
}
It says cannot resolve method subscribe, so I'm probably calling it wrong..
fun getDataFromApi(): Single<List<Data>> {
return service.getData()
.map { jsonApiObject ->
...
return#map data
}
}
Hopefully you are aware that blocking is a strong antipattern in RxJava and you should avoid blocking whenever you can.
Saying that, if you really need to block, you have two options:
use blockingGet() which - as the name indicates - blocks current thread and directly returns value of publisher (Single in your case). This is probably what you were looking for. In your case:
newData = repository.getDataFromApi(false).blockingGet();
data.addAll(newData);
synchronize with Java classes, like CountDownLatch - more complicated and I would use blockingGet() because it's more straightforward. But it's a possibility.
While iterating and fetching web responses the chain stops when it encounters an error.
I used .onErrorResumeNext(Observable.empty()) to keep the iteration going but want to do some error handling too. How can this be done?
.getRepos()
.flatMap { itemList ->
//fetches all repos
Observable.fromIterable(itemList)
}
.concatMapEager { githubItem ->
//fetches commits of each repos
networkModule.getCommits().map { commitItem ->
dataBaseModule.insertRepo(commitItem)
}.onErrorResumeNext(Observable.empty())
//here I want to take some action instead of just passing an empty observable
}
You can use the doOnError() operator just before the onErrorResumeNext() to perform an action.
...
.doOnError( error -> {} )
.onErrorResumeNext( Observable.empty() )
...
I want to execute a background task on a set of items from a database using RxJava 2. The list of items may be empty, which means I can't use flatMapIterable as it throws an exception on an empty list:
Observable
.fromCallable(() -> SQLite
.select()
.from(VideoUpload.class)
.where(VideoUpload_Table.status.eq(VIDEO_UPLOAD_IN_PROGRESS))
.queryList())
.flatMapIterable(videoUploads -> videoUploads)
.map(videoUpload -> {
videoUpload.setStatus(VIDEO_UPLOAD_NOT_STARTED);
return videoUpload;
})
.firstElement()
.subscribeOn(Schedulers.io())
.subscribe(/* TODO */);
I can move everything into the callable, do the filtering there and so on, but I was thinking there may be a more elegant solution based on the code above.
Turns out that flatMapIterable does not throw an exception on RxJava-2, I remember it throwing an exception on RxJava-1. Anyway, the way to do it is as follows
Observable
.fromCallable(() -> {
List<VideoUpload> videos = SQLite
.select()
.from(VideoUpload.class)
.where(VideoUpload_Table.status.in(VIDEO_UPLOAD_IN_PROGRESS, VIDEO_UPLOAD_QUEUED))
.queryList();
return videos;
})
.flatMapIterable(videoUploads -> videoUploads)
.map(videoUpload -> {
videoUpload.setStatus(VIDEO_UPLOAD_NOT_STARTED);
return videoUpload;
})
.firstElement()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(/* TODO */);