I have list coming back from a REST endpoint. I need to break that list down into categories (category is an item in each entry of the list). Individual categories will be written to a cache for faster lookup later.
I didn't know if I could .map() the entries and supply multiple filter() or some type of case statement to put the category entries in the right bucket.
Does something like this sound reasonable to implement with rxJava?
UPDATE:
Non-working version
private Map<String, List<VideoMetadataInfoEntity>> buildCategories( Observable<List<VideoMetadataInfoEntity>> videoList ) {
Map<String, List<VideoMetadataInfoEntity>> categoryMap = new HashMap<>();
videoList
.flatMap( Observable::from )
.subscribe( videoMetadataInfoEntity -> mapCategory(videoMetadataInfoEntity, categoryMap ) );
Observable.just( categoryMap )
.doOnNext( saveCategoriesToCacheAction );
return categoryMap;
}
These fire in sequence, however, and this is my understanding, the second observable is not sending anything the saveCategoriesToCacheAction since it hasn't subscribed to the result of the first observable.
I am starting to think I should modify my cache strategy. The list will always have all the details. The service doesn't provide me a subset that I can use for listing and then another call to get the full details. It is either full list or full details for one item. It might be a better approach to just cache each one individually and into their own category caches right now. I was trying to do the map so that this network call could return the requested category, but subsequent calls would come from the cache, until such time as the cache has expired and a new network call refreshes it.
My solution is:
Observable.range(1, 20)
.groupBy(number -> number % 2)
.flatMap(groupedObservable -> groupedObservable.toList())
.toMap(list -> list.get(0) % 2);
As a result I have [{0=[2, 4, 6, 8, 10, 12, 14, 16, 18, 20], 1=[1, 3, 5, 7, 9, 11, 13, 15, 17, 19]}]
Explanation:
range(1, 20) - creates an observable which emits first twenty numbers
groupBy(number -> number % 2) - creates an observable that emits group observables where each group observable holds items grouped with the grouping function(here it is x % 2)
flatMap(groupedObservable -> groupedObservable.toList()) - turns each group into an observable that emits all its items as a list
toMap(list -> list.get(0) % 2) - creates the map
RxJava is more for asynchronous message processing, but as it also espouses functional programming principles it could be used as a poor man's stream api. If you are using Java 8 consider using streams to do this job, but as you are asking this question I assume you are using Java 7.
To do what you want you could try (forgive the lambda, substitute it with an anonymous inner class if you are not using Retrolambda):
Observable.from(list).subscribe(item -> groupItemInCategoryBucket(item));
where groupItemInCategoryBucket is your method that contains the switch statement or whatever other way you have of caching the items.
Please note that this is the equivalent of a for loop, and although it is idiomatic to use this style in many other nice languages, a lot of Java developers might be a bit puzzled when they see this code.
Generally grouping of items can be achieved using a groupBy operator (for more information about it visit this page).
Map<Integer, List<Integer>> groupedValues = new HashMap<>(4);
Observable.range(1, 20)
.groupBy(i -> i % 2, i -> i)
.subscribe(go -> {
List<Integer> groupValues = new ArrayList<>();
groupedValues.put(go.getKey(), groupValues);
go.subscribe(t -> add(t, groupValues));
});
How it works:
Firstly, observable emits items 1 through 20 (this happens in range method)
Which then are emitted to separate observables based on their
parity(groupBy method, after this method you operate on GroupedObservable)
You then subscribe to the grouped observable, receiving (in subscribers onNext) separate observables that will contain grouped items and the key they were grouped by.
Remember to either subscribe to the grouped observables or issue take(0) on them if their content does not interest you to prevent memory leaks.
I am not sure whether it is the most efficient way or not and would welcome some input about this solution.
Related
I have stared learning RxAndroid and have understood the reactive programming approach and power of different operators like flatmap, map, and xmap.
I have the following situation, I have a list of Reports say List<Reports> (obtained at runtime). I have to make 3 API calls for a report in a sequence (API 1 -> API 2 -> API 3) and repeat the same process for all reports in a sequence.
Example:
For Reports A, B, C, and D in List<Reports> make 3 API call for A and then for B and then for C and then for D. If anyone of 3 API calls fails for a report then stop the remaining call and continue for the next Report. Here the size of List<Reports> is known at runtime only.
How can I use RXjava to solve this problem?
Supposing you have the 4 API calls, a combineLatest operation would ideally return type Reports.
Observable<Report> result = Observable.combineLatest(
apiA,
apiB,
apiC,
apiD,
(respA, respB, respC, respD) -> respA + respB + respC + respD
)
.subscribeOn(
....
To be able to achieve the sequential order, you will need to wrap them in a flatMap operation:
Observable o = Observable.just(.....)// your initial data
o
.flatMap(value -> doCombineLatestOperation(value))
.subscribe(resp -> //Deal with data);
Hope that helps.
My observable looks like following:
obs
.doOnNext { logger("Items changed (${it.size})") }
.distinctUntilChanged()
.doOnNext { logger("Items changed (${it.size})- EMIITTED") }
Log looks like following:
Items changed (7)
Items changed (7)- EMIITTED
Items changed (8)
// => missing EMIITTED message although it.size has changed => WHY?
Using the default comparator with a list of comparable items seems to fail here. Why? If the observables emitted list item size changed, the data is different, so distinctUntilChanged should not filter out the new list. But it seems like this happens here. Why?
Do I really need to provide my own comparator for distinctUntilChanged if I emit a list of items that compares the list size and the items one by one?
Edit
My obs basically looks like following:
obs = Observable.combineLatest(
RxDBDataManager.appsManager.observeList(),
RxDBDataManager.widgetsManager.observeList(),
RxDBDataManager.shortcutsManager.observeList(),
RxDBDataManager.customItemsManager.observeList(),
RxDBDataManager.foldersManager.observeList(),
Function5<List<IDBApp>, List<IDBWidget>, List<IDBShortcut>, List<IDBCustomItem>, List<IDBFolder>, List<IFolderOrSidebarItem>> { t1, t2, t3, t4, t5 ->
val list = ArrayList<IFolderOrSidebarItem>()
list.addAll(t1)
list.addAll(t2)
list.addAll(t3)
list.addAll(t4)
list.addAll(t5)
list
}
.flatMapSingle {
Observable.fromIterable(it)
.filter { it.parentType == parentType && it.parentId == parentId }
.cast(T::class.java)
.toList()
}
.flatMapSingle {
Observable.fromIterable(it)
.sorted(comparator)
.toList()
}
Additionally I apply some sorting and filtering on this data with
Based on the exchange in the comments:
RxJava users are encouraged to use immutable data types in its flows which prevents concurrency issues such as modifying the same object at different stages from different threads resulting in broken operator behavior and seemingly impossible business logic failures.
In this case, distinctUntilChanged didn't work as expected because mutable items were changed in a way that two subsequent onNext signals basically had the same content and the operator filtered them out as being non-distinct.
A way to detect if the items involved are in fact the same unintentionally is to use the bi-predicate version of the operator and then placing a breakpoint in the custom lambda. This lets one inspect the previous and current values and see if they are truly equal even if they shouldn't be:
source.distinctUntilChanged((prev, curr) -> {
// breakpoint the next line
return prev.equals(curr);
});
As in this case, broken behavior was due to a mutable item changed somewhere and thus evaluating as the same as the current/previous. With Lists, it is often not practical to breakpoint all mutation methods (such as add, addAll, set, remove etc.) but one can turn a mutable list into an immutable one and send it along the sequence. The built-in way is to convert it via the Collections::unmodifiableList:
source
.toList()
.map(Collections::unmodifiableList)
;
This will crash whenever a mutation is attempted on the now unmodifiable list instance, pointing to the logic that should be investigated further.
I'm trying to get my head around the golden rule (if any) about:
When to use BehaviorSubject ?
and
When to use PublishSubject ?
The difference between them is very clear
There are many kinds of subjects. For this specific requirement, a PublishSubject works well because we wish to continue the sequence from where it left off. So assuming events 1,2,3 were emitted in (B), after (A) connects back we only want to see 4, 5, 6. If we used a ReplaySubject we would see [1, 2, 3], 4, 5, 6; or if we used a BehaviorSubject we would see 3, 4, 5, 6 etc.
(source : How to think about Subjects in RxJava (Part 1))
I have seen that Subject's are used in two contexts (at least), UI context and listener context.
UI context (MVVM as example)
For example here a BehaviorSubject is used, and it's clear why they use Subject and not Observable but I have changed the BehaviorSubject to PublishSubject but the app behavior still the same.
Listener context
Why they make project field a BehaviorSubject and not PublishSubject ?
The main difference between PublishSubject and BehaviorSubject is that the latter one remembers the last emitted item. Because of that BehaviorSubject is really useful when you want to emit states.
Why they make project field a BehaviorSubject and not PublishSubject ?
Probably because they want to be able to retrieve the last emitted project with this method:
#Override public #NonNull Observable<Project> project() {
return this.project;
}
PublishSubject: Starts empty and only emits new elements to subscribers.
There is a possibility that one or more items may be lost between the time the Subject is created and the observer subscribes to it because PublishSubject starts emitting elements immediately upon creation.
BehaviorSubject: It needs an initial value and replays it or the latest element to new subscribers. As BehaviorSubject always emits the latest element, you can’t create one without giving a default initial value.
BehaviorSubject is helpful for depicting "values over time". For example, an event stream of birthdays is a Subject, but the stream of a person's age would be a BehaviorSubject.
Publish Subject: Here, if a student entered late into the classroom, he just wants to listen from that point of time when he entered the classroom. So, Publish will be the best for this use-case.
Behavior Subject: Here, if a student entered late into the classroom, he wants to listen the most recent things(not from the beginning) being taught by the professor so that he gets the idea of the context. So, here we will use Behavior.
The difference on BehaviourSubject and PublishSubject relies on how long they keep the data they captures, in instance the PublishSubject only keeps the data available at moment and keeps updating on every entry while BehaviourSubject keeps the last data inserted, so you may use for example to confirm password on a signup form and as an example for PublishSubject, performing a search and it has to update the data constantly in order to give accurate results and there's no too much necessity to compare data that are being inserted.
As reference i leave this two photos from http://reactivex.io/documentation/subject.html
PublishSubject
BehaviourSubject
I am trying to implement an Android app which needs to obtain a big amount of data from a backend service and save it to a db to later work on it.
The below code describes the process:
itemsService
.getAllItemIds() //This returns Single<List<Int>> from backend
.subscribeOn(Schedulers.io())
.subscribe({
Observable.fromIterable(it)
.map({
itemsService
.getItemById(it) //This gets one item details from backend
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
//Add item details to db
}, {
//Some error
})
})
}, {
//Some error
})
I obtain a list of ids and then map each of these ids to a network call to obtain the full object.
This works for a test set of, say, 10 items, but the production set contains over 50 000 ids. It works initially, saving the items, but around 5-10% it grinds to a halt and the app dies.
I assume the reason here would be that Rx keeps the reference between the source and the mapped value.
My question is: is there a way to "pool" the source emissions to, let's say, 10 at a time? Or maybe there is some other mechanism I am not aware of?
You didn't mention what exactly "grinds to a halt" means, but it makes sense that you will get out of memory in real case of 50,000 items, cause you will basically try to create 50,000 threads at once to fetch each items details.
moreover, instead of chaining Observables using operators, you're creating nested chains at subscribe/map, you can read here why you shouldn't.
regarding limiting the work to 10 at a time, there is an flatMap overload for that, at the end it might look something like this:
itemsService
.getAllItemIds() //This returns List<Int> from backend
.flatMapIterable { t -> t }
.flatMap({
itemsService
.getItemById(it) //This gets one item details from backend
.subscribeOn(Schedulers.io())
}, 10) //limit flat map parallelism by desired value
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
//Add item details to db
}, {
//Some error
})
I have to improve an old service which is making multiple consecutive calls to the same service and merging all the results in a single list.
Imagine we have a list of IDS = [ 1 ,2 3, .. 6] So I have to call to the same API with each ID (with retrofit and Observables).
For doing this I'd like to use Rx Android but I have doubts about how to merge the results.
Integer[] ids = {1, 2, 3};
Observable.from(ids)
.map(id -> mApi.getData(id))
Can we do parallel calls and merge the result following the same order?
For improving the time response the idea is to do this in Parallel but I don't know how to be sure the result will be combined following the same order we do the server calls.
I thought to use concat (http://reactivex.io/documentation/operators/concat.html) but it waits the previous observable to wait.
See concatMapEager and its variants.
Observable.fromArray(1, 2, 3)
.concatMapEager(id ->
Observable.fromCallable(() -> mApi.getData(id))
.subscribeOn(Schedulers.io())
)
.subscribe(...);