Develop Reference

RxJava 2 Nested Network Requests

in the app I am currently working on I use retrofit to create an Observable <ArrayList<Party>>.
Party has a hostId field as well as a field of type User which is null at the point of creation by Retrofits GsonConverter. I now want to use hostId to make a second request getting the user from id and adding the User to the initial Party. I have been looking into flatmap but I haven't found an example in which the first observable's results are not only kept but also modified.
Currently, to get all parties without the User I am doing :
Observable<ArrayList<Party>> partiesObs = model.getParties();
partiesObs.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(this::handlePartyResponse, this::handleError);
How would I go about adding User to every Party without having to call model.getUsers() in the onSuccess() method of the inital call and then having to iterate through the two lists?
I understand that flatmap() returns a new Observable while map doesn't but I am unsure about how to use either in this scenario.
Thank you

As in the comment, you should try and get the backend API changed for something like this to avoid an inelegant and inefficient solution.
If this is not feasible, you could probably do something like this:
.flatMapIterable(list -> list)
.flatMap(party -> model.getUser(party.hostId),
(party, user) -> new Party(user, party.hostId, party.dontCare))
Where:
flatMapIterable flattens the Observable<ArrayList<Party>> into an Observable<Party>
The overload of flatMap takes a Function for transforming emissions (Party objects) into an ObservableSource (of User objects) as the first parameter. The second parameter is a BiFunction for combining the Party and User objects which you can use to create a fully fledged Party object.
The last step is much easier if you have a copy or clone operation on the Party object that takes a previous instance and adds fields to it.

Rxjava Filtering the duplicate item in List

I am using RxJava and Retrofit in My App.Here's the setup of the app.When the app is launched the app make two request one to the database and other to the Network Api (using Retrofit) and both request return a Observable<List<Article>>. So what I did is basically merged the two Observable. Now the problem is sometimes the network return Articles that are already present in the Database. So how do I filter out the duplicate item from the List. Here's my Code.
return Observable.merge(dataSource.getArticles(source), remoteSource.getArticles(source))
.distinct();
So I tried distinct operator but it's not filtering the Articles out.Here's the output looks like form db.
Article1
Article2
Article3
Article4
Output from Network
Article7
Articke8
Article3
Article4
What I want is a distinct list of Article

Assuming your Article has proper equals implementation,
you could collect them into a set:
dataSource.getArticles(source)
.mergeWith(remoteSource.getArticles(source))
.collect(HashSet::new, (set, list) -> set.addAll(list))
or you could unroll each list and apply distinct followed by toList:
dataSource.getArticles(source)
.mergeWith(remoteSource.getArticles(source))
.flatMapIterable(v -> v)
.distinct()
.toList()

That's because they are returning different lists. So the distinct method recognize them as different items
If you want to emit first the database items and then add the server ones... This may be a bit more complex but not too much ;)
Observable<List<Article>> databaseArticles = ...
Observable<List<Article>> serverArticles = ...
Observable<List<Article>> allArticles =
Observable.combineLatest(
databaseArticles,
serverArticles
.startWith(emptyList()), // so it doesn't have to wait until server response
(dbItems, sItems) => {
// Combine both lists without duplicates
// e.g.
Set<Article> all = new HashSet<>();
Collections.addAll(all, dbItems);
Collections.addAll(all, sItems);
return new ArrayList<>(all);
});

Combining database calls using RxJava

In my application I am accessing my business objects using a Repository and RxJava. From my Presenter layer I can ask the Repository to get me ObjectA or get me ObjectB and the Respository will return an Observable or Observable respectively.
Sometimes I have the need to get both ObjectA and ObjectB. I'm wondering what the options are for fetching both objects using RxJava that also allows unit testing of my Presenter layer.
The way I originally implemented this was to use the flatMap() operator. So I'd do something like this:
//In Presenter layer
Observable<ObjectA> obsA = repository.getObjectA();
Observable<ObjectB> obsB = repository.getObjectB();
Observable<ObjectB> obsResult = obsA.flatMap(new Func1<ObjectA, Observable<ObjectB>>() {
#Override
public Observable<ObjectB> call(ObjectA a) {
mObjectA = a;
return obsB;
}
});
When I subscribe to the obsResult, obsA runs and I can access its result in the flatMap() operator. I get a handle to its result and store it as a field in my Presenter. Then obsB runs and I get its result in my Subscriber. This works just fine but I can't help but think I'm not really doing it right. One issue is that although I may have 100% test coverage in my Repository class, now I'm manipulating the observables that come out of the Repository class and I'm having a hard time figuring out how to test the obsResult or to verify that the code I write in flatMap() is correct.
Should I be adding methods to my Repository layer such as getObjectAandObjectB and return an Observable and do all of the work in the Repository layer? This would allow me to test the resultant Observable in the Repository layer tests rather than trying to create a new Observable from two separate Observables and figure out how to test it in my Presenter layer.
Another thing I've looked at is using the zip() operator. This would look something like this:
Observable<ObjectA> obsA = repository.getObjectA();
Observable<ObjectB> obsB = repository.getObjectB();
Observable<CombinedObjectAandObjectB> resultObs = obsA.zipWith(obsB, new Func2<ObjectA, ObjectB, CombinedObjectAandObjectB>() {
#Override
public Object call(ObjectA a, ObjectB b) {
return new CombinedObjectAandObjectB(a,b);
}
});
Now when I subscribe to my resultObs I get a composite object returned with both ObjectA and Object B. This is pretty slick but still requires me to write code in Func2 that needs to be tested.
What are some ways that I can achieve my goals of combining calls for my business objects while also allowing for testability of the resultant Observable?

You almost anwser your question yourself. According to Clean Architecture it's better to pass ready-to-use models to presentation layer. Combining logic belongs to Domain layer. So use zip() in your Repository (although it's Data layer). E.g.:
Observable.zip(modelAObservable, modelBOsbervable, (A, B) -> {
combineModels(A, B);
})...
private CombinedModel combineModels(modelA, modelB) {
//combining logic
}
And test it as usual. E.g.:
ModelA A = new ModelA...
ModelB B = new ModelB...
CombinedModel expectedResult = new CombinedModel...
assertEquals(combineModels(A, B), expectedResult);

Combine Single Observable with Observable.from(array)

Given the following input:
Observable<Class1> obs = {{super complex long method}}
List<Class2> inputList = {{simple list}}
I'd like to be able to create the following:
Observable<Class3> output
that emits the result of applying method input.transform(Class1 c) to each of the inputs in the inputList.
What I've come up with so far is a combination of zip & repeat:
Observable<Class3> output = Observable.zip(obs.repeat(), Observable.from(inputList),
(class1, class2) -> class2.transform(class1));
However, the repeat is way too extreme, it emits multiple repeated items before the zip kicks in.
Another thing I tried was using combineLatest, but since my List emits first, I ended up with only the last item of the list being combined with the class1 instance.
What other combination of operators might make sense?

You can just change the parameter order, like zip(Observable.from(inputList), obs.repeat(), ...).
zip will subscribe the first Observable and then the second Observable. In your example, the first Observable is infinite (e.g., obs.repeat()), RxJava will request 128 items at first. That's why you saw obs.repeat() emitted a lot of items before subscribing the second Observable.
If changing the parameter order to Observable.from(inputList), Observable.from(inputList), RxJava will subscribe Observable.from(inputList) at first, since it's a synchronous Observable, RxJava will consume it and know its length at once (assume its length is less than 128), then RxJava will request items from the second obs.repeat() using this length. So it won't require more than the necessary items.

It sounds like what you want to do is to take each input from obs, apply a set number of functions defined in a list to each of the items in obs, then flatten that output back into an Observable of type Class3. In that case I think flatMap is a good choice, because it clearly signals the intent: that you are applying many functions per item of input, then flattening the stream.
Here's an example (pardon the Java6-ness of it):
Observable<Class3> output = obs.flatMap(new Func1<Class1, Observable<Class3>>() {
#Override
public Observable<Class3> call(final Class1 class1) {
return Observable.from(inputList).map(new Func1<Class2, Class3>() {
#Override
public Class3 call(Class2 class2) {
return class2.transform(class1);
}
});
}
});

RxJava and Cached Data

I'm still fairly new to RxJava and I'm using it in an Android application. I've read a metric ton on the subject but still feel like I'm missing something.
I have the following scenario:
I have data stored in the system which is accessed via various service connections (AIDL) and I need to retrieve data from this system (1-n number of async calls can happen). Rx has helped me a ton in simplifying this code. However, this entire process tends to take a few seconds (upwards of 5 seconds+) therefore I need to cache this data to speed up the native app.
The requirements at this point are:
Initial subscription, the cache will be empty, therefore we have to wait the required time to load. No big deal. After that the data should be cached.
Subsequent loads should pull the data from cache, but then the data should be reloaded and the disk cache should be behind the scenes.
The Problem: I have two Observables - A and B. A contains the nested Observables that pull data from the local services (tons going on here). B is much simpler. B simply contains the code to pull the data from disk cache.
Need to solve:
a) Return a cached item (if cached) and continue to re-load the disk cache.
b) Cache is empty, load the data from system, cache it and return it. Subsequent calls go back to "a".
I've had a few folks recommend a few operations such as flatmap, merge and even subjects but for some reason I'm having trouble connecting the dots.
How can I do this?

Here are a couple options on how to do this. I'll try to explain them as best I can as I go along. This is napkin-code, and I'm using Java8-style lambda syntax because I'm lazy and it's prettier. :)
A subject, like AsyncSubject, would be perfect if you could keep these as instance states in memory, although it sounds like you need to store these to disk. However, I think this approach is worth mentioning just in case you are able to. Also, it's just a nifty technique to know.
AsyncSubject is an Observable that only emits the LAST value published to it (A Subject is both an Observer and an Observable), and will only start emitting after onCompleted has been called. Thus, anything that subscribes after that complete will receive the next value.
In this case, you could have (in an application class or other singleton instance at the app level):
public class MyApplication extends Application {
private final AsyncSubject<Foo> foo = AsyncSubject.create();
/** Asynchronously gets foo and stores it in the subject. */
public void fetchFooAsync() {
// Gets the observable that does all the heavy lifting.
// It should emit one item and then complete.
FooHelper.getTheFooObservable().subscribe(foo);
}
/** Provides the foo for any consumers who need a foo. */
public Observable<Foo> getFoo() {
return foo;
}
}
Deferring the Observable. Observable.defer lets you wait to create an Observable until it is subscribed to. You can use this to allow the disk cache fetch to run in the background, and then return the cached version or, if not in cache, make the real deal.
This version assumes that your getter code, both cache fetch and non- catch creation, are blocking calls, not observables, and the defer does work in the background. For example:
public Observable<Foo> getFoo() {
Observable.defer(() -> {
if (FooHelper.isFooCached()) {
return Observable.just(FooHelper.getFooFromCacheBlocking());
}
return Observable.just(FooHelper.createNewFooBlocking());
}).subscribeOn(Schedulers.io());
}
Use concatWith and take. Here we assume our method to get the Foo from the disk cache either emits a single item and completes or else just completes without emitting, if empty.
public Observable<Foo> getFoo() {
return FooHelper.getCachedFooObservable()
.concatWith(FooHelper.getRealFooObservable())
.take(1);
}
That method should only attempt to fetch the real deal if the cached observable finished empty.
Use amb or ambWith. This is probably one the craziest solutions, but fun to point out. amb basically takes a couple (or more with the overloads) observables and waits until one of them emits an item, then it completely discards the other observable and just takes the one that won the race. The only way this would be useful is if it's possible for the computation step of creating a new Foo to be faster than fetching it from disk. In that case, you could do something like this:
public Observable<Foo> getFoo() {
return Observable.amb(
FooHelper.getCachedFooObservable(),
FooHelper.getRealFooObservable());
}
I kinda prefer Option 3. As far as actually caching it, you could have something like this at one of the entry points (preferably before we're gonna need the Foo, since as you said this is a long-running operation) Later consumers should get the cached version as long as it has finished writing. Using an AsyncSubject here may help as well, to make sure we don't trigger the work multiple times while waiting for it to be written. The consumers would only get the completed result, but again, that only works if it can be reasonably kept around in memory.
if (!FooHelper.isFooCached()) {
getFoo()
.subscribeOn(Schedulers.io())
.subscribe((foo) -> FooHelper.cacheTheFoo(foo));
}
Note that, you should either keep around a single thread scheduler meant for disk writing (and reading) and use .observeOn(foo) after .subscribeOn(...), or otherwise synchronize access to the disk cache to prevent concurrency issues.

I’ve recently published a library on Github for Android and Java, called RxCache, which meets your needs about caching data using observables.
RxCache implements two caching layers -memory and disk, and it counts with several annotations in order to configure the behaviour of every provider.
It is highly recommended to use with Retrofit for data retrieved from http calls. Using lambda expression, you can formulate expression as follows:
rxCache.getUser(retrofit.getUser(id), () -> true).flatmap(user -> user);
I hope you will find it interesting :)

Take a look at the project below. This is my personal take on things and I have used this pattern in a number of apps.
https://github.com/zsiegel/rxandroid-architecture-sample
Take a look at the PersistenceService. Rather than hitting the database (or MockService in the example project) you could simply have a local list of users that are updated with the save() method and just return that in the get().
Let me know if you have any questions.