Develop Reference

How to create Observables and update them with Retrofit later RxJava?

Basically I need to create a List of Observables without initial values. I subscribe to that list of Observables and will provide required results based on the responses from all Observables. I use zip operator.
The problem is that I need to create Observables initially, add them to the list and use zip operator. Only later I do network request with Retrofit and I need to update the value of observable in the list so my whole zip operator will be working.
However, I did not find a way to force update an observable in the list with the response from Retrofit. It seems very easy but I did not found any solutions.. only idea is to use tons of subjects and add them to the list instead...
List<Observable<Object>> listObservables = new ArrayList<>();
//Adding initial request
Observable<Object> testObservable = RetrofitFactory.create().startProcess();
listObservables.add(testObservable);
Observable.concatDelayError(listObservables).subscribe(response ->
{
//This is where all results should be managed
Log.d("response", String.valueOf(response));
},
error ->
{
Log.d("response", String.valueOf(error));
});
//Actual request occurs much later in application
listObservables.get(0).subscribeOn(Schedulers.io()).
observeOn(AndroidSchedulers.mainThread()).subscribe(response ->
{
// the response of this, should notify concatDelayError
Log.d("respoonse", String.valueOf(response));
});

If I understand correctly, you want to implement sub requests model. For this task you can break chain of operators to different execution flows and combine it back, for example, with zip operator. With this approach you can create completely independent data flow with sole trigger.
Subject<Event> eventSubject = PublishSubject.create();
Observable<TriggerObject> mainRequest = eventSubject.flatMap((event) ->
RetrofitFactory.create().startProcess());
Observable<FirstSubResult> firstSubRequest = mainRequest.flatMap(tigger -> {
// make first sub request
});
Observable<SecondSubResult> secondSubRequest = mainRequest.flatMap(tigger -> {
// make second sub request
});
Observable<CompleteResult> resultObservable = Observable.zip(firstSubRequest, secondSubRequest,
// zip function
(first, second) -> {
// combine result of sub requests to complete result object
});
Now you can start request flow by your event:
// post your event. On button clicked for evxample
eventSubject.doOnNext(yourEvent);
NOTE: this answer show main idea of chaining data flow sequences. This applicable to to other types of requests, and you can use this approach without retrofit

Create single onComplete for dynamic list of Completables

I'm using this library for wrapping Firebase transactions with RxJava. I'm new to RxJava, so this is mainly a question regarding how to use it.
Scenario: There is a many-to-many relationship between Persons and Labels. A Person can have multiple Labels, and a Label can be given to many Persons. When a Person is created, I must:
add them to the list of Persons
update each Label given to them to allow for querying all Persons that belong to a specific label
I have a list of Labels I want to write to my Firebase database.
List<Label> labels; // Let's assume it's been instantiated and added to
I want to write each of these to the DB:
FirebaseDatabase database = FirebaseDatabase.getInstance();
DatabaseReference peopleRef = database.getReference().child("people");
DatabaseReference labelsRef = database.getReference().child("labels");
int newPersonId = peopleRef.push().getKey();
I can do this easily if I don't care about whether the calls are successful.
// Let's assume I already saved the Person to the DB
for (Label label : labels){
// For each label, index the Person saved (Looks like 'personId: true')
labelsRef.child(label).child(newPersonId).setValue(true);
}
But what if I do care about the result? If I want to react to all Labels being updated (like navigate away from the current Activity), I need to know if they've all been updated successfully.
RxFirebase is implemented such that setting a value in the DB returns a Completable. I essentially want to zip together n number of Completables and do something only when they succeed or fail.
So far, I can do this if I only want to update one Label, but I want to update n Labels.
The following code snippet chains 2 Completables together, but only saves 1 Label
RxFirebaseDatabase.setValue(peopleRef.child(newPersonId), person) // Save the Person
.andThen(RxFirebaseDatabase.setValue(labelsRef.child(label).child(newPersonId), true)) // I can index 1 Label, and this returns a Completable
How would I do this? If you know Firebase well enough, is this even the right way to be saving a List of items?

If I understood your main question correctly, you have a collection of Completable and you need to subscribe to them as one.
The way to solve this is using the Completable.concat or Completable.merge operators.
Completable.concat: Returns a Completable which completes only when all sources complete, one after another.
Completable.merge: Returns a Completable instance that subscribes to all sources at once and completes only when all source Completables complete or one of them emits an error.
Example:
List<Completable> tasks; // initialized elsewhere
Completable
.concat(tasks)
.subscribe(
() -> Log.d(TAG, "All successful"),
throwable -> Log.w(TAG, "One or more failed"))
About your second question, I don't know Firebase well enough.
Update: to obtain the List<Completable> you can do something similar to this:
List<Completable> tasks = new ArrayList<>();
for ( ... ) {
tasks.add(RxFirebaseDatabase.setValue(peopleRef.child(newPersonId), person));
}
Completable.concat(tasks).etc

Android RxJava/Kotlin - map large set of data to network calls

I am trying to implement an Android app which needs to obtain a big amount of data from a backend service and save it to a db to later work on it.
The below code describes the process:
itemsService
.getAllItemIds() //This returns Single<List<Int>> from backend
.subscribeOn(Schedulers.io())
.subscribe({
Observable.fromIterable(it)
.map({
itemsService
.getItemById(it) //This gets one item details from backend
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
//Add item details to db
}, {
//Some error
})
})
}, {
//Some error
})
I obtain a list of ids and then map each of these ids to a network call to obtain the full object.
This works for a test set of, say, 10 items, but the production set contains over 50 000 ids. It works initially, saving the items, but around 5-10% it grinds to a halt and the app dies.
I assume the reason here would be that Rx keeps the reference between the source and the mapped value.
My question is: is there a way to "pool" the source emissions to, let's say, 10 at a time? Or maybe there is some other mechanism I am not aware of?

You didn't mention what exactly "grinds to a halt" means, but it makes sense that you will get out of memory in real case of 50,000 items, cause you will basically try to create 50,000 threads at once to fetch each items details.
moreover, instead of chaining Observables using operators, you're creating nested chains at subscribe/map, you can read here why you shouldn't.
regarding limiting the work to 10 at a time, there is an flatMap overload for that, at the end it might look something like this:
itemsService
.getAllItemIds() //This returns List<Int> from backend
.flatMapIterable { t -> t }
.flatMap({
itemsService
.getItemById(it) //This gets one item details from backend
.subscribeOn(Schedulers.io())
}, 10) //limit flat map parallelism by desired value
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe({
//Add item details to db
}, {
//Some error
})

Rxjava Filtering the duplicate item in List

I am using RxJava and Retrofit in My App.Here's the setup of the app.When the app is launched the app make two request one to the database and other to the Network Api (using Retrofit) and both request return a Observable<List<Article>>. So what I did is basically merged the two Observable. Now the problem is sometimes the network return Articles that are already present in the Database. So how do I filter out the duplicate item from the List. Here's my Code.
return Observable.merge(dataSource.getArticles(source), remoteSource.getArticles(source))
.distinct();
So I tried distinct operator but it's not filtering the Articles out.Here's the output looks like form db.
Article1
Article2
Article3
Article4
Output from Network
Article7
Articke8
Article3
Article4
What I want is a distinct list of Article

Assuming your Article has proper equals implementation,
you could collect them into a set:
dataSource.getArticles(source)
.mergeWith(remoteSource.getArticles(source))
.collect(HashSet::new, (set, list) -> set.addAll(list))
or you could unroll each list and apply distinct followed by toList:
dataSource.getArticles(source)
.mergeWith(remoteSource.getArticles(source))
.flatMapIterable(v -> v)
.distinct()
.toList()

That's because they are returning different lists. So the distinct method recognize them as different items
If you want to emit first the database items and then add the server ones... This may be a bit more complex but not too much ;)
Observable<List<Article>> databaseArticles = ...
Observable<List<Article>> serverArticles = ...
Observable<List<Article>> allArticles =
Observable.combineLatest(
databaseArticles,
serverArticles
.startWith(emptyList()), // so it doesn't have to wait until server response
(dbItems, sItems) => {
// Combine both lists without duplicates
// e.g.
Set<Article> all = new HashSet<>();
Collections.addAll(all, dbItems);
Collections.addAll(all, sItems);
return new ArrayList<>(all);
});

rxjava + retrofit streaming one by one a long list of data

I need to get list of users:
API Endpoint:
Observable<List<User>> getUsers();
onNext(List<User> usersList)
I do not want to wait for the download of entire list.
I want to get user one by one.
Like this:
onNext(TempData singleUser)
how can I do that?

If you just want to convert Observable<List<User>> to Observable<User>, you can use getUsers().flatMapIterable(l -> l).
If you want to downloads and process the user list one by one, you need to change your API, since Observable<List<User>> getUsers() won't emit anything until the whole list is downloaded.