I'm saving the user's location in the app local database and then send it to the server. Once the server return a success, I delete the location that was sent.
Each time a point has been saved in the database I call this method:
public void sendPoint(){
amazonRetrofit.postAmazonPoints(databaseHelper.getPoints())
.map(listIdsSent -> deleteDatabasePoints(listIdsSent))
.doOnCompleted(() -> emitStoreChange(finalEvent))
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(AndroidSchedulers.from(backgroundLooper))
.subscribe();
}
I query the database for the point to be send to the server
I received from the server the list of point successfully sent
Using .map(), I gather the point successfully sent and delete them from the local database
Sometimes, It happens that I call this method repeatedly without having wait for the previous request to be completed and deleted the point sent. So, when I call that method again, it will post the same point as the previous request because that previous request isn't completed yet thus haven't deleted the point using the .map() yet. Causing the server to receive duplicates...
Timeline
1st Call to postPoint()
Retrive point A,B,C from the database
Post point A,B,C to the server
2nd call to postPoint()
Retrive point A,B,C,D from the database
Post point A,B,C,D to the server
Receive success from the 1st request
Deleting A,B,C from the local database
Receive success from the 2nd request
Deleting A,B,C,D from the local database
Result:
The server database now have received : A,B,C,A,B,C,D
Each request occurs sequentially but somehow the same location points are sent to the server when I call sendPoint() too quickly. How can I fix this?
First to all you are not using observerOn operator properly, observeOn operator is applied over the steps in your pipeline, once is defined.
So if you define at the end of the pipeline just before subscribeOn, then none of your previous steps will be executed in that thread.
Also, since you need to wait until the response of your server call, you can use the callbacks handlers that Subscriber already provide (onNext(), onComplete())
public void sendPoint(){
Observable.from(databaseHelper.getPoints())
.observeOn(AndroidSchedulers.mainThread())
.flatMap(poins-> amazonRetrofit.postAmazonPoints(points))
.subscribeOn(AndroidSchedulers.from(backgroundLooper))
.subscribe(listIdsSent-> deleteDatabasePoints(listIdsSent), () -> emitStoreChange(finalEvent));
}
if you want to see more examples of ObserverOn and SubscribeOn you can take a look here. https://github.com/politrons/reactive/blob/master/src/test/java/rx/observables/scheduler/ObservableAsynchronous.java
You should have some kind of validation on the client side or/and on the backend side.
Client side:
The simplest solution is to add two columns to the table with locations like "processing" and "uploaded".
When you select locations from database and clausure where processing=false and uploaded=false.
Then when you have rows ready to sent set processing=true and when the server returns success set done=true.
Backend side (optional, depends on requirements):
You should send location with timestamp to the server (probably one more additional column in your client side table). If the server gets a location with timestamp older than the last one in a database it shouldn't store it.
RxJava solution:
You can implement a similar solution with memory cache which is kept around all sendPoint as List.
Pseudocode:
public void sendPoint(){
databaseHelper.getPoints()
.filter(points -> pointsNotInCache())
.map(points -> amazonRetrofit.postAmazonPoints())
.map(points -> addToCache())
.map(listIdsSent -> deleteDatabasePoints(listIdsSent))
.map(listIdsSent -> removeSentPointsFromCache()) //if you would like save memory
.doOnCompleted(() -> emitStoreChange(finalEvent))
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(AndroidSchedulers.from(backgroundLooper))
.subscribe();
}
It looks like, as everyone else is saying, you need an intermediate cache.
i.e.
HashSet<Point> mHashSet = new HashSet<>();
public void sendPoint() {
Observable.from(databaseHelper.getPoints())
.filter(point -> !mHashSet.contains(point))
.doOnNext(mHashSet::put)
.toList()
.flatMap(amazonRetrofit::postAmazonPoints)
.map(this::deleteDatabasePoints)
.doOnCompleted(() -> emitStoreChange(finalEvent))
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(AndroidSchedulers.from(backgroundLooper))
.subscribe();
}
Related
For the first time I want to retrieve data from server cache it and next times show data on UI from local storage and request from server and update local storage and UI as
I have tried
(getCachedData()).concatWith(getRemoteData())
getCachedData returns Single
return apiSeResource.getData()
.doAfterSuccess { response ->
saveData(response.body())
}
}
.onErrorReturn {
return#onErrorReturn emptyList()
}
}```
The problem with `concat` is that the subsequent observable doesn't even start until the first Observable completes. That can be a problem. We want all observables to start simultaneously but produce the results in a way we expect.
I can use `concatEager` : It starts both observables but buffers the result from the latter one until the former Observable finishes.
Sometimes though, I just want to start showing the results immediately.
I don't necessarily want to "wait" on any Observable. In these situations, we could use the `merge` operator.
However the problem with merge is: if for some strange reason an item is emitted by the cache or slower observable after the newer/fresher observable, it will overwrite the newer content.
So none of mentioned above solution is not proper ,what is your solution?
Create 2 data sources one local data source and one remote and use the flatMap for running the Obervables. You can publish the data from the cache and when u get data from remote save data to cache and publish.
Or you can also try Observable.merge(dataRequestOne, dataRequestTwo) . run both the Observables on different threads
I am investigating the use of RxJava in my latest Android application.
I have a two lists of related updated Database model objects
ListDB1 and ListDB2
the logic I am attempting to implement is as follows
1). For each item in ListDB1
1.1). Transform it to a Network model object
1.2). Execute an Update RESTful API
2). Once all network updates have completed successfully
2.1). Persist ListDB1 to my local database.
2.2). Persist ListDB2 to my local database.
So far I have this code which should call my network API's
Observable.just(getDatabaseList())
.subscribeOn(Schedulers.io())
.flatMapIterable(x -> x)
.flatMap(database -> transformDatabase(database, DB_1_MAPPER))
.doOnNext(NetworkController::updateRemote)
.observeOn(AndroidSchedulers.mainThread())
.doOnComplete(getOnComplete())
.doOnError(getOnError())
.subscribe();
No API calls are executed though
I would rather use Single that Observable as my API calls respond with Single<Response>, however I couldnt see how to achieve Observable.just(<>) with a Single.
Also I cannot see how to commence my Rx process by processing each List item separately for the Network calls, then performing Database calls with a complete list as I am using Realm as my local database which can accept lists of database objects in a single update.
In pseudo code my process resembles:
for each database list item
convert to network model item
call remote update API
When all all network calls are successful
update database with ListDB1
update database with ListDB2
end
Is this possible in one Rx process "stream"?
1.
I would rather use Single that Observable as my API calls respond with Single, however I couldnt see how to achieve Observable.just(<>) with a Single.
You can do this:
Single.just(getDatabaseList()) // Single<>
However, Single is not suitable in this case because you are not working with a single item, and what you need instead is to iterate through multiple items and work on the items one by one.
2.
processing each List item separately for the Network calls, then performing Database calls with a complete list
You can use toList() operator which emits entire list of items when the observable completes.
3.
The purpose of do operators such as doOnNext, doOnComplete, and doOnError is to create side effect that does not affect the stream. An example of this kind of operations is logging. You should not do any meaningful operation that affect the stream in such operators.
Instead you should be using operators such as map, flatMap, etc.
4.
Putting everything together:
Observable.fromIterable(getDatabaseList())
.subscribeOn(Schedulers.io())
.flatMap(database -> transformDatabase(database, DB_1_MAPPER))
.flatMap(NetworkController::updateRemote)
.toList() // This has type of Single<List<>>
.flatMap(list -> {
// Update db1 and db2 with the result of server update.
return Single.zip(updateDb1(list), updateDb2(list), (a, b) -> {
// Combine result of db updates
});
})
.observeOn(AndroidSchedulers.mainThread())
.subscribe(result -> {
// Handle success case
}, err -> {
// Handle error case
});
My problem is i want to load the data in an activity from server using an observable , when i subscribe to it in onCreate() i automatically get the data from the server and also there is an swipe to refresh when an user swipe , i want same observable to retrieve data from server again,
questions
1) is it a good idea to create an subscription again and again when a user swipes to refresh since there can more than 10 subscription if user swipes 10 times
2) if the above is not a good idea , what to do to make observable fetch data from server without creating a new subscription or subscriber
Usual practice - Single for server request, instead of Observable
If you want to use just a single object - you can do something like:
var networkDisposable: Disposable? = null
fun doRequest() {
networkDisposable?.dispose()
networkDisposable = yourNetworkRequest().subscribe()
}
I have an API that returns FamilyDetails. I use retrofit to fetch them, and it is modelled as follows:
Observable<FamilyDetails> getFamilyDetails(#Query int personId);
Once fetched, those details will be used to update a Person. My API delivers those FamilyDetails as soon as it has some information to show, which means that the details may be incomplete, and thus there is a complete = false flag delivered in responses when the server hasn't finished fetching all family details - in turn, my app should send another request to fetch the remaining FamilyDetails if we receive complete = false in the first response, and keep doing so for as long as the details remain incomplete.
I have "somewhat" achieved what I wanted, but not entirely with this code:
personApiService.getFamilyDetails(personId)
// repeat call
.flatMap(familyDetails -> personApiService.getFamilyDetails(personId))
// until family details are complete
.takeUntil(familyDetails -> familyDetails.isComplete())
// update the person object with the family details
.flatMap(familyDetails -> Observable.just(updatePerson(familyDetails))
//subscribe!
.subscribe(personSubscriber);
My personSubscriber returns an updated Person.
The problem I have with this implementation is that partial updates don't go through personSubscriber's onNext(), as I only get one call to onNext() with the updated Person object with its complete FamilyDetails.
I would like to model this using RxJava with the following requirements:
if I receive incomplete details, I update the Person
object those details belong to, and deliver it via onNext().
If we receive incomplete details, we keep querying the API for complete details, and deliver updated Person objects via onNext() in the same observer.
Once we get the last onNext() call with the updated Person, then onComplete() is called and we are done.
Thanks in advance!
personApiService.getFamilyDetails(personId)
// repeat call
.repeat()
// until family details are complete
.takeUntil(familyDetails -> familyDetails.isComplete())
//subscribe (do your updates here)
.subscribe(personSubscriber);
In my Android app I am using domain level Repository interface, which is backed with local DB implemented using SqlBrite and network api with Retrofit observables. So I have method getDomains(): Observable<List<Domain>> in Repository and two corresponding methods in my Retrofit and SqlBrite.
I don't want to concatenate or merge, or amb these two observables. I want my Repository to take data only from SqlBrite and since SqlBrite returns QueryObservable, which triggers onNext() every time underlying data changed, I can run my network request independently and store results to SqlBrite and have my Observable updated with fetched from network and stored to DB data.
So I tried to implement my Repository's getDomains() method as follow:
fun getDomains(): Observable<List<Domain>> {
return db.getDomains()
.doOnSubscribe {
networkClient.getDomains()
.doOnNext { db.putDomains(it) }
.onErrorReturn{ emptyList() }
.subscribe()
}
}
But in this case every time the client should subscribe, every time it would make network requests, that is not so good. I thought about other do... operators to move requests there, but doOnCompleted() in case of QueryObservable would never be called, until I call toBlocking() somewhere, which I won't, doOnEach() also not good as it makes requests every time item from db extracted.
I also tried to use replay() operator, but though the Observable cached in this case, the subscription happens and results in network requests.
So, how can combine these two Observables in the desired way?
Ok, it depends on the concrete use case you have: i.e. assuming you want to display the latest data from your local database and from time to time update the database by doing a network request in the background.
Maybe there is a better way, but maybe you could do something like this
fun <T> createDataAwareObservable(databaseQuery: Observable<T>): Observable<T> =
stateDeterminer.getState().flatMap {
when (it) {
State.UP_TO_DATE -> databaseQuery // Nothing to do, data is up to date so observable can be returned directly
State.NO_DATA ->
networkClient.getDomains() // no data so first do the network call
.flatMap { db.save(it) } // save network call result in database
.flatMap { databaseQuery } // continue with original observable
State.SYNC_IN_BACKGROUND -> {
// Execute sync in background
networkClient.getDomains()
.flatMap { db.save(it) }
.observeOn(backgroundSyncScheduler)
.subscribeOn(backgroundSyncScheduler)
.subscribe({}, { Timber.e(it, "Error when starting background sync") }, {})
// Continue with original observable in parallel, network call will then update database and thanks to sqlbrite databaseQuery will be update automatically
databaseQuery
}
}
}
So at the end you create your SQLBrite Observable (QueryObservable) and pass it into the createDataAwareObservable() function. Than it will ensure that it loads the data from network if no data is here, otherwise it will check if the data should be updated in background (will save it into database, which then will update the SQLBrite QueryObservable automatically) or if the data is up to date.
Basically you can use it like this:
createDataAwareObservable( db.getAllDomains() ).subscribe(...)
So for you as user of this createDataAwareObservable() you always get the same type Observable<T> back as you pass in as parameter. So essentially it seems that you were always subscribing to db.getAllDomains() ...
if your problem is that you have to subscribe your observer every time that you want to get data you can use relay, which never unsubscribe the observers because does not implement onComplete
/**
* Relay is just an observable which subscribe an observer, but it wont unsubscribe once emit the items. So the pipeline keep open
* It should return 1,2,3,4,5 for first observer and just 3, 4, 5 fot the second observer since default relay emit last emitted item,
* and all the next items passed to the pipeline.
*/
#Test
public void testRelay() throws InterruptedException {
BehaviorRelay<String> relay = BehaviorRelay.create("default");
relay.subscribe(result -> System.out.println("Observer1:" + result));
relay.call("1");
relay.call("2");
relay.call("3");
relay.subscribe(result -> System.out.println("Observer2:" + result));
relay.call("4");
relay.call("5");
}
Another examples here https://github.com/politrons/reactive/blob/master/src/test/java/rx/relay/Relay.java