Develop Reference

Rxjava2 & Retrofit - multiple calls return as list

API calls should be made ~30 times which only differ by one parameter:
https://api.website.com/getProducts?category_id=10
https://api.website.com/getProducts?category_id=11
These calls responds are limited by 100 products per call. If a category has more products, I need to append a offset parameter. The following call will give me the subset from 101-200. A total is also provided in the response so I know when to stop.
https://api.website.com/getProducts?category_id=10&offset=100
https://api.website.com/getProducts?category_id=10&offset=200 <--category has 260 products, I stop here.
I can make the initial calls (no offset) and n-offset calls with Retrofit easily. But from Retrofit I can't register any callbacks when all calls are finished nor return the data as a list to update my UI just once.
So I like to try it with RxJava2. I accomplished the same as with Retrofit already (returning a Observable after each parent call or call with offset).
private Observable<SearchResponse> search(int category, int offset) {
Observable<SearchResponse> call = retrofitSearchService.search(category, offset);
return call.flatMap(new Function<SearchResponse, ObservableSource<?>>() {
#Override
public ObservableSource<SearchResponse> apply(#NonNull SearchResponse searchResponse) throws Exception {
if (searchResponse.getTotal() > (searchResponse.getOffset() + searchResponse.getLimit())) {
search(category, searchResponse.getOffset() + 100); <--recursion here
}
return Observable.just(searchResponse);
}
}).toList().toObservable()
.cast(SearchResponse.class);
}
Returning every response as one makes my UI update like crazy (Android livedata).
I still like to:
return a full list of all categories and children if they have any.
get one callback when all requests are made.
This looks promising (How To Do Recursive Observable Call in RxJava?). But I can't wrap my head around it.
If you can, please disperse of lambda. Function, Consumer, Subscriber gives me more clarity. Great!

Android PagedList Scroll to bottom

I'm using Android Jetpack Paging library(2.1.2) for a chat view.
Data is served from a local database(room) and network by using PagedList.BoundaryCallback mechanism. Since the Room provide paged data to the RecyclerView adapter I'm unable to scroll to the bottom of list as the list size is unknown.
I also registerd registerAdapterDataObserver for adapter to observe the changes. However
itemCount value in onItemRangeInserted callback changed from time to time.
So how do you guys scroll to bottom of the RecyclerView when you are using PagedListAdapter with BoundaryCallback ?

For this we need to extend the positional datasource, and override loadInitial() method. Basically room uses the limit and offset in sqlite to load specific range of data from the specific position. Codes are in Kotlin but can be easily understandable and convertable in Java
override fun loadInitial(
params: LoadInitialParams,
callback: LoadInitialCallback<ChatListItem>
) {
val newRequestedSize = params.requestedStartPosition + params.requestedLoadSize
val dbMessage = groupMessagesDao.getPagedMessageForGroup(
channelId,
0,
newRequestedSize
)
if (dbMessage.isNotEmpty()) {
callback.onResult(dbMessage, 0)
} else {
callback.onResult(emptyList(), params.requestedStartPosition)
}
}
In the above code we are forcing the room(you will have to implement your own Dao of course, it just uses the offset and limit as I mentioned earlier) to load the initial data from position 0 to the newRequestedSize which is the sum of current offset and load size the paging library wants us to load. Once, we load the data from position 0 to the current page position, we need to set the callback with the data and the position which is 0. Now, the paging library loads the data up to the current page position and our
private fun scrollToBottom() {
val position = adapter.itemCount - 1
if (position > 0) {
recyclerView.scrollToPosition(0)
}
}
function works as expected.
One thing we also need to handle is to listen for table change, which you can again find in the LimitOffsetDatasource on how the room does it.
init {
val tableObserver = object : InvalidationTracker.Observer(tableName) {
override fun onInvalidated(tables: MutableSet<String>) {
// invalidate the dataSource
Log.v("dataSource invalidate requested!")
invalidate()
}
}
database.invalidationTracker.addObserver(tableObserver)
addInvalidatedCallback {
Log.v("invalidation callback received, removing table observer")
database.invalidationTracker.removeObserver(tableObserver)
}
}
The performance implication are there in this approach, but in our case the user is most of the time messaging which is always at the first page. So, when the user is at bottom this implementation is same as that of the LimitOffsetDatasource. Only when the user scrolls to see the old messages, and at the same time a new message arrives for the user only then we reload all the messages up to the user current position. But when new message arrives most probably the user will scroll to bottom. So, the performance penalty is virtually zero.

Why is Parse Server saveEventually with callback taking so long?

I'm using Parse Server for my Android app and everything is working fine, but every time I call saveEventually on a new or old ParseObject, it is taking a really long time. Sometimes it's more than 1 minute for 1 item to return the callback.
Anyone had this problem?
Example:
orderObject.p.apply {
put(ORDER_STATE, ORDER_STATE_FINISHED)
put(ORDER_NEXT_DATE, orderEndDate)
}
createLog("FinishOrderSeq", "OrderActivity - saveOrder - before saveEvent")
orderObject.p.saveEventuallyEx(isOnline(this)){ e ->
createLog("FinishOrderSeq", "OrderActivity - saveOrder - after saveEvent")
if (e == null){
createToast(getString(R.string.order_dialog_success), this)
createOrderCopy(orderObject, dialog)
} else {
createToast(getString(R.string.order_dialog_err), this)
changeButtonState(posBtn, true)
changeButtonState(negBtn, true)
}
}
fun ParseObject.saveEventuallyEx(isOnline: Boolean, callback: (ParseException?) -> Unit){
if (isOnline){
saveEventually{ err ->
callback(err)
}
} else {
saveEventually()
callback(null)
}
}
Also logs as I replaced it with saveInBackground with callback(still 30 seconds):
2020-05-28 14:53:49.805 18673-18673/? I/FinishOrderSeq: OrderActivity - saveOrder - before saveEvent
2020-05-28 14:54:15.694 18673-18673/? I/FinishOrderSeq: OrderActivity - saveOrder - after saveEvent
UPDATE:
So I figured out from parse dashboard, that ParseObject is saved as record in table immediatelly, but callback from saveEventually is sent after 30sec - 2 minutes.
UPDATE 2:
I also tried to use saveInBackground() if user is online (with callback). This also took 30seconds to 2 minutes for callback to return. Object was saved to parse database with all data after 100ms (checked from Parse Dashboard).
Then I thought something is wrong with ParseSDK threads, so I used save() inside Coroutine. Same problem occured here, save() took up to 2 minutes to perform.
Code with coroutine:
fun ParseObject.saveAsync(context: CoroutineContext, scope: CoroutineScope, isOnline: Boolean, callback: (ParseException?) -> Unit){
if (isOnline){
scope.launch {
var ex: ParseException? = null
try {
save()
} catch (e: ParseException){
ex = e
}
withContext(context){
callback(ex)
}
}
}
}
There is some serious problem with callbacks in ParseSDK for Android and I don't know what can cause this. No exception no error on server side.
UPDATE 3:
After deeper investigation, I found which function is taking long time to proceed.
ParseObject.State result = saveTask.getResult();
Approximately 30 seconds - 2 minutes to get into next line of code.
This is lowest level of function I can get inside SDK.
Inside function save() or saveInBackground() there is this inner function in Java:
Task<Void> saveAsync(final String sessionToken, final Task<Void> toAwait) {
if (!isDirty()) {
return Task.forResult(null);
}
final ParseOperationSet operations;
synchronized (mutex) {
updateBeforeSave();
validateSave();
operations = startSave();
}
Task<Void> task;
synchronized (mutex) {
// Recursively save children
/*
* TODO(klimt): Why is this estimatedData and not... I mean, what if a child is
* removed after save is called, but before the unresolved user gets resolved? It
* won't get saved.
*/
task = deepSaveAsync(estimatedData, sessionToken);
}
return task.onSuccessTask(
TaskQueue.<Void>waitFor(toAwait)
).onSuccessTask(new Continuation<Void, Task<ParseObject.State>>() {
#Override
public Task<ParseObject.State> then(Task<Void> task) {
final Map<String, ParseObject> fetchedObjects = collectFetchedObjects();
ParseDecoder decoder = new KnownParseObjectDecoder(fetchedObjects);
return getObjectController().saveAsync(getState(), operations, sessionToken, decoder);
}
}).continueWithTask(new Continuation<ParseObject.State, Task<Void>>() {
#Override
public Task<Void> then(final Task<ParseObject.State> saveTask) {
ParseObject.State result = saveTask.getResult(); <--- THIS IS TAKING LONG TIME
return handleSaveResultAsync(result, operations).continueWithTask(new Continuation<Void, Task<Void>>() {
#Override
public Task<Void> then(Task<Void> task) {
if (task.isFaulted() || task.isCancelled()) {
return task;
}
// We still want to propagate saveTask errors
return saveTask.makeVoid();
}
});
}
});
}

From the docs:
Most save functions execute immediately, and inform your app when the save is complete. If you don’t need to know when the save has finished, you can use saveEventually instead.
It can take a long time because with saveEventually you are basically saying "save it soon". If you want to "save it as soon a possible" then use saveInBackground as described in the docs.
Further it says:
All calls to saveEventually (and deleteEventually) are executed in the order they are called, so it is safe to call saveEventually on an object multiple times. If you have the local datastore enabled, then any object you saveEventually will be pinned as long as that save is in progress. That makes it easy to retrieve your local changes while waiting for the network to be available.
Which means that you can save and modify the object locally multiple times and the latest version will be stored in the database as soon as the network connection is reestablished.

Single.concat report error and continue to the next

I'm trying to use RxJava with Android to asynchronously update my view. When user clicks the movie from the list in the RecyclerView, I want to present him first with the movie from the database, if present. Then I want to fetch the latest information and update the database as well as UI. I'm trying to use concat method and its variant but it does not work.
I have skipped other codes only to post the relevant RxJava methods that are fetching data as the rest is working fine.
When I disable network connection with the code below (hence remote returns error), the code below does not display data from the database at all. Only it reports the error. Which means the local is not resolving.
public Flowable<Movie> getMovie(final int id) {
return Single.concat(mLocal.getMovie(id), mRemote.getMovie(id).doOnSuccess(data -> {
mLocal.save(data);
})).onErrorResumeNext(error->{
return Flowable.error(error);
});
}
And in this code, it works fine, except now that I don't get the error message (and rightly so, since I have replaced it with new stream from the database)
public Flowable<Movie> getMovie(final int id) {
return Single.concat(mLocal.getMovie(id), mRemote.getMovie(id).doOnSuccess(data -> {
mLocal.save(data);
})).onErrorResumeNext(error->{
return mLocal.getMovie(id).toFlowable();
});
}
Now, how can I get database data first and then fire network call next to update data and get errors from the database or network call?
UPDATE
The latest method code
// calling getMovie on mLocal or mRemote returns Single
public Flowable<Movie> getMovie(final int id) {
return Single.concat(mLocal.getMovie(id), mRemote.getMovie(id).doOnSuccess(data -> {
mLocal.insertMovie(data);
})).onErrorResumeNext(error -> {
return Flowable.error(error);
});
}
Here is how I call them
public void loadMovie(int id)
{
Disposable d = mRepo.getMovie(id)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread(), true)
.doOnSubscribe(subscription -> {
subscription.request(Long.MAX_VALUE);
//post progress here
})
.subscribe(data -> {
//onNext
},
error -> {
//onError
},
() -> {
//onComplete
}
);
mDisposables.add(d);
}

With affirmation that my code works and guides on troubleshooting from #akarnokd I found the latest code (see OP) works flawlessly. The result of RxJava chain is posted to LiveData object which should update View. Unfortunately it only posts the latest data (which is an error) and skips the first (which is the data from the database).
I will deal with that but since the post deals with RxJava, I will consider this solved!

How to manage parallel and serial Retrofit API calls

I have 4 API calls in the same activity. 3 of them are independent of each other.I would like to call number 4 after first three finished and I am not sure the execution of first 3 in every time. I get data from database then it will call. It may 1 API call or 2 or 3 among first three.
I tried to call one after another sequentially but sometimes number 4 starts before first 3 finished. some of my efforts given below:
if(true){ // data 1 is available in database
firstRetrofitCall();
}else{
//show no data
}
if(true){ // data 2 is available in database
secondRetrofitCall();
}else{
//show no data
}
if(true){ // data 3 is available in database
thirdRetrofitCall();
}else{
//show no data
}
fourthRetrofitCall(); // I would like to execute this after first three finished
is it possible to manage using RxJava?

Use Rxjava2 adapter with Retrofit and then you can use Rxjava's zip operator to combine first three calls like this(assuming your calls return X,Y,Z values respectively and XYZwrapper is just container for these) and then flatMap operator to do the fourth call.
Single.zip(
firstRetrofitCall(),
secondRetrofitCall(),
thirdRetrofitCall(),
Function3<X, Y, Z, XYZwrapper> { x, y, z -> return#Function3 XYZwrapper(x, y, z) }
)
.subscribeOn(Schedulers.io())
.flatMap { XYZwrapper -> fourthRetrofitCall().subscribe() }//chaining
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy( onError = {}, onSuccess = {})

Declare a Boolean array of size 3 and initialize its indexes to false. Update the index to true in each 1st three API call's onResponse method. For example set index 0 to true for API call 1 and so on. And check in onResponse method that each of the array indexes are true if true then call the fourth API.

Add a boolean flag for each of those calls
boolean isFirstExecuted;
boolean isSecondExecuted;
boolean isThirdExecuted;
if(true){ // data 1 is available in database
firstRetrofitCall();
}else{
isFirstExecuted = true;
}
if(true){ // data 2 is available in database
secondRetrofitCall();
}else{
isSecondExecuted = true;
}
if(true){ // data 3 is available in database
thirdRetrofitCall();
}else{
isThirdExecuted = true;
}
checkAndExceuteFourth();
onFirstResponse(){
isFirstExecuted = true;
checkAndExceuteFourth();
}
onSecondResponse(){
isSecondExecuted = true;
checkAndExceuteFourth();
}
onThirdResponse(){
isThirdExecuted = true;
checkAndExceuteFourth();
}
Method for checking and executing fourth
public void checkAndExceuteFourth(){
if(isFirstExecuted && isFirstExecuted && isFirstExecuted ){
fourthRetrofitCall();
}
}