Develop Reference

RxJava2 Observable.zip(list) executes Network calls twice

I'm getting an unwanted behavior/flaw when passing a list of Observable Network calls to Observable.Zip(), similar to this accepted answer:
How to make multiple request and wait until data is come from all the requests in retrofit 2.0 - android
And the unwanted behavior is, the network calls are being fired twice...
It's fired once when the Observable is added to the List, and then it's fired again during the Observable.zip()
Here's a boiled down snippet from my project that's reproducing the behavior:
fun buildListOfObservableNetworkCalls(): Observable<Map<String, String?>> {
val clients = mutableListOf<Observable<NetworkResponse>>()
if (NetworkClient1.featureFlag) {
val postBody = someMethodToBuildPostBody()
clients.add(NetworkClient1().executeClient(postBody))
}
//There will be multiple NetworkClients in the near future
return executeAllNetworkClients(headerBidClients)
}
private fun executeAllNetworkClients(clients: List<Observable<NetworkResponse>>): Observable<Map<String, String?>> =
if (clients.isNotEmpty()) {
Observable.zip(clients) {
it
}
.map { clientResults ->
clientResults.forEach { response ->
if (response is NetworkResponse) {
map[MY_KEY] += response.stringResult
}
}
map
}.doOnSubscribe {
android.util.Log.d("LOGGER", "zip ON SUBSCRIBE")
}
} else {
Observable.just(mapOf())
}
//**** My NetworkClient1 class containing the RxJava function that executes the network call ****//
override fun executeClient(postBody: CustomPostBody): Observable<NetworkResponse> =
retrofitApiInterface.networkCall1Request(postBody)
.subscribeOn(Schedulers.io())
.doOnSuccess { response ->
Log.d("LOGGER", "Client1 ON SUCCESS")
}
.flatMapObservable { response ->
Observable.just(
NetworkResponse(
response
)
)
}.onErrorResumeNext { throwable: Throwable? ->
android.util.Log.d("LOGGER", "Client1 ON ERROR")
Observable.just(
NetworkResponse(
""
)
)
}.doOnSubscribe {
android.util.Log.d("LOGGER", "Client1 ON SUBSCRIBE")
}
//***** And my custom Interceptor which logs the double requests ****////
class MyInterceptor: Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
Log.d("LOGGER", "intercept request at ${System.currentTimeMillis()}")
val response = chain.proceed(request)
Log.d("LOGGER", "intercept response at ${System.currentTimeMillis()}")
return response
}
}
And the log output is:
D/LOGGER: zip ON SUBSCRIBE
D/LOGGER: Client1 ON SUBSCRIBE
D/LOGGER: intercept request at 1650059924358
D/LOGGER: intercept response at 1650059925747
D/LOGGER: Client1 ON SUCCESS
D/LOGGER: intercept request at 1650059925782
D/LOGGER: intercept response at 1650059925928
As you can see, the same network call is being executed twice.. and secondly, i'm also a bit puzzled as to why doOnSuccess isnt also called twice.
So, my main questions is, is there a way that I can build a list of Observables and pass it to Observable.zip() without executing the network call twice?
I see the issue is that I'm creating a List<Observable<NetworkResponse>> and in order to add network calls that return <Observable<NetworkResponse>>, I have to invoke the method as i'm adding them to the list. I know this may sound like a dumb question.. but is it at all possible to have a set-up where i'm able to add the Observable functions to the List without executing them? Probably over doing it, but would creating an extension function of .zip(iterable) which accepts a list of NetworkClients as the sources param and within the extension function, execute source.executeClient() be a feasible or stable solution?
I feel it would be inefficient if this was the unavoidable consequence of building a list of Observables to pass to zip(iterable), so i'm hoping that this is just a matter of my set-up rather than an overlooked consequence of the .zip(iterable) method.
I'm aware that I could avoid the above scenario by trying to pass each Observable Network call individually into the .zip() and use some sort of BiFunction to tie it all together. However, that doesn't seem very intuitive for my use case, being that I have to featureFlag check and build Post objects for each request that i'll be doing. Additionally, I'll be adding more NetWorkClients who's responses will all be returning the same base response type over the next few months, so I find the .zip(iterable) methodology as a clean and very scalable way of plugging in new NetworkClients.

How do I pause execution in Kotlin whilst APIs complete

I am trying to write a simple app in Android Studio using Kotlin. It is a very steep learning curve for me, but I am almost there. My final problem is getting the app to wait for the APIs to complete before moving the next Intent.
I have three calls each uploading data via my API. They are called from a button and only when the three are uploaded, should the button send the user to the next intent/screen.
My API calls are working and I can see the data in the database. However, since enqueue is asynchronous the calls are firing and the code is moving on the start the next intent before the data is present.
The code below is executed 3 times (once for each upload). I realise this is probably not the best way to do it, but I'm trying to get it working before I finesse the code.
I thought that perhaps I could have a variable, UploadedReadCount, that I increment in the onResponse, but this doesn't seem to be working properly.
Could someone offer some advice as to how I should be pausing the code until the APIs complete? For example, is there an enqueue methos that isn't async?
ReadInterface.create().AddRead("new", rFuel, rRegister, rReadDate, rRead)
.enqueue(object : Callback<UploadedRead> {
override fun onFailure(call: Call<UploadedRead>, t: Throwable) {
Log.d("Err: ", t.localizedMessage!!)
t.printStackTrace()
}
override fun onResponse(call: Call<UploadedRead>, response: Response<UploadedRead>) {
Log.d("Response: ", response.body().toString())
val p = response.body()?.APIResult!![0]
msgShow("Gas read " + rRead.toString() + " uploaded")
UploadedReadCount += 1
}
})
while ( UploadedReadCount < 3) {
Log.d("Waiting ", UploadedReadCount.toString() + " reads uploaded...")
}
val intent = Intent(this, Billing::class.java).apply {
putExtra("ReadDate", txtReadDate.text.toString())
}
startActivity(intent)

In most cases you don't want to pause execution while API call returns, Instead you want to follow the reactive model, that is when you call API you specify some callbacks (onResponse, onFailure), and once these callbacks are invoked then you react.
code is moving on the start the next intent before the data is
present.
Move all of your code that depends on data received from API in onResponse or onFailure methods (callbacks), When API is ready with some response one of those callbacks will be invoked and then depending on the data that you receive from API you can continue your work.
is there an enqueue methos that isn't async?
There are options available to call an API in blocking manner but I don't think that is good idea. Instead of doing a blocking API call, you should try to do reactive programming that is as soon as any callback (onResponse, onFailure) is called only then you continue.

There is an alternative to enqueue that is suspending instead of async, so you can call your code sequentially without blocking the main thread in a coroutine. The function is await() and it returns the successful result or throws an HttpException on failure.
But to run three requests in parallel, you need to use the async coroutine builder. This can be done by mapping a list of Calls to async calls that await the individual results, and then using awaitAll() on the list of Deferreds to wait for all three. So, it's more complicated than just running sequential code in a coroutine, but I think this is still easier than trying to run and wait for three parallel calls using callbacks.
I'm not exactly sure what your other two calls are so I'll just make up some and assume this function already has all the data it needs to make the calls. I also don't know how you want to handle failure, so I'm just making it stop early if any of the three calls fail.
lifecycleScope.launch {
val requests: List<Call<UploadedRead>> = listOf(
ReadInterface.create().AddRead("new", rFuel, rRegister, rReadDate, rRead),
ReadInterface.create().AddRead("new2", rFuel, rRegister, rReadDate, rRead),
ReadInterface.create().AddRead("new3", rFuel, rRegister, rReadDate, rRead)
)
val responses: List<UploadedRead> = try {
coroutineScope { // any failure in this block cancels them all
requests.map { async { it.await() } } // run them simultaneously with async
.awaitAll()
}
} catch (e: HttpException) {
Log.d("Err: ", e.localizedMessage.toString())
printStackTrace(e)
return#launch
}
// Do something with the list of three UploadedReads here.
}
I just duplicated the functionality of your code above, but it doesn't look like you're using the response for anything and you have an unused variable p.
Edit: If this is a pattern you use frequently, this helper function might be useful. I didn't check this thoroughly or test it.
/**
* Await the result of all the Calls in parallel. Any exception thrown by any item
* in the list will cancel all unfinished calls and be rethrown.
*/
suspend fun <T: Any> Iterable<Call<T>>.awaitAll(): List<T> =
coroutineScope { map { async { it.await } }.awaitAll() }
//...
lifecycleScope.launch {
val requests: List<Call<UploadedRead>> = listOf(
//...
)
val responses: List<UploadedRead> = try {
requests.awaitAll()
} catch (e: HttpException) {
//...
return#launch
}
//...
}

Unit Test for RxJava and Retrofit

I have This method that calls a Rest API and returns the result as an Observable (Single):
fun resetPassword(email: String): Single<ResetPassword> {
return Single.create { emitter ->
val subscription = mApiInterfacePanda.resetPassword(email)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe({ resetPasswordResponse ->
when(resetPasswordResponse.code()) {
200 -> {
resetPasswordResponse?.body()?.let { resetPassword ->
emitter.onSuccess(resetPassword)
}
}
else -> emitter.onError(Exception("Server Error"))
}
}, { throwable ->
emitter.onError(throwable)
})
mCompositeDisposable.add(subscription)
}
}
Unit Test:
#Test
fun resetPassword_200() {
val response = Response.success(200, sMockResetPasswordResponse)
Mockito.`when`(mApiInterfacePanda.resetPassword(Mockito.anyString()))
.thenReturn(Single.just(response))
mTokenRepository.resetPassword(MOCK_EMAIL)
val observer = mApiInterfacePanda.resetPassword(MOCK_EMAIL)
val testObserver = TestObserver.create<Response<ResetPassword>>()
observer.subscribe(testObserver)
testObserver.assertSubscribed()
testObserver.awaitCount(1)
testObserver.assertComplete()
testObserver.assertResult(response)
}
My Problem is only this line gets covered and the other lines won't run and that has a lot of impact on my total test coverage:
return Single.create { emitter ->

There's more than one thing going on here if I'm not mistaken. Let's take it in parts.
First, your "internal" observer:
mApiInterfacePanda.resetPassword(email)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe({ resetPasswordResponse -> ... })
Is observing on the android main thread and executing on a background thread. To the best of my knowledge, in most cases, the test thread will end before your mApiInterfacePanda .resetPassword has a chance to finish and run. You didn't really post the test setup, so I'm not sure if this is an actual issue, but in any case it's worth mentioning. Here's 2 ways to fix this:
RxJavaPlugins and RxAndroidPlugins
RxJava already provides a way to change the schedulers that are provided. An example is RxAndroidPlugins.setMainThreadSchedulerHandler. Here's how it could help:
#Before
fun setUp() {
RxAndroidPlugins.setInitMainThreadSchedulerHandler { Schedulers.trampoline() }
RxJavaPlugins.setInitIoSchedulerHandler { Schedulers.trampoline() }
}
The above methods make sure that everywhere you use the main thread scheduler and the io scheduler, it'll instead return the trampoline scheduler. This is a scheduler that guarantees that the code is executed in the same thread that was executing previously. In other words, it'll make sure you run it on the unit test main thread.
You will have to undo these:
#After
fun tearDown() {
RxAndroidPlugins.reset()
RxJavaPlugins.reset()
}
You can also change other schedulers.
Inject the schedulers
You can use kotlin's default arguments to help out with injecting schedulers:
fun resetPassword(
email: String,
obsScheduler: Scheduler = AndroidSchedulers.mainThread(),
subScheduler: Scheduler = Schedulers.io()
): Single<ResetPassword> {
return Single.create { emitter ->
val subscription = mApiInterfacePanda.resetPassword(email)
.observeOn(obsScheduler)
.subscribeOn(subScheduler)
.subscribe({ resetPasswordResponse ->
when(resetPasswordResponse.code()) {
200 -> {
resetPasswordResponse?.body()?.let { resetPassword ->
emitter.onSuccess(resetPassword)
}
}
else -> emitter.onError(Exception("Server Error"))
}
}, { throwable ->
emitter.onError(throwable)
})
mCompositeDisposable.add(subscription)
}
}
At test time you can just call it like resetPassword("foo#bar.com", Schedulers.trampoline(), Schedulers.trampoline() and for the application just pass in the email.
The other thing I see here is maybe not related to the problem, but I think it's still good to know. First, you're creating a single, but you don't need to do this.
Single.create is usually used when you don't have reactive code. However, mApiInterfacePanda.resetPassword(email) already returns a reactive component and although I'm not sure, let's just assume it's a single. If not, it should be fairly simple to convert it to something else.
You're also holding on to a disposable, which from what I can tell shouldn't be necessary.
Lastly, you're using retrofit according to your tags so you don't need to make the call return a raw response unless extremely necessary. This is true because retrofit checks the status code for you and will deliver the errors inside onError with an http exception. This is the Rx way of handling the errors.
With all this in mind, I'd rewrite the entire method like this:
fun resetPassword(email: String) = mApiInterfacePanda.resetPassword(email)
(note that resetPassword must not return a raw response, but Single<ResetPassword>
It actually shouldn't need anything else. Retrofit will make sure things end up in either onSuccess or onError. You don't need to subscribe to the result of the api here and handle disposables - let whoever is calling this code handle it.
You may also notice that if this is the case, then the solution for the schedulers is not needed. I guess this is true in this case, just remember some operators operate in some default schedulers and you may need to override them in some cases.
So how would I test the above method?
Personally I'd just check if the method calls the api with the right parameters:
#Test
fun resetPassword() {
mTokenRepository.resetPassword(MOCK_EMAIL)
verify(mApiInterfacePanda).resetPassword(MOCK_EMAIL)
}
I don't think there's much more needed here. There's no more logic I can see in the rewritten method.

Retrofit + Debouce on EditText is causing an InterruptedIOException

I currently have an EditText for the user to enter a search. I'm trying to use RxJava with debounce to only search every so often, instead of each character. However, I'm getting an InterruptedIOException while I'm testing, which kills the stream.
private val subject = BehaviorSubject.create<String>()
init {
configureAutoComplete()
}
private fun configureAutoComplete() {
subject.debounce(200, TimeUnit.MILLISECONDS)
.flatMap {
getSearchResults(query = it)
}
.subscribe({ result ->
handleResult(result)
}, { t: Throwable? ->
Logger.e(t, "Failed to search")
})
}
fun getSearchResults(query: String): Observable<List<MyObject>> {
val service = NetworkService.create() // get retrofit service
return service.search(query)
}
fun search(text: String) {
subject.onNext(text)
}
As you can see, I'm creating a BehaviorSubject, and within init I'm setting it up with debounce.
getSearchResult returns an Observable and does my network request.
But as I'm testing, if I type at a specific rate ( usually quick-ish, like typing another character while the request is ongoing ) it'll throw an Exception.
Failed to search : java.io.InterruptedIOException
at okhttp3.internal.http2.Http2Stream.waitForIo(Http2Stream.java:579)
at okhttp3.internal.http2.Http2Stream.takeResponseHeaders(Http2Stream.java:143)
at okhttp3.internal.http2.Http2Codec.readResponseHeaders(Http2Codec.java:125)
I was looking at this, https://stackoverflow.com/a/47276430/3106174, and it seems like I'm doing everything correctly.

After more testing, I realized that the network request was on the main thread.
You can test this by replacing your network call with Observerable.create{ ... } and throwing a Thread.sleep(1000) inside.
I was following this tutorial, https://proandroiddev.com/building-an-autocompleting-edittext-using-rxjava-f69c5c3f5a40, and one of the comments mention this issue.
"But I think one thing is misleading in your code snippet, and it’s
that subjects aren’t thread safe. And the thread that your code will
run on will be the thread that you emitting on (in this case the main
thread). "
To solve this issue, you need to force it to run on Schedulers.io(). Make sure it's after the debounce or it won't work.
private fun configureAutoComplete() {
subject.debounce(200, TimeUnit.MILLISECONDS)
.observeOn(Schedulers.io()) // add this here
.distinctUntilChanged()
.switchMap {
getSearchResults(query = it)
}
.subscribe({ result ->
handleResult(result)
}, { t: Throwable? ->
Logger.e(t, "Failed to search")
})
}

android rxjava repeating a request

I am trying to achieve the following. I load a list of objects I want to get values to put in a list later.
First I gather all the values into an array (to mountain order) using flatmap and then when everything is done I populate an adapter.
The thing I am unable to do is to repeat the operation ever xxx seconds. I understand its done using an interval. Still I get no result at all, or only none repeating one result.
Here’s my code:
Observable.fromIterable(URLList)
.concatMap(url -> standartRequest(App.getInstance().getApi().getService().getData(currency.getUrl())))
.retry(Constants.RETRY_COUNT)
.timeout(Constants.TIMEOUT_IN_SECONDS, TimeUnit.SECONDS)
.observeOn(AndroidSchedulers.mainThread())
.subscribe(this::success, this::error, this::valuesRetrieved);
recyclerView = ((CurrencyListFragment) controller).getRecyclerView();
LinearLayoutManager linearManager = new LinearLayoutManager(controller.getContext());
recyclerView.setLayoutManager(linearManager);
}
private void valuesRetrieved() {
listAdapter adapter = new listAdapter(valuesFromResponse);
recyclerView.setAdapter(adapter);
}
private void success(Object response) {
valuesFromResponse.add(response);
}
Where do I put
.interval(5, TimeUnit.SECONDS).timeInterval()

Well actually, you do not put interval anywhere, for repeating the operation every x interval, you should use repeat operator variant called repeatWhen where you can provide your interval logic in this way:
.repeatWhen(completed -> completed.delay(5, TimeUnit.SECONDS))
repeatWhen() will hand to you an Observable that transform your source Observable onCompleted() events as onNext() (with void), you should return Observable that emits onNext() which signals to resubscribe to your source Observable - meaning repeat the operation. While onCompleted()/onError() will be delivered as onCompleted()/onError() on your source Observable.
recommended reading regarding repeatWhen/retryWhen.
One thing to consider, as repeatWhen() will basically swallows all your onCompleted events (as you're repeating the operation there is no onCompleted(), your Observable will not stop from by itself!), then you should gather and update the adapter differently, I guess you can simply use toList() to gather all items to single onNext() (a replacement to your success() logic) and then on each onNext updates the list (what you're doing on onCompleted right now), to sum it up:
Observable.fromIterable(URLList)
.concatMap(url -> standartRequest(App.getInstance().getApi().getService().getData(currency.getUrl())))
.retry(Constants.RETRY_COUNT)
.timeout(Constants.TIMEOUT_IN_SECONDS, TimeUnit.SECONDS)
.toList()
.repeatWhen(completed -> completed.delay(5, TimeUnit.SECONDS))
.observeOn(AndroidSchedulers.mainThread())
.subscribe(this::valuesRetrieved, this::error);
EDIT:
Your timeout and retry logic are applied to the entire operation chain, so if all the network requests together take more than Constants.TIMEOUT_IN_SECONDS you will get timeout exception, you probably just want to retry and time out each individual request. like this:
Observable.fromIterable(URLList)
.concatMap(url -> standartRequest(App.getInstance()
.getApi().getService().getData(currency.getUrl())
.retry(Constants.RETRY_COUNT)
.timeout(Constants.TIMEOUT_IN_SECONDS, TimeUnit.SECONDS)))
.toList()
.repeatWhen(completed -> completed.delay(5, TimeUnit.SECONDS))
.observeOn(AndroidSchedulers.mainThread())
.subscribe(this::valuesRetrieved, this::error);

I am repeating my retrofit call every 2 second after it is completed
//Retrofit Builder
val retrofitBuilder = Retrofit.Builder()
.addConverterFactory(GsonConverterFactory.create())
.addCallAdapterFactory(RxJava2CallAdapterFactory.create())
.baseUrl("http://worldtimeapi.org/")
.build()
val timeApi = retrofitBuilder.create(TimeApi::class.java)
val timeObservable = timeApi.getTime()
timeObservable.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.repeatWhen { completed -> completed.delay(2, TimeUnit.SECONDS) }
.subscribe(object : Observer<Time> {
override fun onComplete() {
Log.e("MainActivity", "It is completed")
}
override fun onSubscribe(d: Disposable) {
Log.e("MainActivity", "you have successfully subscribed ")
}
override fun onNext(t: Time) {
progress.visibility = View.INVISIBLE
txtTime.text = t.unixtime
Log.e("MainActivity", "OnNext Called" + t.unixtime)
}
override fun onError(e: Throwable) {
Log.e("MainActivity", "ERROR")
}
})
}
See the Log Cat , onNext is called after every 2 second.

The repeatWhen can do the job, but in my sense, the interval could do the job as well, just like :
Observalbe.interval(5, TimeUnit.SECONDS)
.flatMap( /* Your Observabler.fromItere().concatMap().retry().timeout()*/)
.subscribe{ /* refresh RecyclerView* / }
In this way, you use flatMap to switch one stream (interval) to another stream (your business logic).