I'm getting an unwanted behavior/flaw when passing a list of Observable Network calls to Observable.Zip(), similar to this accepted answer:
How to make multiple request and wait until data is come from all the requests in retrofit 2.0 - android
And the unwanted behavior is, the network calls are being fired twice...
It's fired once when the Observable is added to the List, and then it's fired again during the Observable.zip()
Here's a boiled down snippet from my project that's reproducing the behavior:
fun buildListOfObservableNetworkCalls(): Observable<Map<String, String?>> {
val clients = mutableListOf<Observable<NetworkResponse>>()
if (NetworkClient1.featureFlag) {
val postBody = someMethodToBuildPostBody()
clients.add(NetworkClient1().executeClient(postBody))
}
//There will be multiple NetworkClients in the near future
return executeAllNetworkClients(headerBidClients)
}
private fun executeAllNetworkClients(clients: List<Observable<NetworkResponse>>): Observable<Map<String, String?>> =
if (clients.isNotEmpty()) {
Observable.zip(clients) {
it
}
.map { clientResults ->
clientResults.forEach { response ->
if (response is NetworkResponse) {
map[MY_KEY] += response.stringResult
}
}
map
}.doOnSubscribe {
android.util.Log.d("LOGGER", "zip ON SUBSCRIBE")
}
} else {
Observable.just(mapOf())
}
//**** My NetworkClient1 class containing the RxJava function that executes the network call ****//
override fun executeClient(postBody: CustomPostBody): Observable<NetworkResponse> =
retrofitApiInterface.networkCall1Request(postBody)
.subscribeOn(Schedulers.io())
.doOnSuccess { response ->
Log.d("LOGGER", "Client1 ON SUCCESS")
}
.flatMapObservable { response ->
Observable.just(
NetworkResponse(
response
)
)
}.onErrorResumeNext { throwable: Throwable? ->
android.util.Log.d("LOGGER", "Client1 ON ERROR")
Observable.just(
NetworkResponse(
""
)
)
}.doOnSubscribe {
android.util.Log.d("LOGGER", "Client1 ON SUBSCRIBE")
}
//***** And my custom Interceptor which logs the double requests ****////
class MyInterceptor: Interceptor {
override fun intercept(chain: Interceptor.Chain): Response {
val request = chain.request()
Log.d("LOGGER", "intercept request at ${System.currentTimeMillis()}")
val response = chain.proceed(request)
Log.d("LOGGER", "intercept response at ${System.currentTimeMillis()}")
return response
}
}
And the log output is:
D/LOGGER: zip ON SUBSCRIBE
D/LOGGER: Client1 ON SUBSCRIBE
D/LOGGER: intercept request at 1650059924358
D/LOGGER: intercept response at 1650059925747
D/LOGGER: Client1 ON SUCCESS
D/LOGGER: intercept request at 1650059925782
D/LOGGER: intercept response at 1650059925928
As you can see, the same network call is being executed twice.. and secondly, i'm also a bit puzzled as to why doOnSuccess isnt also called twice.
So, my main questions is, is there a way that I can build a list of Observables and pass it to Observable.zip() without executing the network call twice?
I see the issue is that I'm creating a List<Observable<NetworkResponse>> and in order to add network calls that return <Observable<NetworkResponse>>, I have to invoke the method as i'm adding them to the list. I know this may sound like a dumb question.. but is it at all possible to have a set-up where i'm able to add the Observable functions to the List without executing them? Probably over doing it, but would creating an extension function of .zip(iterable) which accepts a list of NetworkClients as the sources param and within the extension function, execute source.executeClient() be a feasible or stable solution?
I feel it would be inefficient if this was the unavoidable consequence of building a list of Observables to pass to zip(iterable), so i'm hoping that this is just a matter of my set-up rather than an overlooked consequence of the .zip(iterable) method.
I'm aware that I could avoid the above scenario by trying to pass each Observable Network call individually into the .zip() and use some sort of BiFunction to tie it all together. However, that doesn't seem very intuitive for my use case, being that I have to featureFlag check and build Post objects for each request that i'll be doing. Additionally, I'll be adding more NetWorkClients who's responses will all be returning the same base response type over the next few months, so I find the .zip(iterable) methodology as a clean and very scalable way of plugging in new NetworkClients.
Related
I am trying to write a simple app in Android Studio using Kotlin. It is a very steep learning curve for me, but I am almost there. My final problem is getting the app to wait for the APIs to complete before moving the next Intent.
I have three calls each uploading data via my API. They are called from a button and only when the three are uploaded, should the button send the user to the next intent/screen.
My API calls are working and I can see the data in the database. However, since enqueue is asynchronous the calls are firing and the code is moving on the start the next intent before the data is present.
The code below is executed 3 times (once for each upload). I realise this is probably not the best way to do it, but I'm trying to get it working before I finesse the code.
I thought that perhaps I could have a variable, UploadedReadCount, that I increment in the onResponse, but this doesn't seem to be working properly.
Could someone offer some advice as to how I should be pausing the code until the APIs complete? For example, is there an enqueue methos that isn't async?
ReadInterface.create().AddRead("new", rFuel, rRegister, rReadDate, rRead)
.enqueue(object : Callback<UploadedRead> {
override fun onFailure(call: Call<UploadedRead>, t: Throwable) {
Log.d("Err: ", t.localizedMessage!!)
t.printStackTrace()
}
override fun onResponse(call: Call<UploadedRead>, response: Response<UploadedRead>) {
Log.d("Response: ", response.body().toString())
val p = response.body()?.APIResult!![0]
msgShow("Gas read " + rRead.toString() + " uploaded")
UploadedReadCount += 1
}
})
while ( UploadedReadCount < 3) {
Log.d("Waiting ", UploadedReadCount.toString() + " reads uploaded...")
}
val intent = Intent(this, Billing::class.java).apply {
putExtra("ReadDate", txtReadDate.text.toString())
}
startActivity(intent)
In most cases you don't want to pause execution while API call returns, Instead you want to follow the reactive model, that is when you call API you specify some callbacks (onResponse, onFailure), and once these callbacks are invoked then you react.
code is moving on the start the next intent before the data is
present.
Move all of your code that depends on data received from API in onResponse or onFailure methods (callbacks), When API is ready with some response one of those callbacks will be invoked and then depending on the data that you receive from API you can continue your work.
is there an enqueue methos that isn't async?
There are options available to call an API in blocking manner but I don't think that is good idea. Instead of doing a blocking API call, you should try to do reactive programming that is as soon as any callback (onResponse, onFailure) is called only then you continue.
There is an alternative to enqueue that is suspending instead of async, so you can call your code sequentially without blocking the main thread in a coroutine. The function is await() and it returns the successful result or throws an HttpException on failure.
But to run three requests in parallel, you need to use the async coroutine builder. This can be done by mapping a list of Calls to async calls that await the individual results, and then using awaitAll() on the list of Deferreds to wait for all three. So, it's more complicated than just running sequential code in a coroutine, but I think this is still easier than trying to run and wait for three parallel calls using callbacks.
I'm not exactly sure what your other two calls are so I'll just make up some and assume this function already has all the data it needs to make the calls. I also don't know how you want to handle failure, so I'm just making it stop early if any of the three calls fail.
lifecycleScope.launch {
val requests: List<Call<UploadedRead>> = listOf(
ReadInterface.create().AddRead("new", rFuel, rRegister, rReadDate, rRead),
ReadInterface.create().AddRead("new2", rFuel, rRegister, rReadDate, rRead),
ReadInterface.create().AddRead("new3", rFuel, rRegister, rReadDate, rRead)
)
val responses: List<UploadedRead> = try {
coroutineScope { // any failure in this block cancels them all
requests.map { async { it.await() } } // run them simultaneously with async
.awaitAll()
}
} catch (e: HttpException) {
Log.d("Err: ", e.localizedMessage.toString())
printStackTrace(e)
return#launch
}
// Do something with the list of three UploadedReads here.
}
I just duplicated the functionality of your code above, but it doesn't look like you're using the response for anything and you have an unused variable p.
Edit: If this is a pattern you use frequently, this helper function might be useful. I didn't check this thoroughly or test it.
/**
* Await the result of all the Calls in parallel. Any exception thrown by any item
* in the list will cancel all unfinished calls and be rethrown.
*/
suspend fun <T: Any> Iterable<Call<T>>.awaitAll(): List<T> =
coroutineScope { map { async { it.await } }.awaitAll() }
//...
lifecycleScope.launch {
val requests: List<Call<UploadedRead>> = listOf(
//...
)
val responses: List<UploadedRead> = try {
requests.awaitAll()
} catch (e: HttpException) {
//...
return#launch
}
//...
}
Occasionally APIs can decide to do a ui job before finishing a network call (e.g. showing a webview dialog). Those flags are inside response headers and the value is usually the data, like the url of that webview.
This is the interceptor
override fun intercept(chain: Interceptor.Chain): Response {
val response = chain.proceed(request)
return when (response.code()) {
200 -> {
response.headers().forEach { /* do something in ui layer if needed */}
response;
}
}
}
Avoiding EventBus is intended, so I would add a meta data object to the response and then pass it to the repository and parse it in viewmodel, and produce the events for Activities and Fragments, but is not known to us when this interception is fired so one has to check for all types of events in all of the viewmodels for every api call.
The solution is to add an InterceptionMetaData to the api result.
data class InterceptionMetaData(
val key: String,
val data: String
)
Add this data class to the response (usually the success type, or we can add another Intercepted response type)
when (this) {
is Success -> success?.invoke(this.data, metaData)
is Error -> error?.invoke(this.error)
is Loading -> loading?.invoke()
is Intercepted -> onIntercepted?.invoke(this.data, interceptionMetaDataList)
}
in our interceptor we check for response headers (I have used all headers here for demonstration)
val interceptionMetaDataList: List<InterceptionMetaData> = response.headers().run {
names().map { key ->
get(key)?.let { InterceptionMetaData(key, it) }
}
}.filterNotNull()
and in our repositories we can handle the request and return the actual type to viewmodels.
In viewmodel, another response callback handler can be added (or it can be handled in the already used success callback)
repository.getNewsList().collectResult(
onIntercepted = { data, interceptionList ->
Log.d(NewsViewModel::class.java.simpleName, "getNewsList: $interceptionList")
},
success = { data ->
hideProgress()
handleData(data)
},
error = { error ->
hideProgress()
showNoData(error)
},
loading = {
showProgress()
})
Usually we have base viewmodels, so we can have our BaseViewModel build an event and pass it to our BaseFragment/BaseActivity to handle our actual ui job (e.g starting a dialog). That functionality is used where I have put Log.d.
I've spent hours trying to figure this thing out, and I still can figure it out.
I'm trying to retrieve data from a website using JSON.
If the website is live and everything, it works, but if the website returns something else than the data, like a 403 error, or any other error, then it crashes. I tried to debug it, but I still don't understand what is going on here.
Here is my code:
I have a NetworkModule with an interceptor that is supposed to check is the response is valid or not, and from what I can tell it works, because my variable isDataRetrievable is false (the value by default):
val networkModule = module {
single {
val customGson =
GsonBuilder().registerTypeAdapter(Lesson::class.java, LessonDeserializer())
.create()
Retrofit.Builder()
.client(get())
.addConverterFactory(
GsonConverterFactory.create(customGson)
)
.baseUrl(BuildConfig.URL)
.build()
}
factory {
OkHttpClient.Builder()
.addInterceptor(Interceptor { chain ->
chain.withConnectTimeout(1,TimeUnit.SECONDS)
val request: Request = chain.request()
val response = chain.proceed(request)
if (response.isSuccessful){
networkStatus.isDataRetrievable = true
}
response
}).build()
}
factory {
get<Retrofit>().create(LessonApi::class.java)
}
}
Next, I have my API to get the data:
interface LessonApi {
#GET("/JSON/json_get_data.php")
suspend fun getLessons(): Call<Lesson>
}
Then, for some reason, I have a repository (I'm not the only one working on this code, I didn't do this part):
class LessonRepository(private val service: LessonApi) {
suspend fun getLessons() = service.getLessons()
}
Then, I have my splash screen view model, that is supposed to retrieve the data if possible:
if (networkStatus.isNetworkConnected && networkStatus.isWebsiteReachable) {
var tmp = repository.getLessons()
tmp.enqueue(object : Callback<Lesson> {
override fun onFailure(call: Call<Lesson>, t: Throwable) {
Log.d("DataFailure",t.message.toString())
nextScreenLiveData.postValue(false)
}
override fun onResponse(call: Call<Lesson>, response: Response<Lesson>) {
Log.d("DataFailure","Test")
}
})
}else{
nextScreenLiveData.postValue(false)
}
The problem is that when the program get to the line repository.getLessons(), it crashes with the error:
retrofit2.HttpException: HTTP 403
at retrofit2.KotlinExtensions$await$2$2.onResponse(KotlinExtensions.kt:49)
at retrofit2.OkHttpCall$1.onResponse(OkHttpCall.java:129)
at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:919)
So onFailure or onResponse are never called. I tried to run the debugger, to step in, but I cannot figure it out when it fails.
I thought it was because it was trying to deserialize invalid data, but I put breakpoints everywhere in my deserializer, and it never hits the breakpoints.
I'm not a professional android developer, but I'm very confused here.
What I'd like to do is that if the request is unsuccessful, just discard the response (do not deserialize it), and display a message or exit.
Please help, it's so frustrating. I'm not sure how to intercept errors or what to do if Interceptors get an unsuccessful request (for now I just set a variable but it's unused).
Thanks.
Edit: What I'm trying to do, is to retrieve data from a webserver. If it cannot (for any reason), I don't want the gson to parse data (because it will probably be garbage and will not correspond to my deserializer). However, I feel like this okhttp / retrofit is a pipeline, where okhttp get the response from the webserver and pass it to a gson converter. What I want to do is intercept this response, and if it's not successful, to NOT pass it to gson, set a variable, so that the rest of my application knows what to do. But the thing is, for now, it just crash even before it gets to the callback in enqueue. The interceptor works just fine, except I'd like him to drop the response if it's not successful. Is it possible?
I tried something like that, and it worked to handle bad codes (>400), but I also wanted to handle malformed JSON data, so I added the onResponse and onFailure callbacks, but it never worked, because when I receive a malformed JSON, it also trigger an exception, and then go in the catch before it can go on the 'enqueue', so I'm not sure what this is used for.
try {
val lessons = repository.getLessons().enqueue(object : Callback<List<Lesson>> {
override fun onResponse(call: Call<List<Lesson>>, response: Response<List<Lesson>>) {
networkStatus.isDataRetrievable = response.isSuccessful
Log.d("Retrofit", "Successful response")
nextScreenLiveData.postValue(response.isSuccessful)
}
override fun onFailure(call: Call<List<Lesson>>, t: Throwable) {
Log.d("Retrofit", "Failure response")
nextScreenLiveData.postValue(false)
}
})
nextScreenLiveData.postValue(true)
} catch (e: Exception) {
nextScreenLiveData.postValue(false)
}
Anyway, just this code works for everything in the end:
try {
val lessons = repository.getLessons().filter {
it.lesson.contains("video")
}.filter {
DataUtils.isANumber(it.id)
}
lessonDao.insertLessons(lessons)
networkStatus.isDataRetrievable = true
} catch (e: Exception) {
networkStatus.isDataRetrievable = false
}
But in my API, I don't return callbacks, I directly return the objects, as such:
#GET("/JSON/json_get_dat.php")
suspend fun getLessons(): List<Lesson>
I don't know if this is the right way to do it, but it works. I hope this might help others.
Retrofit first request with Single blocks UI thread. Below is relevant code, and more text:
RetrofitProvider
object RetrofitProvider {
private val TAG: String = RetrofitProvider::class.java.simpleName
val retrofit: Retrofit by lazy {
val httpClient = OkHttpClient.Builder()
.addInterceptor {
val request = it.request()
if (BuildConfig.DEBUG) {
Log.d(TAG, "${request.method()}: ${request.url()}")
}
it.proceed(request)
}
.build()
Retrofit.Builder()
.client(httpClient)
.baseUrl("http://192.168.0.10:3000")
.addCallAdapterFactory(RxJava2CallAdapterFactory.createWithScheduler(Schedulers.io()))
.addConverterFactory(JacksonConverterFactory.create(jacksonObjectMapper()))
.build()
}
}
ProductApi
interface ProductApi {
#GET("/products")
fun getProducts(): Single<List<Product>>
}
MainViewModel
fun fetchProducts() {
productData.value = Resource.Loading()
productApi.getProducts() // <- This call is a problem (even when I comment out all code below)
.subscribeOn(Schedulers.io())
.subscribe(
{
productData.postValue(Resource.Success(it))
},
{
productData.postValue(Resource.Fail(it.message))
})
.addTo(disposableContainer)
}
MainFragment
...
button.setOnClickListener {
Toast.makeText(requireContext(), "click", Toast.LENGTH_SHORT).show()
mainViewModel.fetchProducts()
}
...
App flow is simple, clicking a button on MainFragment calls MainViewModel's fetchProducts() which uses retrofit to fetch some stuff.
productApi.getProducts() happens on UI thread and blocks it significantly(~half a second), even Toast is delayed, even though it should be shown immediately on button click, before getProducts() call.
productApi.getProducts() by itself, without subscribe doesn't send network request (I checked on server side), it just prepares Single.
Important note, delay DOES NOT happen on subsequent clicks to button. Just the first time, I guess creating Single<> is expensive operation.
So my question is, why is UI thread blocked on first request, and how do I fix it the way it isn't ugly/hacking.
Also Observable acts the same, but Completable works much faster, but I need the data, so can't use Completable.
I think your problem lies with the lazy initialisation of your Retrofit object.
It will be deferred to the last possible moment, so I guess the first time you click on the button, you create the expensive retrofit button (this is done on the main thread).
My suggestion is to remove the lazy initialisation and try running the app once again.
Returning Completable also blocks the UI thread but for less time than returning Single or Observable so it seems like it doesn't have any impact but it does.
Invoking the API call on a background thread will not block your UI as the converter creation will not happen on the UI thread.
Something like this does the trick.
Completable.complete()
.observeOn(Schedulers.io())
.subscribe {
productApi.getProducts()
.subscribe(
{
productData.postValue(Resource.Success(it))
},
{
productData.postValue(Resource.Fail(it.message))
}
)
.addTo(disposableContainer)
}
.addTo(disposableContainer)
Another thing you can do instead of using the converter is to make a wrapper class around the Retrofit API which will call it in a fitting observable on a background thread.
fun getProducts() = Single.create<List<Product>> { emitter ->
try {
val response = productApi.getProducts().execute()
if (!response.isSuccessful) {
throw HttpException(response)
}
emitter.onSuccess(response.body()!!)
} catch (e: Exception) {
emitter.onError(e)
}
}.observeOn(Schedulers.io())
When you invoke a RxJava action, for example, a retrofit request you can to tell it where to perform the action and where to get the result the default location is where you subscribe to it
in order to change it you need to add two lines
observeOn(Where you will receive the result)
subscribeOn(Where the action will be executed)
In your case, it should be something like this
productApi.getProducts() // <- This call is a problem (even when I comment out all code below)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io()) //or .subscribeOn(Schedulers.newThread())
.subscribe({Success},{Failure})
I have made a library that has a lot of utilities/extensions for Android development in kotlin.
One of the packages is there to make it simple to avoid this issue.
All you need to do is type:
yourObservable //or any other reactive type
.runSafeOnMain() //it will perform you action in another thread and it will return the result in main
.subscribe({}, {])
I'm trying to make two requests when user clicks on a button.
However, the request might take a little while.
When the user is on battery save mode and screens lock his device while the request is still being done, the request will not complete and will give a socket timeout exception.
I made a sample project to try this out and you can find it here.
I'm using retrofit and RxJava to make my requests like this:
networkFactory.request()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(
{ result -> Log.d("TAG", "The value is this $result") },
{ error -> Log.e("TAG", "Ohoh an error ${Log.getStackTraceString(error)}")
})
My networkFactory request() is:
fun request(): Observable<Doc> {
return service.request(API_KEY)
}
with the following interface:
#GET("articlesearch.json")
fun request(#Query("api-key") apiKey : String) : Observable<Doc>
Am I doing something wrong here?