I'm using Retrofit in order to make some network requests. I'm also using the Coroutines in combination with 'suspend' functions.
My question is: Is there a way to improve the following code. The idea is to launch multiple requests in parallels and wait for them all to finish before continuing the function.
lifecycleScope.launch {
try {
itemIds.forEach { itemId ->
withContext(Dispatchers.IO) { itemById[itemId] = MyService.getItem(itemId) }
}
} catch (exception: Exception) {
exception.printStackTrace()
}
Log.i(TAG, "All requests have been executed")
}
(Note that "MyService.getItem()" is a 'suspend' function.)
I guess that there is something nicer than a foreach in this case.
Anyone with an idea?
I've prepared three approaches to solving this, from the simplest to the most correct one. To simplify the presentation of the approaches, I have extracted this common code:
lifecycleScope.launch {
val itemById = try {
fetchItems(itemIds)
} catch (exception: Exception) {
exception.printStackTrace()
}
Log.i(TAG, "Fetched these items: $itemById")
}
Before I go on, a general note: your getItem() function is suspendable, you have no need to submit it to the IO dispatcher. All your coroutines can run on the main thread.
Now let's see how we can implement fetchItems(itemIds).
1. Simple forEach
Here we take advantage of the fact that all the coroutine code can run on the main thread:
suspend fun fetchItems(itemIds: Iterable<Long>): Map<Long, Item> {
val itemById = mutableMapOf<Long, Item>()
coroutineScope {
itemIds.forEach { itemId ->
launch { itemById[itemId] = MyService.getItem(itemId) }
}
}
return itemById
}
coroutineScope will wait for all the coroutines you launch inside it. Even though they all run concurrently to each other, the launched coroutines still dispatch to the single (main) thread, so there is no concurrency issue with updating the map from each of them.
2. Thread-Safe Variant
The fact that it leverages the properties of a single-threaded context can be seen as a limitation of the first approach: it doesn't generalize to threadpool-based contexts. We can avoid this limitation by relying on the async-await mechanism:
suspend fun fetchItems(itemIds: Iterable<Long>): Map<Long, Item> = coroutineScope {
itemIds.map { itemId -> async { itemId to MyService.getItem(itemId) } }
.map { it.await() }
.toMap()
}
Here we rely on two non-obvious properties of Collection.map():
It performs all the transformation eagerly, so the first transformation to a collection of Deferred<Pair<Long, Item>> is completely done before entering the second stage, where we await on all of them.
It is an inline function, which allows us to write suspendable code in it even though the function itself is not a suspend fun and gets a non-suspendable lambda (Deferred<T>) -> T.
This means that all the fetching is done concurrently, but the map gets assembled in a single coroutine.
3. Flow-Based Approach with Improved Concurrency Control
The above solved the concurrency for us, but it lacks any backpressure. If your input list is very large, you'll want to put a limit on how many simultaneous network requests you're making.
You can do this with a Flow-based idiom:
suspend fun fetchItems(itemIds: Iterable<Long>): Map<Long, Item> = itemIds
.asFlow()
.flatMapMerge(concurrency = MAX_CONCURRENT_REQUESTS) { itemId ->
flow { emit(itemId to MyService.getItem(itemId)) }
}
.toMap()
Here the magic is in the .flatMapMerge operation. You give it a function (T) -> Flow<R> and it will execute it sequentially on all the input, but then it will concurrently collect all the flows it got. Note that I couldn't simplify flow { emit(getItem()) } } to just flowOf(getItem()) because getItem() must be called lazily, while collecting the flow.
Flow.toMap() is not currently provided in the standard library, so here it is:
suspend fun <K, V> Flow<Pair<K, V>>.toMap(): Map<K, V> {
val result = mutableMapOf<K, V>()
collect { (k, v) -> result[k] = v }
return result
}
If you are looking for just a nicer way to write it and eliminate foreach
lifecycleScope.launch {
try {
itemIds.asFlow()
.flowOn(Dispatchers.IO)
.collect{ itemId -> itemById[itemId] = MyService.getItem(itemId)}
} catch (exception: Exception) {
exception.printStackTrace()
}
Log.i(TAG, "All requests have been executed")
}
Also please look at lifecycleScope I suspect it is using Dispatchers.Main. If that is the case you can remove this .flowOn(Dispatchers.IO) extra dispatcher declaration.
For more info: Kotlin Asynchronous Flow
Related
Basically I want to make a network request when initiated by the user, collect the Flow returned by the repository and run some code depending on the result. My current setup looks like this:
Viewmodel
private val _requestResult = MutableSharedFlow<Result<Data>>()
val requestResult = _requestResult.filterNotNull().shareIn(
scope = viewModelScope,
started = SharingStarted.WhileViewSubscribed,
replay = 0
)
fun makeRequest() {
viewModelScope.launch {
repository.makeRequest().collect { _requestResult.emit(it) }
}
}
Fragment
buttonLayout.listener = object : BottomButtonLayout.Listener {
override fun onButtonClick() {
viewModel.makeRequest()
}
}
lifecycleScope.launchWhenCreated {
viewModel.requestResult.collect { result ->
when (result) {
Result.Loading -> {
doStuff()
}
is Result.Success -> {
doDifferentStuff(result.data)
}
is Result.Failure -> {
handleError()
}
}
}
}
The first time the request is made everything seems to work. But starting with the second time the collect block in the fragment does not run anymore. The request is still made, the repository returns the flow as expected, the collect block in the viewmodel runs and emit() also seems to be executed successfully.
So what could be the problem here? Something about the coroutine scopes? Admittedly I lack any sort of deeper understanding of the matter at hand.
Also is there a more efficient way of accomplishing what I'm attempting using Kotlin Flows in general? Collecting a flow and then emitting the same flow again seems a bit counterintuitive.
Thanks in advance:)
According to the documentation there are two recommended alternatives:
viewLifecycleOwner.lifecycleScope.launch {
viewLifecycleOwner.repeatOnLifecycle(Lifecycle.State.STARTED) {
//your thing
}
}
I rather the other alternative:
viewLifecycleOwner.lifecycleScope.launch {
viewModel.makeReques().flowWithLifecycle(viewLifecycleOwner.lifecycle, Lifecycle.State.STARTED)
.collect {
// Process the value.
}
}
I like the flowWithLifecycle shorter syntax and less boiler plate. Be carefull thar is bloking so you cant have anything after that.
The oficial docs
https://developer.android.com/topic/libraries/architecture/coroutines
Please be aware you need the lifecycle aware library.
In my sample I'm calling network operation and emitting success case but on error e.g 404 app crashes wihout emitting exception. Surrendering with try catch prevent crashes but I want to pass error till the ui layer like success case.
suspend fun execute(
params: Params,
):
Flow<Result<Type>> = withContext(Dispatchers.IO) {
flow {
emit(Result.success(run(params)))
}.catch {
emit(Result.failure(it))
}
}
There is a helpful function runCatching for creating a Result easily, but the problem in coroutines is that you don't want to be swallowing CancellationExceptions. So below, I'm using runCatchingCancellable from my answer here.
This shouldn't be a Flow since it returns a single item.
If run is a not a blocking function (it shouldn't be if you are using Retrofit with suspend functions), your code can simply be:
suspend fun execute(params: Params): Result<Type> = runCatchingCancellable {
run(params)
}
If it is a blocking function you can use:
suspend fun execute(params: Params): Result<Type> = runCatchingCancellable {
withContext(Dispatchers.IO) {
run(params)
}
}
If you were going to return a Flow (which you shouldn't for a returning a single item!!), then you shouldn't make this a suspend function, and you should catch the error inside the flow builder lambda:
fun execute(params: Params): Flow<Result<Type>> = flow {
emit(runCatchingCancellable {
run(params)
})
}
// or if run is blocking (it shouldn't be):
fun execute(params: Params): Flow<Result<Type>> = flow {
emit(runCatchingCancellable {
withContext(Dispatchers.IO) { run(params) }
})
}
If you want to use flows you can use the catch method of flows.
As you said you can use try-catch but it would break the structured concurrency since it would catch the cancellation exception as well or it would avoid the cancellation exception to be thrown.
One thing that you can do is to use an Exception handler at the point where you launch the root coroutine that calls the suspend function.
val coroutineExceptionHandler = CoroutineExceptionHandler { _, exception ->
// handle it
}
scope.launch(handler) { // root coroutine
execute(params)
somethingThatShouldBeExecutedOnlyIfPreviousCallDoesNotThrow()
}
This solution is good for both flows and non-flow coroutines.
In the solution with the runCatching you will have to manually check the result of the first execute to avoid the second one to run.
One interesting thread is here.
I have these piece of code and I want it to make it more optimal
I guess I can use kotlin-flow's flatMapMerge
but I don't think how should I convert my code to flow
val quiries = listof("s","b","s","g","d","r","t")
quiries.map { query ->
viewModelScope.launch {
val results = fetchColumns(query)
resultMutableData.postValue(results)
}
}
and fetchColumns() are suspended functions
I am thinking maybe I need to have flows of queries ???? what is the way of using flatMapMerge()?
https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/flat-map-merge.html
Try using something like this:
listOf("s","b","s","g","d","r","t").asFlow()
.map { fetchColumns(it) }
.onEach { resultMutableData.postValue(it) }
.launchIn(viewModelScope)
Since you don't switch onto another flow, there is no need for any of flatMap* functions, just map will be enough. Moreover, map parameter is already declared as suspend, so you won't block your thread.
But map operator was designed to process data sequentially, so these transformations won't be run in parallel. To achieve parallel processing, a workaround using flatMapMerge can be used:
listOf("s","b","s","g","d","r","t").asFlow()
.onEach { println("onEach: $it") }
.flatMapMerge {
flow {
emit(fetchColumns(it))
}
}
.onEach { resultMutableData.postValue(it)) }
.launchIn(viewModelScope)
I've recently dove into Kotlin coroutines
Since I use a lot of Google's libraries, most of the jobs is done inside Task class
Currently I'm using this extension to suspend coroutine
suspend fun <T> awaitTask(task: Task<T>): T = suspendCoroutine { continuation ->
task.addOnCompleteListener { task ->
if (task.isSuccessful) {
continuation.resume(task.result)
} else {
continuation.resumeWithException(task.exception!!)
}
}
}
But recently I've seen usage like this
suspend fun <T> awaitTask(task: Task<T>): T = suspendCoroutine { continuation ->
try {
val result = Tasks.await(task)
continuation.resume(result)
} catch (e: Exception) {
continuation.resumeWithException(e)
}
}
Is there any difference, and which one is correct?
UPD: second example isn't working, idk why
The block of code passed to suspendCoroutine { ... } should not block a thread that it is being invoked on, allowing the coroutine to be suspended. This way, the actual thread can be used for other tasks. This is a key feature that allows Kotlin coroutines to scale and to run multiple coroutines even on the single UI thread.
The first example does it correctly, because it invokes task.addOnCompleteListener (see docs) (which just adds a listener and returns immediately. That is why the first one works properly.
The second example uses Tasks.await(task) (see docs) which blocks the thread that it is being invoked on and does not return until the task is complete, so it does not allow coroutine to be properly suspended.
One of the ways to wait for a Task to complete using Kotlin Coroutines is to convert the Task object into a Deferred object by applying Task.asDeferred extension function. For example for fetching data from Firebase Database it can look like the following:
suspend fun makeRequest() {
val task: Task<DataSnapshot> = FirebaseDatabase.getInstance().reference.get()
val deferred: Deferred<DataSnapshot> = task.asDeferred()
val data: Iterable<DataSnapshot> = deferred.await().children
// ... use data
}
Dependency for Task.asDeferred():
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-play-services:1.5.2'
To call suspend function we need to launch a coroutine:
someCoroutineScope.launch {
makeRequest()
}
someCoroutineScope is a CoroutineScope instance. In android it can be viewModelScope in ViewModel class and lifecycleScope in Activity or Fragment, or some custom CoroutineScope instance. Dependencies:
implementation 'androidx.lifecycle:lifecycle-viewmodel-ktx:2.4.0'
implementation 'androidx.lifecycle:lifecycle-runtime-ktx:2.4.0'
I'm trying to use Kotlin's coroutines to avoid callback hell, but it doesnt look like I can in this specific situation, I would like some thougths about it.
I have this SyncService class which calls series of different methods to send data to the server like the following:
SyncService calls Sync Student, which calls Student Repository, which calls DataSource that makes a server request sending the data through Apollo's Graphql Client.
The same pattern follows in each of my features:
SyncService -> Sync Feature -> Feature Repository -> DataSource
So every one of the method that I call has this signature:
fun save(onSuccess: ()-> Unit, onError:()->Unit) {
//To Stuff here
}
The problem is:
When I sync and successfully save the Student on server, I need to sync his enrollment, and if I successfully save the enrollment, I need to sync another object and so on.
It all depends on each other and I need to do it sequentially, that's why I was using callbacks.
But as you can imagine, the code result is not very friendly, and me and my team starting searching for alternatives to keep it better. And we ended up with this extension function:
suspend fun <T> ApolloCall<T>.execute() = suspendCoroutine<Response<T>> { cont ->
enqueue(object: ApolloCall.Callback<T>() {
override fun onResponse(response: Response<T>) {
cont.resume(response)
}
override fun onFailure(e: ApolloException) {
cont.resumeWithException(e)
}
})
}
But the function in DataSource still has a onSuccess() and onError() as callbacks that needs to be passed to whoever call it.
fun saveStudents(
students: List<StudentInput>,
onSuccess: () -> Unit,
onError: (errorMessage: String) -> Unit) {
runBlocking {
try {
val response = GraphQLClient.apolloInstance
.mutate(CreateStudentsMutation
.builder()
.students(students)
.build())
.execute()
if (!response.hasErrors())
onSuccess()
else
onError("Response has errors!")
} catch (e: ApolloException) {
e.printStackTrace()
onError("Server error occurred!")
}
}
}
The SyncService class code changed to be like:
private fun runSync(onComplete: () -> Unit) = async(CommonPool) {
val syncStudentProcess = async(coroutineContext, start = CoroutineStart.LAZY) {
syncStudents()
}
val syncEnrollmentProcess = async(coroutineContext, start = CoroutineStart.LAZY) {
syncEnrollments()
}
syncStudentProcess.await()
syncEnrollmentProcess.await()
onComplete()
}
It does execute it sequentially, but I need a way to stop every other coroutine if any got any errors. Error that might come only from Apollo's
So I've been trying a lot to find a way to simplify this code, but didn't get any good result. I don't even know if this chaining of callbacks can be simplify at all. That's why I came here to see some thoughts on it.
TLDR: I want a way to execute all of my functions sequentially, and still be able to stop all coroutines if any got an exception without a lot o chaining callbacks.