Develop Reference

Why use withContext() vs Async-await, isn't the goal of coroutines to achieve concurrency?

I am learning about threading and kotlin coroutines to load my UI and repository concurrently, but then "withContext()" as an alternative to async-await.
If I understood "withContext()" correctly and it executes one task after another waiting for the previous task to finish, why ever use it? Is there another concept I'm missing?

withContext is a suspending function that allows to execute a specific piece of code in a different coroutine context. It is in particular useful when you want to execute something in a different dispatcher.
For instance, you could have some code run in the default dispatcher with multiple threads, but then use a value produced by that code to update some UI in the UI thread. In that case, you're not looking for concurrency, the computation on the default dispatcher has to happen before updating the UI because you need the result:
val result = someComputation()
withContext(Dispatchers.Main) {
updateUI(result)
}
Of course, even if the computation and the update of the UI are not concurrent, their sequence can be concurrent with other pieces of code:
scope.launch(Dispatchers.Default) {
val result = someComputation()
withContext(Dispatchers.Main) {
updateUI(result)
}
}
If you need to execute concurrent things, you can use coroutine builders like launch and async. However, using async { ... } immediately followed by .await() defeats the purpose of concurrency, because the code between the async and await() calls is precisely what will run concurrently with the async's body:
val deferred = async { computeSomeValue() }
somethingConcurrentWithAsyncBody()
val result = deferred.await()
You can read more about how to organize your calls to achieve concurrency in the part of the doc about composing suspend functions.

Android Coroutines in SyncAdapter being Interrupted

I'm having a tough time figuring out why do I observe InterruptedExceptions in the SyncAdapter#onPerformSync method.
Brief intro
My project originally contained only Java code and had a working SyncAdapter pattern implemented.
In the onPerfromSync method I did the following operations for each resource that I needed to sync:
query local sqlite database
send data to web server over HTTP
save data to local sqlite database
As the project progressed I introduced Kotlin and Coroutines and decided to use them in the SyncAdapter.
Most of the resources kept the existing logic, and for a few I decided to use Coroutines.
I had to bridge the onPerformSync method with a CoroutineScope so that I can launch coroutines. After doing so I started observing InterruptedExceptions occurring.
Code before (no exceptions)
class MySyncAdapter extends AbstractThreadedSyncAdapter {
...
#Override
public void onPerformSync(...) {
syncResourceX();
syncResourceY();
}
private void syncResourceX() {
// query db
// send to server
// store locally
}
}
Code After (with exceptions)
class MySyncAdapter(...): AbstractThreadedSyncAdapter(...) {
...
override fun onPerformSync(...) {
runBlocking {
syncResourceX();
syncResourceY();
}
}
private suspend fun syncResourceX() {
// query db
// send to server
// store locally
}
}
After this refactoring I started receiving InterruptedExceptions where runBlocking is being invoked.
Initially I thought this might be due to not performing network operations on the current thread as the docs of the sync adapter state that the system monitors network traffic and might interrupt the sync process if no traffic is generated.
But runBlocking should cause any network requests to be executed on the current thread right?
After which I started thinking that this process has existed in the Java code as well. It's just that there is nothing to report the interruption of the sync process. It is up until I started using coroutines and runBlocking that this problem has revealed it self. Or so I think now.
Question:
Any thoughts or explanations why I previously did not observe InterruptedException with the Java code and do observe InterruptedException with the Kotlin code is more than welcome.
Notes
I haven't overriden onSyncCanceled
The sync process usually takes less then 20 seconds to complete

Android kotlin coroutine: await might silently drop exceptions

While reading https://developer.android.com/kotlin/coroutines I stumbled upon the following warning:
Warning: launch and async handle exceptions differently. Since async expects an eventual call to await at some point, it holds exceptions and rethrows them as part of the await call. This means if you use await to start a new coroutine from a regular function, you might silently drop an exception. These dropped exceptions won't appear in your crash metrics or be noted in logcat.
However I'm not able to find any example of this silent-dropping behavior while browsing https://kotlinlang.org/docs/reference/coroutines/exception-handling.html or any other resources returned by https://www.google.com/search?q=kotlin+await+exception+handling - on the contrary, all resources indicate that exception thrown in an async/await block will cause a failure in the whole coroutine scope which is correct and expected.
I'm afraid I'm missing something here, can you provide an example where this silent exception dropping occurs which cannot be noted in logcat?

The passage you quote from Kotlin documentation is outdated. It used to be true in the experimental phase, and you really could get swallow exceptions if you weren't very pedantic.
The release version of coroutines acquired an additional key component: structured concurrency. When following the simple guidelines not to use GlobalScope and similar, your code will automatically be organized such that each coroutine has a parent and there's a well-defined scope within which all coroutines must complete either normally or abruptly, and the owner of that scope can await its completion, getting the exception that caused the abnormal completion.
Within this new discipline, launch and async are on the same footing. If an exception thrown inside one of them escapes its top-level block, it goes on to cancel the coroutine, notify the parent of this, and then the parent may opt to cancel all the other children so that the whole scope ends prematurely.
Note that getting an exception from the await call is an entirely different mechanism. The thing you await on is not the async block itself, but a standalone object of type Deferred, which is identical to Java's Future except that await suspends instead of blocks. The async block, upon completion, always does the same: it completes the Deferred with either the return value or the exception. You may retrieve that result from anywhere you pass the Deferred, it's completely decoupled from the destiny of the async coroutine.

Ok, I found an example which is in line with Marko's answer concerning structured concurrency. Code snippet below presents the silent exception dropping usecase.
This example comes from https://medium.com/androiddevelopers/coroutines-on-android-part-ii-getting-started-3bff117176dd
val unrelatedScope = MainScope()
// example of a lost error
suspend fun lostError() {
// async without structured concurrency
unrelatedScope.async {
throw InAsyncNoOneCanHearYou("except")
}
}
Note this code is declaring an unrelated coroutine scope that will launch a new coroutine without structured concurrency. [...]
The error is lost in this code because async assumes that you will eventually call await where it will rethrow the exception. However, if you never do call await, the exception will be stored forever waiting patiently to be raised.

Exceptions from launch are thrown "instantly" into exception handler, while those from async will be thrown the moment you call await.
Sample code:
import kotlinx.coroutines.*
fun main(args: Array<String>) = runBlocking{
println("main start")
val l = GlobalScope.launch{ justThrow("launch") }
val a = GlobalScope.async { justThrow("async") }
delay(500)
// a.await()
println("main finished")
}
suspend fun justThrow(who : String){
println("starting $who")
delay(100)
throw Exception("Test exception from $who")
}
Output:
main start
starting launch
starting async
Exception in thread "DefaultDispatcher-worker-2 #coroutine#2" java.lang.Exception: Test exception from launch
at FileKt.justThrow(File.kt:15)
(... entire stack here )
main finished
You can see launch throwing exception in worker thread but there's nothing from async, even though it executed the same function that must fail.
For next run modify the code by removing the launch for clarity and uncommenting a.await():
fun main(args: Array<String>) = runBlocking{
println("main start")
val a = GlobalScope.async { justThrow("async") }
delay(500)
a.await()
println("main finished")
}
Output:
main start
starting async
Exception in thread "main" java.lang.Exception: Test exception from async
at FileKt.justThrow (File.kt:15)
(... entire stack here )
Exception is held instead of crashing in worker thread and is instead re-thrown the moment await() is called which causes the app to crash. You can notice it happened on main thread and there was no "main finished" print because of that.

Is NetworkOnMainThreadException valid for a network call in a coroutine?

I'm putting together a simple demo app in Kotlin for Android that retrieves the title of a webpage with Jsoup. I'm conducting the network call using Dispatchers.Main as context.
My understanding of coroutines is that if I call launch on the Dispatchers.Main it does run on the main thread, but suspends the execution so as to not block the thread.
My understanding of android.os.NetworkOnMainThreadException is that it exists because network operations are heavy and when run on the main thread will block it.
So my question is, given that a coroutine does not block the thread it is run in, is a NetworkOnMainThreadException really valid? Here is some sample code that throws the given Exception at Jsoup.connect(url).get():
class MainActivity : AppCompatActivity() {
val job = Job()
val mainScope = CoroutineScope(Dispatchers.Main + job)
// called from onCreate()
private fun printTitle() {
mainScope.launch {
val url ="https://kotlinlang.org"
val document = Jsoup.connect(url).get()
Log.d("MainActivity", document.title())
// ... update UI with title
}
}
}
I know I can simply run this using the Dispatchers.IO context and providing this result to the main/UI thread, but that seems to dodge some of the utility of coroutines.
For reference, I'm using Kotlin 1.3.

My understanding of coroutines is that if I call launch on the Dispatchers.Main it does run on the main thread, but suspends the execution so as to not block the thread.
The only points where execution is suspended so as to not block the thread is on methods marked as suspend - i.e., suspending methods.
As Jsoup.connect(url).get() is not a suspending method, it blocks the current thread. As you're using Dispatchers.Main, the current thread is the main thread and your network operation runs directly on the main thread, causing the NetworkOnMainThreadException.
Blocking work like your get() method can be made suspending by wrapping it in withContext(), which is a suspending method and ensures that the Dispatchers.Main is not blocked while the method runs.
mainScope.launch {
val url ="https://kotlinlang.org"
val document = withContext(Dispatchers.IO) {
Jsoup.connect(url).get()
}
Log.d("MainActivity", document.title())
// ... update UI with title
}

Coroutine suspension is not a feature that magically "unblocks" an existing blocking network call. It is strictly a cooperative feature and requires the code to explicitly call suspendCancellableCoroutine. Because you're using some pre-existing blocking IO API, the coroutine blocks its calling thread.
To truly leverage the power of suspendable code you must use a non-blocking IO API, one which lets you make a request and supply a callback the API will call when the result is ready. For example:
NonBlockingHttp.sendRequest("https://example.org/document",
onSuccess = { println("Received document $it") },
onFailure = { Log.e("Failed to fetch the document", it) }
)
With this kind of API no thread will be blocked, whether or not you use coroutines. However, compared to blocking API, its usage is quite unwieldy and messy. This is what coroutines help you with: they allow you to continue writing code in exactly the same shape as if it was blocking, only it's not. To get it, you must first write a suspend fun that translates the API you have into coroutine suspension:
suspend fun fetchDocument(url: String): String = suspendCancellableCoroutine { cont ->
NonBlockingHttp.sendRequest(url,
onSuccess = { cont.resume(it) },
onFailure = { cont.resumeWithException(it) }
)
}
Now your calling code goes back to this:
try {
val document = fetchDocument("https://example.org/document")
println("Received document $document")
} catch (e: Exception) {
Log.e("Failed to fetch the document", e)
}
If, instead, you're fine with keeping your blocking network IO, which means you need a dedicated thread for each concurrent network call, then without coroutines you'd have to use something like an async task, Anko's bg etc. These approaches also require you to supply callbacks, so coroutines can once again help you to keep the natural programming model. The core coroutines library already comes with all the parts you need:
A specialized elastic thread pool that always starts a new thread if all are currently blocked (accessible via Dispatchers.IO)
The withContext primitive, which allows your coroutine to jump from one thread to another and then back
With these tools you can simply write
try {
val document = withContext(Dispatchers.IO) {
JSoup.connect("https://example.org/document").get()
}
println("Received document $it")
} catch (e: Exception) {
Log.e("Failed to fetch the document")
}
When your coroutine arrives at the JSoup call, it will set the UI thread free and execute this line on a thread in the IO thread pool. When it unblocks and gets the result, the coroutine will jump back to the UI thread.

Kotlin Android Coroutines - suspend function doesn't seem to run in the background

I feel like i'm missing some crucial part to my understanding to how this code below is working:
private fun retrieveAndStore() {
launch(UI) {
val service = retrofit.create(AWSService::class.java)
val response = service.retrieveData().await()
store(data = response)
}
}
private suspend fun store(data: JsonData) {
val db = Room.databaseBuilder(applicationContext, AppDatabase::class.java, "app-db").build()
db.appDao().insert(storyData)
}
This is the error i get when run:
java.lang.IllegalStateException: Cannot access database on the main thread since it may potentially lock the UI for a long period of time.
I don't understand why the network code via retrofit works but the store function fails. I'm hoping someone can tell me what's going on?
Interestingly if i wrap db call with async {}.await it works, does that mean coroutines can only call other coroutines?

Coroutines aren't about running in either foreground or background. They are about the ability to get suspended, just like a native thread gets suspended by the OS, but on the level where you are in control of that behavior.
When you say launch(UI) { some code }, you tell Kotlin to submit "some code" as a task to the GUI event loop. It will run on the GUI thread until explicitly suspended; the only difference is that it won't run right away so the next line of code below the launch(UI) block will run before it.
The magic part comes around when your "some code" encounters a suspendCoroutine call: this is where its execution stops and you get a continuation object inside the block you pass to suspendCoroutine. You can do with that object whatever you want, typically store it somewhere and then resume it later on.
Often you don't see the suspendCoroutine call because it's inside the implementation of some suspend fun you're calling, but you can freely implement your own.
One such library function is withContext and it's the one you need to solve your problem. It creates another coroutine with the block you pass it, submits that coroutine to some other context you specify (a useful example is CommonPool) and then suspends the current coroutine until that other one completes. This is exactly what you need to turn a blocking call into a suspendable function.
In your case, it would look like this:
private suspend fun store(data: JsonData) = withContext(CommonPool) {
val db = Room.databaseBuilder(applicationContext, AppDatabase::class.java, "app-db").build()
db.appDao().insert(storyData)
}
I'll also add that you're better off creating your own threadpool instead of relying on the system-wide CommonPool. Refer to this thread for details.