Develop Reference

How to get informed when a coroutine channel finish a job

I have an android app that uses CouchBase lite, I'm trying to save a document and get the acknowledgement using coroutin channel, the reason why I use a channel is to make sure every operation is done on the same scope
here is my try based on the selected answer here
How to properly have a queue of pending operations using Kotlin Coroutines?
object DatabaseQueue {
private val scope = CoroutineScope(IOCoroutineScope)
private val queue = Channel<Job>(Channel.UNLIMITED)
init {
scope.launch(Dispatchers.Default) {
for (job in queue) job.join()
}
}
fun submit(
context: CoroutineContext = EmptyCoroutineContext,
block: suspend CoroutineScope.() -> Unit
) {
val job = scope.launch(context, CoroutineStart.LAZY, block)
queue.trySendBlocking(job)
}
fun submitAsync(
context: CoroutineContext = EmptyCoroutineContext,
id: String,
database: Database
): Deferred<Document?> {
val job = scope.async(context, CoroutineStart.LAZY) {
database.getDocument(id)
}
queue.trySendBlocking(job)
return job
}
fun cancel() {
queue.cancel()
scope.cancel()
}
}
fun Database.saveDocument(document: MutableDocument) {
DatabaseQueue.submit {
Timber.tag("quechk").d("saving :: ${document.id}")
this#saveDocument.save(document)
}
}
fun Database.getDocumentQ(id: String): Document? {
return runBlocking {
DatabaseQueue.submitAsync(id = id, database = this#getDocumentQ).also {
Timber.tag("quechk").d("getting :: $id")
}.await()
}
}
my issue here is that when I have many db operations to write and read the reads are performing faster than the writes which gives me a null results, so,what I need to know is
is this the best way to do it or there is another optimal solution
how can I proccess the job and return the result from the channel in order to avoid the null result

By modifying the original solution you actually made it work improperly. The whole idea was to create an inactive coroutine for each submitted block of code and then start executing these coroutines one by one. In your case you exposed a Deferred to a caller, so the caller is able to start executing a coroutine and as a result, coroutines no longer run sequentially, but concurrently.
The easiest way to fix this while keeping almost the same code would be to introduce another Deferred, which is not directly tight to the queued coroutine:
fun submitAsync(
context: CoroutineContext = EmptyCoroutineContext,
id: String,
database: Database
): Deferred<Document?> {
val ret = CompletableDeferred<Document?>()
val job = scope.launch(context, CoroutineStart.LAZY) {
ret.completeWith(runCatching { database.getDocument(id) })
}
queue.trySendBlocking(job)
return ret
}
However, depending on your case it may be an overkill. For example, if you don't need to guarantee a strict FIFO ordering, a simple Mutex would be enough. Also, please note that classic approach of returning futures/deferreds only to await on them is an anti-pattern in coroutines. We should simply use a suspend function and call it directly.

How to check if coroutines are using the same context?

I have two Coroutines and I want to check if they are running in the same context/Dispatcher. This is a simplified version of my problem, but the answer will apply to the thing I am doing:
#Test
fun test() {
val io = runBlocking(Dispatcher.IO) {
coroutineContext
}
val nonIo = runBlocking() {
coroutineContext
}
assertNotEquals(io, nonIo)
}
However, this is a bad test because I am simply comparing two distinct objects. I want to compare if they use Dispatcher.IO or not.

I think you mean to ask if the the coroutines use the same Dispatcher, right? A dispatcher is just one piece of a CoroutineContext. You can use the get function of a CoroutineContext with the key ContinuationInterceptor to retrieve the Dispatcher.
val io = withContext(Dispatchers.IO) {
coroutineContext[ContinuationInterceptor]
}
val nonIo = withContext(Dispatchers.Default) {
coroutineContext[ContinuationInterceptor]
}
assertNotEquals(io, nonIo)

Coroutine StateFlow stops emitting when CoroutineScope is cancelled

I have a StateFlow coroutine that is shared amongst various parts of my application. When I cancel the CoroutineScope of a downstream collector, a JobCancellationException is propagated up to the StateFlow, and it stops emitting values for all current and future collectors.
The StateFlow:
val songsRelay: Flow<List<Song>> by lazy {
MutableStateFlow<List<Song>?>(null).apply {
CoroutineScope(Dispatchers.IO)
.launch { songDataDao.getAll().distinctUntilChanged().collect { value = it } }
}.filterNotNull()
}
A typical 'presenter' in my code implements the following base class:
abstract class BasePresenter<T : Any> : BaseContract.Presenter<T> {
var view: T? = null
private val job by lazy {
Job()
}
private val coroutineScope by lazy { CoroutineScope( job + Dispatchers.Main) }
override fun bindView(view: T) {
this.view = view
}
override fun unbindView() {
job.cancel()
view = null
}
fun launch(block: suspend CoroutineScope.() -> Unit): Job {
return coroutineScope.launch(block = block)
}
}
A BasePresenter implementation might call launch{ songsRelay.collect {...} }
When the presenter is unbound, in order to prevent leaks, I cancel the parent job. Any time a presenter that was collecting the songsRelay StateFlow is unbound, the StateFlow is essentially terminated with a JobCancellationException, and no other collectors/presenters can collect values from it.
I've noticed that I can call job.cancelChildren() instead, and this seems to work (StateFlow doesn't complete with a JobCancellationException). But then I wonder what the point is of declaring a parent job, if I can't cancel the job itself. I could just remove job altogether, and call coroutineScope.coroutineContext.cancelChildren() to the same effect.
If I do just call job.cancelChildren(), is that sufficient? I feel like by not calling coroutineScope.cancel(), or job.cancel(), I may not be correctly or completely cleaning up the tasks that I have kicked off.
I also don't understand why the JobCancellationException is propagated up the hierarchy when job.cancel() is called. Isn't job the 'parent' here? Why does cancelling it affect my StateFlow?

UPDATE:
Are you sure your songRelay is actually getting cancelled for all presenters? I ran this test and "Song relay completed" is printed, because onCompletion also catches downstream exceptions. However Presenter 2 emits the value 2 just fine, AFTER song relay prints "completed". If I cancel Presenter 2, "Song relay completed" is printed again with a JobCancellationException for Presenter 2's job.
I do find it interesting how the one flow instance will emit once each for each collector subscribed. I didn't realize that about flows.
val songsRelay: Flow<Int> by lazy {
MutableStateFlow<Int?>(null).apply {
CoroutineScope(Dispatchers.IO)
.launch {
flow {
emit(1)
delay(1000)
emit(2)
delay(1000)
emit(3)
}.onCompletion {
println("Dao completed")
}.collect { value = it }
}
}.filterNotNull()
.onCompletion { cause ->
println("Song relay completed: $cause")
}
}
#Test
fun test() = runBlocking {
val job = Job()
val presenterScope1 = CoroutineScope(job + Dispatchers.Unconfined)
val presenterScope2 = CoroutineScope(Job() + Dispatchers.Unconfined)
presenterScope1.launch {
songsRelay.onCompletion { cause ->
println("Presenter 1 Completed: $cause")
}.collect {
println("Presenter 1 emits: $it")
}
}
presenterScope2.launch {
songsRelay.collect {
println("Presenter 2 emits: $it")
}
}
presenterScope1.cancel()
delay(2000)
println("Done test")
}
I think you need to use SupervisorJob in your BasePresenter instead of Job. In general using Job would be a mistake for the whole presenter, because one failed coroutine will cancel all coroutines in the Presenter. Generally not what you want.

OK, so the problem was some false assumptions I made when testing this. The StateFlow is behaving correctly, and cancellation is working as expected.
I was thinking that between Presenters, StateFlow would stop emitting values, but I was actually testing the same instance of a Presenter - so its Job had been cancelled and thus it's not expected to continue collecting Flow emissions.
I also mistakenly took CancellationException messages emitted in onCompletion of the StateFlow to mean the StateFlow itself had been cancelled - when actually it was just saying the downstream Collector/Job had been cancelled.
I've come up with a better implementation of BasePresenter that looks like so:
abstract class BasePresenter<T : Any> : BaseContract.Presenter<T>, CoroutineScope {
var view: T? = null
private var job = Job()
override val coroutineContext: CoroutineContext
get() = job + Dispatchers.Main
override fun bindView(view: T) {
if (job.isCancelled) {
job = Job()
}
this.view = view
}
override fun unbindView() {
job.cancel()
view = null
}
}

Unit testing coroutines runBlockingTest: This job has not completed yet

Please find below a function using a coroutine to replace callback :
override suspend fun signUp(authentication: Authentication): AuthenticationError {
return suspendCancellableCoroutine {
auth.createUserWithEmailAndPassword(authentication.email, authentication.password)
.addOnCompleteListener(activityLifeCycleService.getActivity()) { task ->
if (task.isSuccessful) {
it.resume(AuthenticationError.SignUpSuccess)
} else {
Log.w(this.javaClass.name, "createUserWithEmail:failure", task.exception)
it.resume(AuthenticationError.SignUpFail)
}
}
}
}
Now I would like to unit testing this function. I am using Mockk :
#Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() = runBlockingTest {
val listener = slot<OnCompleteListener<AuthResult>>()
val authentication = mockk<Authentication> {
every { email } returns "email"
every { password } returns "pswd"
}
val task = mockk<Task<AuthResult>> {
every { isSuccessful } returns true
}
every { auth.createUserWithEmailAndPassword("email", "pswd") } returns
mockk {
every { addOnCompleteListener(activity, capture(listener)) } returns mockk()
}
service.signUp(authentication)
listener.captured.onComplete(task)
}
Unfortunately this test failed due to the following exception : java.lang.IllegalStateException: This job has not completed yet
I tried to replace runBlockingTest with runBlocking but the test seems to wait in an infinite loop.
Can someone help me with this UT please?
Thanks in advance

As can be seen in this post:
This exception usually means that some coroutines from your tests were scheduled outside the test scope (more specifically the test dispatcher).
Instead of performing this:
private val networkContext: CoroutineContext = TestCoroutineDispatcher()
private val sut = Foo(
networkContext,
someInteractor
)
fun `some test`() = runBlockingTest() {
// given
...
// when
sut.foo()
// then
...
}
Create a test scope passing test dispatcher:
private val testDispatcher = TestCoroutineDispatcher()
private val testScope = TestCoroutineScope(testDispatcher)
private val networkContext: CoroutineContext = testDispatcher
private val sut = Foo(
networkContext,
someInteractor
)
Then in test perform testScope.runBlockingTest
fun `some test`() = testScope.runBlockingTest {
...
}
See also Craig Russell's "Unit Testing Coroutine Suspend Functions using TestCoroutineDispatcher"

In case of Flow testing:
Don't use flow.collect directly inside runBlockingTest. It should be wrapped in launch
Don't forget to cancel TestCoroutineScope in the end of a test. It will stop a Flow collecting.
Example:
class CoroutinesPlayground {
private val job = Job()
private val testDispatcher = StandardTestDispatcher()
private val testScope = TestScope(job + testDispatcher)
#Test
fun `play with coroutines here`() = testScope.runBlockingTest {
val flow = MutableSharedFlow<Int>()
launch {
flow.collect { value ->
println("Value: $value")
}
}
launch {
repeat(10) { value ->
flow.emit(value)
delay(1000)
}
job.cancel()
}
}
}

This is not an official solution, so use it at your own risk.
This is similar to what #azizbekian posted, but instead of calling runBlocking, you call launch.
As this is using TestCoroutineDispatcher, any tasks scheduled to be run without delay are immediately executed. This might not be suitable if you have several tasks running asynchronously.
It might not be suitable for every case but I hope that it helps for simple cases.
You can also follow up on this issue here:
https://github.com/Kotlin/kotlinx.coroutines/issues/1204
If you know how to solve this using the already existing runBlockingTest and runBlocking, please be so kind and share with the community.
class MyTest {
private val dispatcher = TestCoroutineDispatcher()
private val testScope = TestCoroutineScope(dispatcher)
#Test
fun myTest {
val apiService = mockk<ApiService>()
val repository = MyRepository(apiService)
testScope.launch {
repository.someSuspendedFunction()
}
verify { apiService.expectedFunctionToBeCalled() }
}
}

According to my understanding, this exception occurs when you are using a different dispatcher in your code inside the runBlockingTest { } block with the one that started runBlockingTest { }.
So in order to avoid this, you first have to make sure you inject Dispatchers in your code, instead of hardcoding it throughout your app. If you haven't done it, there's nowhere to begin because you cannot assign a test dispatcher to your test codes.
Then, in your BaseUnitTest, you should have something like this:
#get:Rule
val coroutineRule = CoroutineTestRule()
#ExperimentalCoroutinesApi
class CoroutineTestRule(
val testDispatcher: TestCoroutineDispatcher = TestCoroutineDispatcher()
) : TestWatcher() {
override fun finished(description: Description?) {
super.finished(description)
Dispatchers.setMain(testDispatcher)
}
override fun starting(description: Description?) {
super.starting(description)
Dispatchers.resetMain()
testDispatcher.cleanupTestCoroutines()
}
}
Next step really depends on how you do Depedency Injection. The main point is to make sure your test codes are using coroutineRule.testDispatcher after the injection.
Finally, call runBlockingTest { } from this testDispatcher:
#Test
fun `This should pass`() = coroutineRule.testDispatcher.runBlockingTest {
//Your test code where dispatcher is injected
}

There is an open issue for this problem: https://github.com/Kotlin/kotlinx.coroutines/issues/1204
The solution is to use the CoroutineScope intead of the TestCoroutinScope until the issue is resolved, you can do by replacing
#Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() =
runBlockingTest {
with
#Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() =
runBlocking {

None of these answers quite worked for my setup due to frequent changes in the coroutines API.
This specifically works using version 1.6.0 of kotlin-coroutines-test, added as a testImplementation dependency.
#Test
fun `test my function causes flow emission`() = runTest {
// calling this function will result in my flow emitting a value
viewModel.myPublicFunction("1234")
val job = launch {
// Force my flow to update via collect invocation
viewModel.myMemberFlow.collect()
}
// immediately cancel job
job.cancel()
assertEquals("1234", viewModel.myMemberFlow.value)
}

If you have any
Channel
inside the launch, you must call to
Channel.close()
Example code:
val channel = Channel<Success<Any>>()
val flow = channel.consumeAsFlow()
launch {
channel.send(Success(Any()))
channel.close()
}

runBlockingTest deprecated since 1.6.0 and replaced with runTest.

You need to swap arch background executor with one that execute tasks synchronously. eg. For room suspend functions, live data etc.
You need the following dependency for core testing
androidTestImplementation 'androidx.arch.core:core-testing:2.1.0'
Then add the following at the top of test class
#get:Rule
val instantExecutor = InstantTaskExecutorRule()
Explanations
InstantTaskExecutorRule A JUnit Test Rule that swaps the background executor used by the
Architecture Components with a different one which executes each task
synchronously.
You can use this rule for your host side tests that use Architecture
Components

As I mentioned here about fixing runBlockingTest, maybe it could help you too.
Add this dependency if you don't have it
testImplementation "androidx.arch.core:core-testing:$versions.testCoreTesting" (2.1.0)
Then in your test class declare InstantTaskExecutorRule rule:
#get:Rule
val instantTaskExecutorRule = InstantTaskExecutorRule()

How do I test Kotlin coroutines inside a function?

I am creating a library and I'm using Retrofit with a call-adapter that gives me a Deferred<> value.
In a function in my code I call launch {}, and inside that i try-catch the values, and possible exceptions - calling different callbacks for different results.
The resources I've found on testing coroutines are all about testing suspended functions, and runBlocking {} is the solution to everything. Except for me it isn't
I made a quick example
#Mock
val mockListener: DoSomething.Listener = mock()
#Test
fun testSomething() {
val doer = DoSomething(mockListener)
runBlocking {
doer.doIt()
verify(mockListener).listen(any())
}
}
class DoSomething(val listener: Listener) {
interface Listener {
fun listen(s: String)
}
fun doIt() {
launch {
listener.listen(theThing().await())
}
}
private fun theThing(): Deferred<String> {
return async {
delay(5, TimeUnit.SECONDS)
return#async "Wow, a thing"
}
}
}
What I want is for the actually run all functions. The test should take 5 seconds minimum, but it just runs through the code in a couple of millisconds- ie. it doesn't block.
I've tried adding
runBlocking {
launch {
// doer.doIt()
}.joinChildren()
}
And similar practices but I just can't get the test to actually wait for my launch inside of another class to finish before the test is finished.
Placing the verify(...) outside of the runBlocking also makes the test fail, which it should.
Any input, helpers, good practice etc. is appreciated!

You can provide the CoroutineContext explicitly for your doIt() function:
fun doIt(context: CoroutineContext = DefaultDispatcher) {
launch(context) {
listener.listen(theThing().await()
}
}
With this parameter you could easily change the coroutine context - in your test code you use the blocking context:
runBlocking {
doer.doIt(coroutineContext)
}
BTW: You don't need to use launch and async. With launch you are in a suspendable context and you don't need to run theThing() asynchronously. Especially if you invoke await() in the next step:
fun doIt(context: CoroutineContext = DefaultDispatcher) {
launch(context) {
listener.listen(theThing())
}
}
private suspend fun theThing(): String {
delay(5, TimeUnit.SECONDS)
return "Wow, a thing"
}

Best way would be not to swallow Job in your doIt() function as you do now.
Instead of
fun doIt() {
launch {
listener.listen(theThing().await())
}
}
Do
fun doIt() = launch {
listener.listen(theThing().await())
}
That way your function will return a coroutine, which you can wait for:
doIt().join()
Better still is to use async() instead of launch()
Another comment is that doIt() should be actually doItAsync(), as suggested by Kotlin guidelines.