Develop Reference

kotlin, how to run sequential background threads [duplicate]

I have an instance of CoroutineScope and log() function which look like the following:
private val scope = CoroutineScope(Dispatchers.IO)
fun log(message: String) = scope.launch { // launching a coroutine
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // some blocking operation
}
And I use this test code to launch coroutines:
repeat(5) { item ->
log("Log $item")
}
The log() function can be called from any place, in any Thread, but not from a coroutine.
After a couple of tests I can see not sequential result like the following:
Log 0
Log 2
Log 4
Log 1
Log 3
There can be different order of printed logs. If I understand correctly the execution of coroutines doesn't guarantee to be sequential. What it means is that a coroutine for item 2 can be launched before the coroutine for item 0.
I want that coroutines were launched sequentially for each item and "some blocking operation" would execute sequentially, to always achieve next logs:
Log 0
Log 1
Log 2
Log 3
Log 4
Is there a way to make launching coroutines sequential? Or maybe there are other ways to achieve what I want?
Thanks in advance for any help!

One possible strategy is to use a Channel to join the launched jobs in order. You need to launch the jobs lazily so they don't start until join is called on them. trySend always succeeds when the Channel has unlimited capacity. You need to use trySend so it can be called from outside a coroutine.
private val lazyJobChannel = Channel<Job>(capacity = Channel.UNLIMITED).apply {
scope.launch {
consumeEach { it.join() }
}
}
fun log(message: String) {
lazyJobChannel.trySend(
scope.launch(start = CoroutineStart.LAZY) {
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // some blocking operation
}
)
}

Since Flows are sequential we can use MutableSharedFlow to collect and handle data sequentially:
class Info {
// make sure replay(in case some jobs were emitted before sharedFlow is being collected and could be lost)
// and extraBufferCapacity are large enough to handle all the jobs.
// In case some jobs are lost try to increase either of the values.
private val sharedFlow = MutableSharedFlow<String>(replay = 10, extraBufferCapacity = 10)
private val scope = CoroutineScope(Dispatchers.IO)
init {
sharedFlow.onEach { message ->
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // some blocking or suspend operation
}.launchIn(scope)
}
fun log(message: String) {
sharedFlow.tryEmit(message)
}
}
fun test() {
val info = Info()
repeat(10) { item ->
info.log("Log $item")
}
}
It always prints the logs in the correct order:
Log 0
Log 1
Log 2
...
Log 9
It works for all cases, but need to be sure there are enough elements set to replay and extraBufferCapacity parameters of MutableSharedFlow to handle all items.
Another approach is
Using Dispatchers.IO.limitedParallelism(1) as a context for the CoroutineScope. It makes coroutines run sequentially if they don't contain calls to suspend functions and launched from the same Thread, e.g. Main Thread. So this solution works only with blocking (not suspend) operation inside launch coroutine builder:
private val scope = CoroutineScope(Dispatchers.IO.limitedParallelism(1))
fun log(message: String) = scope.launch { // launching a coroutine from the same Thread, e.g. Main Thread
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // only blocking operation, not `suspend` operation
}
It turns out that the single thread dispatcher is a FIFO executor. So limiting the CoroutineScope execution to one thread solves the problem.

How to run Kotlin coroutines sequentially?

I have an instance of CoroutineScope and log() function which look like the following:
private val scope = CoroutineScope(Dispatchers.IO)
fun log(message: String) = scope.launch { // launching a coroutine
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // some blocking operation
}
And I use this test code to launch coroutines:
repeat(5) { item ->
log("Log $item")
}
The log() function can be called from any place, in any Thread, but not from a coroutine.
After a couple of tests I can see not sequential result like the following:
Log 0
Log 2
Log 4
Log 1
Log 3
There can be different order of printed logs. If I understand correctly the execution of coroutines doesn't guarantee to be sequential. What it means is that a coroutine for item 2 can be launched before the coroutine for item 0.
I want that coroutines were launched sequentially for each item and "some blocking operation" would execute sequentially, to always achieve next logs:
Log 0
Log 1
Log 2
Log 3
Log 4
Is there a way to make launching coroutines sequential? Or maybe there are other ways to achieve what I want?
Thanks in advance for any help!

One possible strategy is to use a Channel to join the launched jobs in order. You need to launch the jobs lazily so they don't start until join is called on them. trySend always succeeds when the Channel has unlimited capacity. You need to use trySend so it can be called from outside a coroutine.
private val lazyJobChannel = Channel<Job>(capacity = Channel.UNLIMITED).apply {
scope.launch {
consumeEach { it.join() }
}
}
fun log(message: String) {
lazyJobChannel.trySend(
scope.launch(start = CoroutineStart.LAZY) {
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // some blocking operation
}
)
}

Since Flows are sequential we can use MutableSharedFlow to collect and handle data sequentially:
class Info {
// make sure replay(in case some jobs were emitted before sharedFlow is being collected and could be lost)
// and extraBufferCapacity are large enough to handle all the jobs.
// In case some jobs are lost try to increase either of the values.
private val sharedFlow = MutableSharedFlow<String>(replay = 10, extraBufferCapacity = 10)
private val scope = CoroutineScope(Dispatchers.IO)
init {
sharedFlow.onEach { message ->
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // some blocking or suspend operation
}.launchIn(scope)
}
fun log(message: String) {
sharedFlow.tryEmit(message)
}
}
fun test() {
val info = Info()
repeat(10) { item ->
info.log("Log $item")
}
}
It always prints the logs in the correct order:
Log 0
Log 1
Log 2
...
Log 9
It works for all cases, but need to be sure there are enough elements set to replay and extraBufferCapacity parameters of MutableSharedFlow to handle all items.
Another approach is
Using Dispatchers.IO.limitedParallelism(1) as a context for the CoroutineScope. It makes coroutines run sequentially if they don't contain calls to suspend functions and launched from the same Thread, e.g. Main Thread. So this solution works only with blocking (not suspend) operation inside launch coroutine builder:
private val scope = CoroutineScope(Dispatchers.IO.limitedParallelism(1))
fun log(message: String) = scope.launch { // launching a coroutine from the same Thread, e.g. Main Thread
println("$message")
TimeUnit.MILLISECONDS.sleep(100) // only blocking operation, not `suspend` operation
}
It turns out that the single thread dispatcher is a FIFO executor. So limiting the CoroutineScope execution to one thread solves the problem.

Kotlin Flow unit testing exceptions and job with FlowOn with different Dispatchers

Using the code below to test exceptions with Flow and MockK
#Test
fun given network error returned from repos, should throw exception() =
testCoroutineScope.runBlockingTest {
// GIVEN
every { postRemoteRepository.getPostFlow() } returns flow<List<PostDTO>> {
emit(throw Exception("Network Exception"))
}
// WHEN
var expected: Exception? = null
useCase.getPostFlow()
.catch { throwable: Throwable ->
expected = throwable as Exception
println("⏰ Expected: $expected")
}
.launchIn(testCoroutineScope)
// THEN
println("⏰ TEST THEN")
Truth.assertThat(expected).isNotNull()
Truth.assertThat(expected).isInstanceOf(Exception::class.java)
Truth.assertThat(expected?.message).isEqualTo("Network Exception")
}
And prints
⏰ TEST THEN
⏰ Expected: java.lang.Exception: Network Exception
And the test fails with
expected not to be: null
Method i test is
fun getPostFlow(): Flow<List<Post>> {
return postRemoteRepository.getPostFlow()
// 🔥 This method is just to show flowOn below changes current thread
.map {
// Runs in IO Thread DefaultDispatcher-worker-2
println("⏰ PostsUseCase map() FIRST thread: ${Thread.currentThread().name}")
it
}
.flowOn(Dispatchers.IO)
.map {
// Runs in Default Thread DefaultDispatcher-worker-1
println("⏰ PostsUseCase map() thread: ${Thread.currentThread().name}")
mapper.map(it)
}
// This is a upstream operator, does not leak downstream
.flowOn(Dispatchers.Default)
}
This is not a completely practical function but just to check how Dispatchers work with Flow and tests.
At the time of writing the question i commented out .flowOn(Dispatchers.IO), the one on top
the test passed. Also changing Dispatchers.IO to Dispatchers.Default and caused test to pass. I assume it's due to using different threads.
1- Is there a function or way to set all flowOn methods to same thread without modifying the code?
I tried testing success scenario this time with
#Test
fun `given data returned from api, should have data`() = testCoroutineScope.runBlockingTest {
// GIVEN
coEvery { postRemoteRepository.getPostFlow() } returns flow { emit(postDTOs) }
every { dtoToPostMapper.map(postDTOs) } returns postList
val actual = postList
// WHEN
val expected = mutableListOf<Post>()
// useCase.getPostFlow().collect {postList->
// println("⏰ Collect: ${postList.size}")
//
// expected.addAll(postList)
// }
val job = useCase.getPostFlow()
.onEach { postList ->
println("⏰ Collect: ${postList.size}")
expected.addAll(postList)
}
.launchIn(testCoroutineScope)
job.cancel()
// THEN
println("⏰ TEST THEN")
Truth.assertThat(expected).containsExactlyElementsIn(actual)
}
When i tried to snippet that is commented out, test fails with
java.lang.IllegalStateException: This job has not completed yet
It seems that test could pass if job has been canceled, the one with launchIn that returns job passes.
2- Why with collect job is not canceled, and it only happens when the first flowOn uses Dispatchers.IO?

Coroutine Unit Testing Fails for the Class

I am facing a weird issue while unit testing Coroutines. There are two tests on the class, when run individually, they both pass and when I run the complete test class, one fails with assertion error.
I am using MainCoroutineRule to use the TestCoroutineScope and relying on the latest Coroutine Testing Library
Here is the test :
#Test
fun testHomeIsLoadedWithShowsAndFavorites() {
runBlocking {
// Stubbing network and repository calls
whenever(tvMazeApi.getCurrentSchedule("US", currentDate))
.thenReturn(getFakeEpisodeList())
whenever(favoriteShowsRepository.allFavoriteShowIds())
.thenReturn(arrayListOf(1, 2))
}
mainCoroutineRule.runBlockingTest {
// call home viewmodel
homeViewModel.onScreenCreated()
// Check if loader is shown
assertThat(LiveDataTestUtil.getValue(homeViewModel.getHomeViewState())).isEqualTo(Loading)
// Observe on home view state live data
val homeViewState = LiveDataTestUtil.getValue(homeViewModel.getHomeViewState())
// Check for success data
assertThat(homeViewState is Success).isTrue()
val homeViewData = (homeViewState as Success).homeViewData
assertThat(homeViewData.episodes).isNotEmpty()
// compare the response with fake list
assertThat(homeViewData.episodes).hasSize(getFakeEpisodeList().size)
// compare the data and also order
assertThat(homeViewData.episodes).containsExactlyElementsIn(getFakeEpisodeViewDataList(true)).inOrder()
}
}
The other test is almost similar which tests for Shows without favorites. I am trying to test HomeViewModel method as:
homeViewStateLiveData.value = Loading
val coroutineExceptionHandler = CoroutineExceptionHandler { _, exception ->
onError(exception)
}
viewModelScope.launch(coroutineExceptionHandler) {
// Get shows from network and favorites from room db on background thread
val favoriteShowsWithFavorites = withContext(Dispatchers.IO) {
val favoriteShowIds = favoriteShowsRepository.allFavoriteShowIds()
val episodes = tvMazeApi.getCurrentSchedule(COUNTRY_US, currentDate)
getShowsWithFavorites(episodes, favoriteShowIds)
}
// Return the combined result on main thread
withContext(Dispatchers.Main) {
onSuccess(favoriteShowsWithFavorites)
}
}
}
I cannot find the actual cause of why the tests if run separately are passing and when the complete class is tested, one of them is failing. Pls help if I am missing something

Retrofit and Room that come with Coroutine support owner the suspend functions and move them off the UI thread by their own. Thus, they reduce the hassles of handling thread callbacks by the developers in a big way. Initially, I was moving the suspend calls of network and DB to IO via Dispatchers.IO explicitly. This was unnecessary and also leading unwanted context-switching leading to flaky test. Since the libraries, automatically do it, it was just about handling the data back on UI when available.
viewModelScope.launch(coroutineExceptionHandler) {
// Get favorite shows from db, suspend function in room will launch a new coroutine with IO dispatcher
val favoriteShowIds = favoriteShowsRepository.allFavoriteShowIds()
// Get shows from network, suspend function in retrofit will launch a new coroutine with IO dispatcher
val episodes = tvMazeApi.getCurrentSchedule(COUNTRY_US, currentDate)
// Return the result on main thread via Dispatchers.Main
homeViewStateLiveData.value = Success(HomeViewData(getShowsWithFavorites(episodes, favoriteShowIds)))
}

Kotlin coroutines using produces and mockito to mock the producing job

I am testing Kotlin coroutines in my Android app and I am trying to do the following unit test
#Test fun `When getVenues success calls explore venues net controller and forwards result to listener`() =
runBlocking {
val near = "Barcelona"
val result = buildMockVenues()
val producerJob = produce<List<VenueModel>>(coroutineContext) { result.value }
whenever(venuesRepository.getVenues(eq(near))) doReturn producerJob // produce corooutine called inside interactor.getVenues(..)
interactor.getVenues(near, success, error) // call to real method
verify(venuesRepository).getVenues(eq(near))
verify(success).invoke(argThat {
value == result.value
})
}
The interactor method is as follows
fun getVenues(near: String, success: Callback<GetVenuesResult>,
error: Callback<GetVenuesResult>) =
postExecute {
repository.getVenues(near).consumeEach { venues ->
if (venues.isEmpty()) {
error(GetVenuesResult(venues, Throwable("No venues where found")))
} else {
success(GetVenuesResult(venues))
}
}
}
postExecute{..} is a method on a BaseInteractor that executes the function in the ui thread through a custom Executor that uses the launch(UI) coroutine from kotlin android coroutines library
fun <T> postExecute(uiFun: suspend () -> T) =
executor.ui(uiFun)
Then the repository.getVenues(..) function is also a coroutine that returns the ProducerJob using produce(CommonPool) {}
The problem is that it seams that success callback in the interactor function doesn't seem to be executed as per the
verify(success).invoke(argThat {
value == result.value
})
However, I do see while debugging that the execution in the interactor function reaches to the if (venues.isEmpty()) line inside the consumeEach but then from there exits and continues with the test, obviously failing on the verify for the success callback.
I am a bit new on coroutines so any help would be appreciated.

I figured this one out. I saw that the problem was just with this producing coroutine and not with the others tests that are also using coroutines and working just fine. I noticed that I actually missed the send on the mocked ProducingJob in order to have it actually produce a value, in this case the list of mocks. I just added that changing the mock of the producing job to
val producerJob = produce { send(result.value) }