Develop Reference

How to define a dependency between two coroutines?

Consider the following code:
init {
// coroutine 1
// this is and needs to be in a separate coroutine as the collection runs indefinite
viewModelScope.launch {
myService.someSharedFlow.collect {
// handle values
}
}
// coroutine 2
viewModelScope.launch {
// this shall not be executed before the subscription to the SharedFlow in coroutine 1 is set up
// to make sure I don't miss any emitted values
withContext(Dispatchers.IO) {
myService.initialize() // will send a value through the flow after initialization
}
}
}
How can I let coroutine 2 wait until the subscription to the SharedFlow in coroutine 1 is set up?

If you want to wait for a collector to subscribe on your flow before pushing values in it, I see two solutions:
If you have a MutableSharedFlow, you can use subscriptionCount property to know if your flow has been subscribed to.
As #broot said, there's also the possibility to use onSubscription on a SharedFlow. This does not require to expose mutable API. However, if there can be multiple collectors, and you want to ensure initialization is triggered only once, you will have to add some manual checks over it.
Here is a sample program that show how to use both solutions:
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.*
import kotlin.time.Duration.Companion.seconds
fun main() : Unit = runBlocking {
val flow = MutableSharedFlow<Int>()
println(
"""
Solution 1: subscriptionCount
-----------------------------
""".trimIndent())
waitForSubsciptionCount(flow)
println(
"""
Solution 2: onSubscription
-----------------------------
""".trimIndent())
reactToSubscription(flow)
}
suspend fun CoroutineScope.waitForSubsciptionCount(flow: MutableSharedFlow<Int>) {
val collectJob = collectAfterDelay(flow)
println("Waiting for subscription")
flow.subscriptionCount.filter { it > 0 }.first()
println("Subscription detected")
flow.initialize()
// Specific to this test program, to avoid it to hang indefinitely
collectJob.cancel()
}
suspend fun CoroutineScope.reactToSubscription(flow: SharedFlow<Int>) {
println("Waiting for subscription")
// This is optional. Helps to detect "end" of the flow.
val initialized = MutableStateFlow(false)
val initializableFlow = flow.onSubscription {
println("Subscription detected")
initialize()
initialized.emit(true)
}
val collectJob = collectAfterDelay(initializableFlow)
// Specific to this test program, to avoid it to indefinitely
initialized.filter { it == true }.first()
collectJob.cancel()
}
private fun CoroutineScope.collectAfterDelay(flow: Flow<Int>) = launch {
delay(1.seconds)
print("Start collecting")
flow.collect { println("collected: $it") }
}
suspend fun FlowCollector<Int>.initialize() {
for (i in 1..3) {
emit(i)
println("Emitted $i")
}
}
The output is:
Solution 1: subscriptionCount
-----------------------------
Waiting for subscription
Start collectingSubscription detected
collected: 1
Emitted 1
collected: 2
Emitted 2
collected: 3
Emitted 3
Solution 2: onSubscription
-----------------------------
Waiting for subscription
Start collectingSubscription detected
collected: 1
Emitted 1
collected: 2
Emitted 2
collected: 3
Emitted 3
NOTE: Both solutions above should also allow you to move initialization trigger inside your service, and hide it completely from consumers. This way, code calling your service would not require to mangle with your service initialization.
EDIT: Here is an example of a service that triggers flow emission after a subscriber start collecting:
class SubscriptionCountService {
private val _flow = MutableSharedFlow<Int>()
public val flow = _flow.asSharedFlow()
init {
_flow.subscriptionCount.filter { it > 0 }
.take(1)
.onCompletion {
println("First subscription detected: initialize")
_flow.initialize()
}
.launchIn(CoroutineScope(Dispatchers.IO))
}
}
fun main() : Unit = runBlocking {
val flow = SubscriptionCountService().flow
launch {
delay(1.seconds)
println("Start collecting")
// Collect a limited number of entries to terminate main program easily
flow.take(3)
.collect { println("collected: $it") }
}
}
The output is:
Start collecting
collected: 1
Emitted 1
collected: 2
Emitted 2
collected: 3
Emitted 3

Kotlin Coroutine Flow: When does wasting resource happen when using Flow

I am reading this article to fully understand the dos and donts of using Flow while comparing it to my implementation, but I can't grasp clearly how to tell if you are wasting resource when using Flow or flow builder. When is the time a flow is being release/freed in memory and when is the time that you are wasting resource like accidentally creating multiple instances of flow and not releasing them?
I have a UseCase class that invokes a repository function that returns Flow. In my ViewModel this is how it looks like.
class AssetViewModel constructor(private val getAssetsUseCase: GetAssetsUseCase) : BaseViewModel() {
private var job: Job? = null
private val _assetState = defaultMutableSharedFlow<AssetState>()
fun getAssetState() = _assetState.asSharedFlow()
init {
job = viewModelScope.launch {
while(true) {
if (lifecycleState == LifeCycleState.ON_START || lifecycleState == LifeCycleState.ON_RESUME)
fetchAssets()
delay(10_000)
}
}
}
fun fetchAssets() {
viewModelScope.launch {
withContext(Dispatchers.IO) {
getAssetsUseCase(
AppConfigs.ASSET_BASE_URL,
AppConfigs.ASSET_PARAMS,
AppConfigs.ASSET_SIZES[AppConfigs.ASSET_LIMIT_INDEX]
).onEach {
when(it){
is RequestStatus.Loading -> {
_assetState.tryEmit(AssetState.FetchLoading)
}
is RequestStatus.Success -> {
_assetState.tryEmit(AssetState.FetchSuccess(it.data.assetDataDomain))
}
is RequestStatus.Failed -> {
_assetState.tryEmit(AssetState.FetchFailed(it.message))
}
}
}.collect()
}
}
}
override fun onCleared() {
job?.cancel()
super.onCleared()
}
}
The idea here is we are fetching data from remote every 10 seconds while also allowing on demand fetch of data via UI.
Just a typical useless UseCase class
class GetAssetsUseCase #Inject constructor(
private val repository: AssetsRepository // Passing interface not implementation for fake test
) {
operator fun invoke(baseUrl: String, query: String, limit: String): Flow<RequestStatus<AssetDomain>> {
return repository.fetchAssets(baseUrl, query, limit)
}
}
The concrete implementation of repository
class AssetsRepositoryImpl constructor(
private val service: CryptoService,
private val mapper: AssetDtoMapper
) : AssetsRepository {
override fun fetchAssets(
baseUrl: String,
query: String,
limit: String
) = flow {
try {
emit(RequestStatus.Loading())
val domainModel = mapper.mapToDomainModel(
service.getAssetItems(
baseUrl,
query,
limit
)
)
emit(RequestStatus.Success(domainModel))
} catch (e: HttpException) {
emit(RequestStatus.Failed(e))
} catch (e: IOException) {
emit(RequestStatus.Failed(e))
}
}
}
After reading this article which says that using stateIn or sharedIn will improve the performance when using a flow, it seems that I am creating new instances of the same flow on-demand. But there is a limitation as the stated approach only works for variable and not function that returns Flow.

stateIn and shareIn can save resources if there are multiple observers, by avoiding redundant fetching. And in your case, you could set it up to automatically pause the automatic re-fetching when there are no observers. If, on the UI side you use repeatOnLifecycle, then it will automatically drop your observers when the view is off screen and then you will avoid wasted fetches the user will never see.
I think it’s not often described this way, but often the multiple observers are just observers coming from the same Activity or Fragment class after screen rotations or rapidly switching between fragments. If you use WhileSubscribed with a timeout to account for this, you can avoid having to restart your flow if it’s needed again quickly.
Currently you emit to from an external coroutine instead of using shareIn, so there’s no opportunity to pause execution.
I haven't tried to create something that supports both automatic and manual refetching. Here's a possible strategy, but I haven't tested it.
private val refreshRequest = Channel<Unit>(Channel.CONFLATED)
fun fetchAssets() {
refreshRequest.trySend(Unit)
}
val assetState = flow {
while(true) {
getAssetsUseCase(
AppConfigs.ASSET_BASE_URL,
AppConfigs.ASSET_PARAMS,
AppConfigs.ASSET_SIZES[AppConfigs.ASSET_LIMIT_INDEX]
).map {
when(it){
is RequestStatus.Loading -> AssetState.FetchLoading
is RequestStatus.Success -> AssetState.FetchSuccess(it.data.assetDataDomain)
is RequestStatus.Failed -> AssetState.FetchFailed(it.message)
}
}.emitAll()
withTimeoutOrNull(100L) {
// drop any immediate or pending manual request
refreshRequest.receive()
}
// Wait until a fetch is manually requested or ten seconds pass:
withTimeoutOrNull(10000L - 100L) {
refreshRequest.receive()
}
}
}.shareIn(viewModelScope, SharingStarted.WhileSubscribed(4000L), replay = 1)

To this I would recommend not using flow as the return type of the usecase function and the api call must not be wrapped inside a flow builder.
Why:
The api call actually is happening once and then again after an interval it is triggered by the view model itself, returning flow from the api caller function will be a bad usage of powerful tool that is actually meant to be called once and then it must be self-reliant, it should emit or pump in the data till the moment it has a subscriber/collector.
One usecase you can consider when using flow as return type from the room db query call, it is called only once and then the room emits data into it till the time it has subscriber.
.....
fun fetchAssets() {
viewModelScope.launch {
// loading true
val result=getusecase(.....)
when(result){..process result and emit on state..}
// loading false
}
}
.....
suspend operator fun invoke(....):RequestStatus<AssetDomain>{
repository.fetchAssets(baseUrl, query, limit)
}
.....
override fun fetchAssets(
baseUrl: String,
query: String,
limit: String
):RequestStatus {
try {
//RequestStatus.Loading()//this can be managed in viewmodel itself
val domainModel = mapper.mapToDomainModel(
service.getAssetItems(
baseUrl,
query,
limit
)
)
RequestStatus.Success(domainModel)
} catch (e: HttpException) {
RequestStatus.Failed(e)
} catch (e: IOException) {
RequestStatus.Failed(e)
}
}

Unit test RxJava with timeout not subscribe

I have Unit test function RxJava with timeout but it doesn't subscribe for unit test.
Function on viewModel
fun loadData() {
loadDataUseCase.loadData(true)
.subscribeOn(Schedulers.io())
.timeout(30L, TimeUnit.SECONDS, schedulers)
.observeOn(AndroidSchedulers.mainThread())
.doOnSubscribe {
onShowLoading.value = true
onShowError.value = false
onShowContent.value = false
}.subscribe(
{
onConnected.value = true
onShowContent.value = true
onShowError.value = false
onShowLoading.value = false
},
{
onShowError.value = true
onShowLoading.value = false
onShowContent.value = false
}
)
.addTo(compositeDisposable)
}
Function on unit test
#Test
fun `Load data is success`() {
// given
whenever(loadDataUseCase.loadData(true)).thenReturn(Observable.just(true))
// when
viewModel.loadData()
// then
viewModel.onShowError().test().assertValue(false).awaitNextValue().assertValue(false)
}
I try to debug this function but it doesn't invoke subscribe

I am a bit confused, what is onShowError() does it return a LiveData?
If I run the same code the test doesn't even finish (well I use only io dispatchers and postValue), for you it might be finished before the subscription even happens:
Since you rely on Schedulers.io() it is possible that your whole Subscription is finished before you get to even test your LiveData.
An other option is that your LiveData already has a false value: .assertValue(false). then the next .doOnSubscribe setting already triggers .awaitNextValue() and your whole test finishes, before the subscription can even be called.
Your tests should be fixed and not dependent on timing. Or if it is unavoidable then you have to synchronize your test somehow, an example of this is here:
#Timeout(1, unit = TimeUnit.SECONDS)
#Test
fun `Load data is success`() {
// given
whenever(loadDataUseCase.loadData(true)).thenReturn(Observable.just(true)
val testObserver = liveData.test()
testObserver.assertNoValue() // assert in correct state before actions
// when
loadData(liveData, mock)
//then
testObserver.awaitValueHistorySize(2)
.assertValueHistory(false, false)
}
fun <T> TestObserver<T>.awaitValueHistorySize(count: Int, delay: Long = 10): TestObserver<T> {
awaitValue()
while (valueHistory().size < count) {
// we need to recheck and don't block, the value we are trying to wait might already arrived between the while condition and the awaitNextValue in the next line, so we use some delay instead.
awaitNextValue(delay, TimeUnit.MILLISECONDS)
}
return this
}

How do you make make a subscriber to a kotlin sharedflow run operations in parallel?

I have a connection to a Bluetooth device that emits data every 250ms
In my viewmodel I wish to subscribe to said data , run some suspending code (which takes approximatelly 1000ms to run) and then present the result.
the following is a simple example of what I'm trying to do
Repository:
class Repo() : CoroutineScope {
private val supervisor = SupervisorJob()
override val coroutineContext: CoroutineContext = supervisor + Dispatchers.Default
private val _dataFlow = MutableSharedFlow<Int>()
private var dataJob: Job? = null
val dataFlow: Flow<Int> = _dataFlow
init {
launch {
var counter = 0
while (true) {
counter++
Log.d("Repo", "emmitting $counter")
_dataFlow.emit(counter)
delay(250)
}
}
}
}
the viewmodel
class VM(app:Application):AndroidViewModel(app) {
private val _reading = MutableLiveData<String>()
val latestReading :LiveData<String>() = _reading
init {
viewModelScope.launch(Dispatchers.Main) {
repo.dataFlow
.map {
validateData() //this is where some validation happens it is very fast
}
.flowOn(Dispatchers.Default)
.forEach {
delay(1000) //this is to simulate the work that is done,
}
.flowOn(Dispatchers.IO)
.map {
transformData() //this will transform the data to be human readable
}
.flowOn(Dispatchers.Default)
.collect {
_reading.postValue(it)
}
}
}
}
as you can see, when data comes, first I validate it to make sure it is not corrupt (on Default dispatcher) then I perform some operation on it (saving and running a long algorithm that takes time on the IO dispatcher) then I change it so the application user can understand it (switching back to Default dispatcher) then I post it to mutable live data so if there is a subscriber from the ui layer they can see the current data (on the Main dispatcher)
I have two questions
a) If validateData fails how can I cancel the current emission and move on to the next one?
b) Is there a way for the dataFlow subscriber working on the viewModel to generate new threads so the delay parts can run in parallel?
the timeline right now looks like the first part, but I want it to run like the second one
Is there a way to do this?
I've tried using buffer() which as the documentation states "Buffers flow emissions via channel of a specified capacity and runs collector in a separate coroutine." but when I set it to BufferOverflow.SUSPEND I get the behaviour of the first part, and when I set it to BufferOverflow.DROP_OLDEST or BufferOverflow.DORP_LATEST I loose emissions
I have also tried using .conflate() like so:
repo.dataFlow
.conflate()
.map { ....
and even though the emissions start one after the other, the part with the delay still waits for the previous one to finish before starting the next one
when I use .flowOn(Dispatchers.Default) for that part , I loose emissions, and when I use .flowOn(Dispatchers.IO) or something like Executors.newFixedThreadPool(4).asCoroutineDispatcher() they always wait for the previous one to finish before starting a new one
Edit 2:
After about 3 hours of experiments this seems to work
viewModelScope.launch(Dispatchers.Default) {
repo.dataFlow
.map {
validateData(it)
}
.flowOn(Dispatchers.Default)
.map {
async {
delay(1000)
it
}
}
.flowOn(Dispatchers.IO) // NOTE (A)
.map {
val result = it.await()
transformData(result)
}
.flowOn(Dispatchers.Default)
.collect {
_readings.postValue(it)
}
}
however I still haven't figured out how to cancel the emission if validatedata fails
and for some reason it only works if I use Dispatchers.IO , Executors.newFixedThreadPool(20).asCoroutineDispatcher() and Dispatchers.Unconfined where I put note (A), Dispatchers.Main does not seem to work (which I expected) but Dispatchers.Default also does not seem to work and I don't know why

First question: Well you cannot recover from an exception in a sense of continuing
the collection of the flow, as per docs "Flow collection can complete with an exception when an emitter or code inside the operators throw an exception." therefore once an exception has been thrown the collection is completed (exceptionally) you can however handle the exception by either wrapping your collection inside try/catch block or using the catch() operator.
Second question: You cannot, while the producer (emitting side) can be made concurrent
by using the buffer() operator, collection is always sequential.
As per your diagram, you need fan out (one producer, many consumers), you cannot
achieve that with flows. Flows are cold, each time you collect from them, they start
emitting from the beginning.
Fan out can be achieved using channels, where you can have one coroutine producing
values and many coroutines that consume those values.
Edit: Oh you meant the validation failed not the function itself, in that case you can use the filter() operator.
The BroadcastChannel and ConflatedBroadcastChannel are getting deprecated. SharedFlow cannot help you in your use case, as they emit values in a broadcast fashion, meaning producer waits until all consumers consume each value before producing the next one. That is still sequential, you need parallelism. You can achieve it using the produce() channel builder.
A simple example:
val scope = CoroutineScope(Job() + Dispatchers.IO)
val producer: ReceiveChannel<Int> = scope.produce {
var counter = 0
val startTime = System.currentTimeMillis()
while (isActive) {
counter++
send(counter)
println("producer produced $counter at ${System.currentTimeMillis() - startTime} ms from the beginning")
delay(250)
}
}
val consumerOne = scope.launch {
val startTime = System.currentTimeMillis()
for (x in producer) {
println("consumerOne consumd $x at ${System.currentTimeMillis() - startTime}ms from the beginning.")
delay(1000)
}
}
val consumerTwo = scope.launch {
val startTime = System.currentTimeMillis()
for (x in producer) {
println("consumerTwo consumd $x at ${System.currentTimeMillis() - startTime}ms from the beginning.")
delay(1000)
}
}
val consumerThree = scope.launch {
val startTime = System.currentTimeMillis()
for (x in producer) {
println("consumerThree consumd $x at ${System.currentTimeMillis() - startTime}ms from the beginning.")
delay(1000)
}
}
Observe production and consumption times.

How to make request calls to remote service one by one usnig rx and kotlin?

I have remote service to which the app have to send data:
Definition in retrofit2:
interface FooRemoteService {
#POST("/foos")
fun postFoos(#Body foos: List<FooPojo>): Observable<Response<List<String>>
}
but the call has a limits no more than X Foos at once.
Each call can returns 206 code "partially successful" with list of unsuccessful uploaded foos. Also 413 "Request Entity Too Large". And of course 400 and 500 as well.
And the app needs to send unknown count of foo items (defined by user in runtime).
To avoid DDoS of service app is required to send this calls one by one.
So I made such implementation in my FooRepositoryImpl:
This is an idea. I'm not happy with below solution and I'm sure that it can be done much better but I'm run out of ideas. So any proposes?
override fun postFoos(foos: List<Foo>) Completable {
val fooChunks = divideListInToChuncksUnderRequestLimit(foos)
val unuploadedFoos = mutableListOf<UnuploadedFoo>()
fooChunks.fold(unuploadedFoos)
{ accu: MutableList<UnuploadedFoo>, chunk ->
fooRemoteService
.postFoos(chunk)
.subscribeOn(Schedulers.io())
.flatMapCompletable {
if (it.isSuccessful) {
Completable.complete()
} else {
Timber.e("$it")
accu.add(it.body())
}
}.blockingAwait()
responses
}
return Completable.complete()
}
At the end the app should display list of all unsuccessful foos or if any available. So I need pass from that fuction list of unuploaded Foos.

If you are OK with modifying the return type of postFoos a bit, something like this could work:
override fun postFoos(foos: List<Foo>): Observable<List<UnuploadedFoo>> {
val chunks = foos.chunked(CHUNK_SIZE)
val posters = chunks.map { chunk ->
fooRemoteService.postFoos(chunk)
.map { response ->
response.unUploaded.takeIf { !response.isSuccessful } ?: emptyList()
}
.filter { it.isNotEmpty() }
.toObservable()
}
return Observable.concatDelayError(posters)
}
I'm imagining your service to have something like:
data class Response(val isSuccessful: Boolean, val unUploaded: List<UnoploadedFoo>)
fun postFoos(foos: List<Foo>): Single<Response>
The trick here is that Concat:
(...) waits to subscribe to each additional Observable that you pass to it until the previous Observable completes.