Develop Reference

How to interpect when a coroutine in an observable has finished?

I have an observable in my foreground service which fetch data from a paging API and save it to the database, the foreground service shows a notification with a progress bar with number of saved items vs the total amount.
Observable which fetch all the data looks like this:
private fun getAllProducts(): Observable<Response<List<ProdottoBarcode>>> {
val lastId = intArrayOf(0)
return Observable.range(1, Integer.MAX_VALUE - 1)
.concatMap { currentPage -> getProducts(currentPage, lastId[0]) }
.takeUntil { response -> response.body()?.isEmpty() == true }
.doOnNext { response ->
lastId[0] = response.headers().get("lastId")?.toInt()!!
}
}
Then the subscription is done in onCreate() like this:
override fun onCreate() {
super.onCreate()
...
getAllProducts().subscribeWith(object: DisposableObserver<Response<List<ProdottoBarcode>>>() {
override fun onNext(response: Response<List<ProdottoBarcode>>) {
if (response.isSuccessful) {
val products = response.body()
val totalItems = response.headers().get("items")?.toInt()
insertProducts(totalItems, products)
}
}
override fun onError(e: Throwable) {
stopService()
}
override fun onComplete() {
}
})
}
And the method which saves all the data to the database looks like this:
private fun insertProducts(totalItems: Int?, products: List<ProdottoBarcode>?) {
if (products != null) {
CoroutineScope(Dispatchers.IO).launch {
for (product in products) {
repository.insert(product)
savedItems += 1
val notification =
totalItems?.let { items ->
NotificationCompat.Builder(baseContext, "progress_channel")
.setSmallIcon(R.drawable.ic_box)
.setContentTitle("Sincronizzati: $savedItems prodotti su $totalItems")
.setProgress(items, savedItems, false)
.setOngoing(true)
.build()
}
notificationManager.notify(notificationId, notification)
}
// TODO: stop the service and dismiss the notification when all items has been saved
if (savedItems == totalItems) {
stopService()
}
}
}
}
The stopService() in insertProducts not always works, while if I try to put stopService in onComplete() it will be executed once all subscriptions are done and NOT when all the items has been saved.
So my question is:
How can I stop my services by using the Coroutine inside the Observable? I need to know when all items from all observables are insert in database and only then to dismiss the service.

Side note: you don't need to do Int wrapping like this in Kotlin like you would in Java. Kotlin has implicit variable wrapping, so you can simply use a var local variable and it will be captured by whatever function you use it in.
val lastId = intArrayOf(0) // can just be var lastId = 0
Starting with getProducts() for fetching a page. I think the code you linked is OK provided your Retrofit service's getProducts function is marked suspend, so it's not blocking. No changes here.
private suspend fun getProducts(
page: Int,
lastId: Int,
itemsPerPage: Int = 50
): Response<List<ProdottoBarcode>> {
val prefs = PreferenceManager.getDefaultSharedPreferences(this)
val urlServer = prefs.getString("server", "http://127.0.0.1/")!!
return RetrofitClient.getInstance().getService()
.getProducts(urlServer, "A", page, lastId, itemsPerPage)
}
Your getAllProducts in your linked code doesn't need backing StateFlows that are never collected--you're using them simply as mutable Int wrappers, which are unnecessary in Kotlin as mentioned way above. I'm not exactly sure how you're consuming these pages, since I'm not very familiar with Rx, but I take the use of concatMap to mean that the Observable is queuing up pages as fast as it can into a buffer, and you are reading out these pages to some local property that the UI uses. I think a buffer should be added so we can be inserting in the database in parallel with fetching the next page. Default buffer arguments are probably appropriate.
private val allProducts: Flow<Response<List<ProdottoBarcode>>> = flow {
var lastId = 0
for (currentPage in 1 until Int.MAX_VALUE) {
emit(getProducts(currentPage, lastId))
lastId = response.headers().get("lastId")!!.toInt()
}
}
.takeWhile { response -> !response.body().isNullOrEmpty() }
.buffer()
Usually, when you collect your flow, you should use an appropriate coroutine scope provided by the Android framework, so it will automatically cancel collection once it goes out of scope. If you inherit your service from LifecycleService, you can use the existing lifecycleScope. This is maybe not so critical in a service in this case, since I think you are only calling stopService() when your flow is complete, but it would make it a little more robust against potential mistakes, I think.
.launchIn is a shortcut that is like wrapping everything above it in launch and calling collect() on it. I prefer the syntax because it has less nesting of code.
override fun onCreate() {
super.onCreate()
// ...
allProducts.onEach { response ->
if (response.isSuccessful) {
val products = response.body()
val totalItems = response.headers().get("items")?.toInt()
insertProducts(totalItems, products)
}
}
.catch { Log.e(TAG, "Failed collecting page.", it) }
.onCompletion { stopService() }
.flowOn(Dispatchers.Default) // don't use main thread since this is a service
.launchIn(lifecycleScope)
}
Since we're using buffer() in the fetching flow, we don't need to launch other coroutines when inserting in the database to achieve parallelism. We can simplify this into a suspend function. We are handling stopping the service in the flow collector, so we don't need to do that here either. I'm assuming repository.insert is a suspend function, not blocking.
private suspend fun insertProducts(totalItems: Int?, products: List<ProdottoBarcode>?) {
if (totalItems == null) {
Log.e(TAG, "Tried to insert items without any item count. Skipping.")
return
}
if (products == null) {
Log.e(TAG, "Tried to insert null products list. Skipping.")
return
}
for (product in products) {
repository.insert(product)
savedItems += 1
val notification = NotificationCompat.Builder(baseContext, "progress_channel")
.setSmallIcon(R.drawable.ic_box)
.setContentTitle("Sincronizzati: $savedItems prodotti su $totalItems")
.setProgress(items, savedItems, false)
.setOngoing(true)
.build()
}
notificationManager.notify(notificationId, notification)
}
}

Kotlin Coroutine Flow: Limit the number of collector

Is there a way to limit the number of collector in a function that returns a Flow using flow builder?
I have this public method in a ViewModel
fun fetchAssets(limit: String) {
viewModelScope.launch {
withContext(Dispatchers.IO){
getAssetsUseCase(AppConfigs.ASSET_PARAMS, limit).onEach {
when (it) {
is RequestStatus.Loading -> {
_assetState.tryEmit(AssetState.FetchLoading)
}
is RequestStatus.Success -> {
_assetState.tryEmit(AssetState.FetchSuccess(it.data.assetDataDomain))
}
is RequestStatus.Failed -> {
_assetState.tryEmit(AssetState.FetchFailed(it.message))
}
}
}.collect()
}
}
}
This method is called on ViewModel's init block, but can also be called manually on UI.
This flow emits value every 10 seconds.
Repository
override fun fetchAssets(
query: String,
limit: String
) = flow {
while (true) {
try {
interceptor.baseUrl = AppConfigs.ASSET_BASE_URL
emit(RequestStatus.Loading())
val domainModel = mapper.mapToDomainModel(service.getAssetItems(query, limit))
emit(RequestStatus.Success(domainModel))
} catch (e: HttpException) {
emit(RequestStatus.Failed(e))
} catch (e: IOException) {
emit(RequestStatus.Failed(e))
}
delay(10_000)
}
}
Unfortunately every time fetch() was invoke from UI, I noticed that it creates another collectors thus can ended up having tons of collector which is really bad and incorrect.
The idea is having a flow that emits value every 10 seconds but can also be invoke manually via UI for immediate data update without having multiple collectors.

You seem to misunderstand what does it mean to collect the flow or you misuse the collect operation. By collecting the flow we mean we observe it for changes. But you try to use collect() to introduce changes to the flow, which can't really work. It just starts another flow in the background.
You should collect the flow only once, so keep it inside init or wherever it is appropriate for your case. Then you need to update the logic of the flow to make it possible to trigger reloading on demand. There are many ways to do it and the solution will differ depending whether you need to reset the timer on manual update or not. For example, we can use the channel to notify the flow about the need to reload:
val reloadChannel = Channel<Unit>(Channel.CONFLATED)
fun fetchAssets(
query: String,
limit: String
) = flow {
while (true) {
try {
...
}
withTimeoutOrNull(10.seconds) { reloadChannel.receive() } // replace `delay()` with this
}
}
fun reload() {
reloadChannel.trySend(Unit)
}
Whenever you need to trigger the manual reload, do not start another flow or invoke another collect() operation, but instead just invoke reload(). Then the flow that is already being collected, will start reloading and will emit state changes.
This solution resets the timer on manual reload, which I believe is better for the user experience.

I ended up moving the timer on ViewModel as I can request on demand fetch while also not having multiple collectors that runs at the same time.
private var job: Job? = null
private val _assetState = defaultMutableSharedFlow<AssetState>()
fun getAssetState() = _assetState.asSharedFlow()
init {
job = viewModelScope.launch {
while(true) {
if (lifecycleState == LifeCycleState.ON_START || lifecycleState == LifeCycleState.ON_RESUME)
fetchAssets()
delay(10_000)
}
}
}
fun fetchAssets() {
viewModelScope.launch {
withContext(Dispatchers.IO) {
getAssetsUseCase(
AppConfigs.ASSET_BASE_URL,
AppConfigs.ASSET_PARAMS,
AppConfigs.ASSET_SIZES[AppConfigs.ASSET_LIMIT_INDEX]
).onEach {
when(it){
is RequestStatus.Loading -> {
_assetState.tryEmit(AssetState.FetchLoading)
}
is RequestStatus.Success -> {
_assetState.tryEmit(AssetState.FetchSuccess(it.data.assetDataDomain))
}
is RequestStatus.Failed -> {
_assetState.tryEmit(AssetState.FetchFailed(it.message))
}
}
}.collect()
}
}
}
override fun onCleared() {
job?.cancel()
super.onCleared()
}
Please correct me if this one is a code smell.

Kotlin Coroutines - Suspend function returning a Flow runs forever

I am making a network repository that supports multiple data retrieval configs, therefore I want to separate those configs' logic into functions.
However, I have a config that fetches the data continuously at specified intervals. Everything is fine when I emit those values to the original Flow. But when I take the logic into another function and return another Flow through it, it stops caring about its coroutine scope. Even after the scope's cancelation, it keeps on fetching the data.
TLDR: Suspend function returning a flow runs forever when currentCoroutineContext is used to control its loop's termination.
What am I doing wrong here?
Here's the simplified version of my code:
Fragment calling the viewmodels function that basically calls the getData()
lifecycleScope.launch {
viewModel.getLatestDataList()
}
Repository
suspend fun getData(config: MyConfig): Flow<List<Data>>
{
return flow {
when (config)
{
CONTINUOUS ->
{
//It worked fine when fetchContinuously was ingrained to here and emitted directly to the current flow
//And now it keeps on running eternally
fetchContinuously().collect { updatedList ->
emit(updatedList)
}
}
}
}
}
//Note logic of this function is greatly reduced to keep the focus on the problem
private suspend fun fetchContinuously(): Flow<List<Data>>
{
return flow {
while (currentCoroutineContext().isActive)
{
val updatedList = fetchDataListOverNetwork().await()
if (updatedList != null)
{
emit(updatedList)
}
delay(refreshIntervalInMs)
}
Timber.i("Context is no longer active - terminating the continuous-fetch coroutine")
}
}
private suspend fun fetchDataListOverNetwork(): Deferred<List<Data>?> =
withContext(Dispatchers.IO) {
return#withContext async {
var list: List<Data>? = null
try
{
val response = apiService.getDataList().execute()
if (response.isSuccessful && response.body() != null)
{
list = response.body()!!.list
}
else
{
Timber.w("Failed to fetch data from the network database. Error body: ${response.errorBody()}, Response body: ${response.body()}")
}
}
catch (e: Exception)
{
Timber.w("Exception while trying to fetch data from the network database. Stacktrace: ${e.printStackTrace()}")
}
finally
{
return#async list
}
list //IDE is not smart enough to realize we are already returning no matter what inside of the finally block; therefore, this needs to stay here
}
}

I am not sure whether this is a solution to your problem, but you do not need to have a suspending function that returns a Flow. The lambda you are passing is a suspending function itself:
fun <T> flow(block: suspend FlowCollector<T>.() -> Unit): Flow<T> (source)
Here is an example of a flow that repeats a (GraphQl) query (simplified - without type parameters) I am using:
override fun query(query: Query,
updateIntervalMillis: Long): Flow<Result<T>> {
return flow {
// this ensures at least one query
val result: Result<T> = execute(query)
emit(result)
while (coroutineContext[Job]?.isActive == true && updateIntervalMillis > 0) {
delay(updateIntervalMillis)
val otherResult: Result<T> = execute(query)
emit(otherResult)
}
}
}

I'm not that good at Flow but I think the problem is that you are delaying only the getData() flow instead of delaying both of them.
Try adding this:
suspend fun getData(config: MyConfig): Flow<List<Data>>
{
return flow {
when (config)
{
CONTINUOUS ->
{
fetchContinuously().collect { updatedList ->
emit(updatedList)
delay(refreshIntervalInMs)
}
}
}
}
}
Take note of the delay(refreshIntervalInMs).

Android WorkManager observe progress

I'm using WorkManager for deferred work in my app.
The total work is divided into a number of chained workers, and I'm having trouble showing the workers' progress to the user (using progress bar).
I tried creating one tag and add it to the different workers, and inside the workers update the progress by that tag, but when I debug I always get progress is '0'.
Another thing I noticed is that the workManager's list of work infos is getting bigger each time I start the work (even if the workers finished their work).
Here is my code:
//inside view model
private val workManager = WorkManager.getInstance(appContext)
internal val progressWorkInfoItems: LiveData<List<WorkInfo>>
init
{
progressWorkInfoItems = workManager.getWorkInfosByTagLiveData(TAG_SAVING_PROGRESS)
}
companion object
{
const val TAG_SAVING_PROGRESS = "saving_progress_tag"
}
//inside a method
var workContinuation = workManager.beginWith(OneTimeWorkRequest.from(firstWorker::class.java))
val secondWorkRequest = OneTimeWorkRequestBuilder<SecondWorker>()
secondWorkRequest.addTag(TAG_SAVING_PROGRESS)
secondWorkRequest.setInputData(createData())
workContinuation = workContinuation.then(secondWorkRequest.build())
val thirdWorkRequest = OneTimeWorkRequestBuilder<ThirdWorker>()
thirdWorkRequest.addTag(TAG_SAVING_PROGRESS)
thirdWorkRequest.setInputData(createData())
workContinuation = workContinuation.then(thirdWorkRequest.build())
workContinuation.enqueue()
//inside the Activity
viewModel.progressWorkInfoItems.observe(this, observeProgress())
private fun observeProgress(): Observer<List<WorkInfo>>
{
return Observer { listOfWorkInfo ->
if (listOfWorkInfo.isNullOrEmpty()) { return#Observer }
listOfWorkInfo.forEach { workInfo ->
if (WorkInfo.State.RUNNING == workInfo.state)
{
val progress = workInfo.progress.getFloat(TAG_SAVING_PROGRESS, 0f)
progress_bar?.progress = progress
}
}
}
}
//inside the worker
override suspend fun doWork(): Result = withContext(Dispatchers.IO)
{
setProgress(workDataOf(TAG_SAVING_PROGRESS to 10f))
...
...
Result.success()
}

The setProgress method is to observe intermediate progress in a single Worker (as explained in the guide):
Progress information can only be observed and updated while the ListenableWorker is running.
For this reason, the progress information is available only till a Worker is active (e.g. it is not in a terminal state like SUCCEEDED, FAILED and CANCELLED). This WorkManager guide covers Worker's states.
My suggestion is to use the Worker's unique ID to identify which worker in your chain is not yet in a terminal state. You can use WorkRequest's getId method to retrieve its unique ID.

According to my analysis I have found that there might be two reasons why you always get 0
setProgress is set just before the Result.success() in the doWork() of the worker then it's lost and you never get that value in your listener. This could be because the state of the worker is now SUCCEEDED
the worker is completing its work in fraction of seconds
Lets take a look at the following code
class Worker1(context: Context, workerParameters: WorkerParameters) : Worker(context,workerParameters) {
override fun doWork(): Result {
setProgressAsync(Data.Builder().putInt("progress",10).build())
for (i in 1..5) {
SystemClock.sleep(1000)
}
setProgressAsync(Data.Builder().putInt("progress",50).build())
SystemClock.sleep(1000)
return Result.success()
}
}
In the above code
if you remove only the first sleep method then the listener only get the progres50
if you remove only the second sleep method then the listener only get the progress 10
If you remove both then the you get the default value 0
This analysis is based on the WorkManager version 2.4.0
Hence I found that the following way is better and always reliable to show the progress of various workers of your chain work.
I have two workers that needs to be run one after the other. If the first work is completed then 50% of the work is done and 100% would be done when the second work is completed.
Two workers
class Worker1(context: Context, workerParameters: WorkerParameters) : Worker(context,workerParameters) {
override fun doWork(): Result {
for (i in 1..5) {
Log.e("worker", "worker1----$i")
}
return Result.success(Data.Builder().putInt("progress",50).build())
}
}
class Worker2(context: Context, workerParameters: WorkerParameters) : Worker(context,workerParameters) {
override fun doWork(): Result {
for (i in 5..10) {
Log.e("worker", "worker1----$i")
}
return Result.success(Data.Builder().putInt("progress",100).build())
}
}
Inside the activity
workManager = WorkManager.getInstance(this)
workRequest1 = OneTimeWorkRequest.Builder(Worker1::class.java)
.addTag(TAG_SAVING_PROGRESS)
.build()
workRequest2 = OneTimeWorkRequest.Builder(Worker2::class.java)
.addTag(TAG_SAVING_PROGRESS)
.build()
findViewById<Button>(R.id.btn).setOnClickListener(View.OnClickListener { view ->
workManager?.
beginUniqueWork(TAG_SAVING_PROGRESS,ExistingWorkPolicy.REPLACE,workRequest1)
?.then(workRequest2)
?.enqueue()
})
progressBar = findViewById(R.id.progressBar)
workManager?.getWorkInfoByIdLiveData(workRequest1.id)
?.observe(this, Observer { workInfo: WorkInfo? ->
if (workInfo != null && workInfo.state == WorkInfo.State.SUCCEEDED) {
val progress = workInfo.outputData
val value = progress.getInt("progress", 0)
progressBar?.progress = value
}
})
workManager?.getWorkInfoByIdLiveData(workRequest2.id)
?.observe(this, Observer { workInfo: WorkInfo? ->
if (workInfo != null && workInfo.state == WorkInfo.State.SUCCEEDED) {
val progress = workInfo.outputData
val value = progress.getInt("progress", 0)
progressBar?.progress = value
}
})
The reason workManager's list of work infos is getting bigger each time the work is started even if the workers finished their work is because of
workManager.beginWith(OneTimeWorkRequest.from(firstWorker::class.java))
instead one need to use
workManager?.beginUniqueWork(TAG_SAVING_PROGRESS, ExistingWorkPolicy.REPLACE,OneTimeWorkRequest.from(firstWorker::class.java))
You can read more about it here

Cancelling a callback inside a suspendCancellableCoroutine

I have wrapped a callback in suspendCancellableCoroutine to convert it to a suspend function:
suspend fun TextToSpeech.speakAndWait(text: String) : Boolean {
val uniqueUtteranceId = getUniqueUtteranceId(text)
speak(text, TextToSpeech.QUEUE_FLUSH, null, uniqueUtteranceId)
return suspendCancellableCoroutine { continuation ->
this.setOnUtteranceProgressListener(object : JeLisUtteranceProgressListener() {
override fun onDone(utteranceId: String?) {
if(utteranceId == uniqueUtteranceId) {
Timber.d("word is read, resuming with the next word")
continuation.resume(true)
}
}
})
}
}
I'm calling this function with the lifecycleScope coroutine scope of the fragment and I was assuming that it was cancelled when fragment is destroyed. However, LeakCanary reported that my fragment was leaking because of this listener and I verified with logs that the callback was called even after the coroutine is cancelled.
So it seems that wrapping with suspendCancellableCoroutine instead of suspendCoroutine does not suffice to cancel the callback. I guess I should actively check whether the job is active, but how? I tried coroutineContext.ensureActive() and checking coroutineContext.isActive inside the callback but IDE gives an error saying that "suspension functions can be called only within coroutine body" What else can I do to ensure that it doesn't resume if the job is cancelled?

LeakCanary reported that my fragment was leaking because of this listener and I verified with logs that the callback was called even after the coroutine is cancelled.
Yes, the underlying async API is unaware of Kotlin coroutines and you have to work with it to explicitly propagate cancellation. Kotlin provides the invokeOnCancellation callback specifically for this purpose:
return suspendCancellableCoroutine { continuation ->
this.setOnUtteranceProgressListener(object : JeLisUtteranceProgressListener() {
/* continuation.resume() */
})
continuation.invokeOnCancellation {
this.setOnUtteranceProgressListener(null)
}
}

If you want to remove your JeLisUtteranceProgressListener regardless of result (success, cancellation or other errors) you can instead use a classic try/finally block:
suspend fun TextToSpeech.speakAndWait(text: String) : Boolean {
val uniqueUtteranceId = getUniqueUtteranceId(text)
speak(text, TextToSpeech.QUEUE_FLUSH, null, uniqueUtteranceId)
return try {
suspendCancellableCoroutine { continuation ->
this.setOnUtteranceProgressListener(object : JeLisUtteranceProgressListener() {
override fun onDone(utteranceId: String?) {
if(utteranceId == uniqueUtteranceId) {
Timber.d("word is read, resuming with the next word")
continuation.resume(true)
}
}
})
} finally {
this.setOnUtteranceProgressListener(null)
}
}

In addition to the accepted answer, I recognized that continuation object has an isActive property as well. So alternatively we can check whether coroutine is still active inside the callback before resuming:
return suspendCancellableCoroutine { continuation ->
this.setOnUtteranceProgressListener(object : JeLisUtteranceProgressListener()
{
override fun onDone(utteranceId: String?) {
if(utteranceId == uniqueUtteranceId) {
if (continuation.isActive) {
continuation.resume(true)
}
}
}
})
continuation.invokeOnCancellation {
this.setOnUtteranceProgressListener(null)
}
}