Develop Reference

Coroutine Thread Safety with Retrofit

I have still a little bit of trouble putting all information together about the thread-safety of using coroutines to launch network requests.
Let's say we have following use-case, there is a list of users we get and for each of those users, I will do some specific check which has to run over a network request to the API, giving me some information back about this user.
The userCheck happens inside a library, which doesn't expose suspend functions but rather still uses a callback.
Inside of this library, I have seen code like this to launch each of the network requests:
internal suspend fun <T> doNetworkRequest(request: suspend () -> Response<T>): NetworkResult<T> {
return withContext(Dispatchers.IO) {
try {
val response = request.invoke()
...
According to the documentation, Dispatchers.IO can use multiple threads for the execution of the code, also the request function is simply a function from a Retrofit API.
So what I did is to launch the request for each user, and use a single resultHandler object, which will add the results to a list and check if the length of the result list equals the length of the user list, if so, then all userChecks are done and I know that I can do something with the results, which need to be returned all together.
val userList: List<String>? = getUsers()
val userCheckResultList = mutableListOf<UserCheckResult>()
val handler = object : UserCheckResultHandler {
override fun onResult(
userCheckResult: UserCheckResult?
) {
userCheckResult?.let {
userCheckResultList.add(
it
)
}
if (userCheckResultList.size == userList?.size) {
doSomethingWithResultList()
print("SUCCESS")
}
}
}
userList?.forEach {
checkUser(it, handler)
}
My question is: Is this implementation thread-safe? As far as I know, Kotlin objects should be thread safe, but I have gotten feedback that this is possibly not the best implementation :D
But in theory, even if the requests get launched asynchronous and multiple at the same time, only one at a time can access the lock of the thread the result handler is running on and there will be no race condition or problems with adding items to the list and comparing the sizes.
Am I wrong about this?
Is there any way to handle this scenario in a better way?

If you are executing multiple request in parallel - it's not. List is not thread safe. But it's simple fix for that. Create a Mutex object and then just wrap your operation on list in lock, like that:
val lock = Mutex()
val userList: List<String>? = getUsers()
val userCheckResultList = mutableListOf<UserCheckResult>()
val handler = object : UserCheckResultHandler {
override fun onResult(
userCheckResult: UserCheckResult?
) {
lock.withLock {
userCheckResult?.let {
userCheckResultList.add(
it
)
}
if (userCheckResultList.size == userList?.size) {
doSomethingWithResultList()
print("SUCCESS")
}
}
}
}
userList?.forEach {
checkUser(it, handler)
}
I have to add that this whole solution seems very hacky. I would go completely other route. Run all of your requests wrapping those in async { // network request } which will return Deferred object. Add this object to some list. After that wait for all of those deferred objects using awaitAll(). Like that:
val jobs = mutableListOf<Job>()
userList?.forEach {
// i assume checkUser is suspendable here
jobs += async { checkUser(it, handler) }
}
// wait for all requests
jobs.awaitAll()
// After that you can access all results like this:
val resultOfJob0 = jobs[0].getCompleted()

Android WorkManager observe progress

I'm using WorkManager for deferred work in my app.
The total work is divided into a number of chained workers, and I'm having trouble showing the workers' progress to the user (using progress bar).
I tried creating one tag and add it to the different workers, and inside the workers update the progress by that tag, but when I debug I always get progress is '0'.
Another thing I noticed is that the workManager's list of work infos is getting bigger each time I start the work (even if the workers finished their work).
Here is my code:
//inside view model
private val workManager = WorkManager.getInstance(appContext)
internal val progressWorkInfoItems: LiveData<List<WorkInfo>>
init
{
progressWorkInfoItems = workManager.getWorkInfosByTagLiveData(TAG_SAVING_PROGRESS)
}
companion object
{
const val TAG_SAVING_PROGRESS = "saving_progress_tag"
}
//inside a method
var workContinuation = workManager.beginWith(OneTimeWorkRequest.from(firstWorker::class.java))
val secondWorkRequest = OneTimeWorkRequestBuilder<SecondWorker>()
secondWorkRequest.addTag(TAG_SAVING_PROGRESS)
secondWorkRequest.setInputData(createData())
workContinuation = workContinuation.then(secondWorkRequest.build())
val thirdWorkRequest = OneTimeWorkRequestBuilder<ThirdWorker>()
thirdWorkRequest.addTag(TAG_SAVING_PROGRESS)
thirdWorkRequest.setInputData(createData())
workContinuation = workContinuation.then(thirdWorkRequest.build())
workContinuation.enqueue()
//inside the Activity
viewModel.progressWorkInfoItems.observe(this, observeProgress())
private fun observeProgress(): Observer<List<WorkInfo>>
{
return Observer { listOfWorkInfo ->
if (listOfWorkInfo.isNullOrEmpty()) { return#Observer }
listOfWorkInfo.forEach { workInfo ->
if (WorkInfo.State.RUNNING == workInfo.state)
{
val progress = workInfo.progress.getFloat(TAG_SAVING_PROGRESS, 0f)
progress_bar?.progress = progress
}
}
}
}
//inside the worker
override suspend fun doWork(): Result = withContext(Dispatchers.IO)
{
setProgress(workDataOf(TAG_SAVING_PROGRESS to 10f))
...
...
Result.success()
}

The setProgress method is to observe intermediate progress in a single Worker (as explained in the guide):
Progress information can only be observed and updated while the ListenableWorker is running.
For this reason, the progress information is available only till a Worker is active (e.g. it is not in a terminal state like SUCCEEDED, FAILED and CANCELLED). This WorkManager guide covers Worker's states.
My suggestion is to use the Worker's unique ID to identify which worker in your chain is not yet in a terminal state. You can use WorkRequest's getId method to retrieve its unique ID.

According to my analysis I have found that there might be two reasons why you always get 0
setProgress is set just before the Result.success() in the doWork() of the worker then it's lost and you never get that value in your listener. This could be because the state of the worker is now SUCCEEDED
the worker is completing its work in fraction of seconds
Lets take a look at the following code
class Worker1(context: Context, workerParameters: WorkerParameters) : Worker(context,workerParameters) {
override fun doWork(): Result {
setProgressAsync(Data.Builder().putInt("progress",10).build())
for (i in 1..5) {
SystemClock.sleep(1000)
}
setProgressAsync(Data.Builder().putInt("progress",50).build())
SystemClock.sleep(1000)
return Result.success()
}
}
In the above code
if you remove only the first sleep method then the listener only get the progres50
if you remove only the second sleep method then the listener only get the progress 10
If you remove both then the you get the default value 0
This analysis is based on the WorkManager version 2.4.0
Hence I found that the following way is better and always reliable to show the progress of various workers of your chain work.
I have two workers that needs to be run one after the other. If the first work is completed then 50% of the work is done and 100% would be done when the second work is completed.
Two workers
class Worker1(context: Context, workerParameters: WorkerParameters) : Worker(context,workerParameters) {
override fun doWork(): Result {
for (i in 1..5) {
Log.e("worker", "worker1----$i")
}
return Result.success(Data.Builder().putInt("progress",50).build())
}
}
class Worker2(context: Context, workerParameters: WorkerParameters) : Worker(context,workerParameters) {
override fun doWork(): Result {
for (i in 5..10) {
Log.e("worker", "worker1----$i")
}
return Result.success(Data.Builder().putInt("progress",100).build())
}
}
Inside the activity
workManager = WorkManager.getInstance(this)
workRequest1 = OneTimeWorkRequest.Builder(Worker1::class.java)
.addTag(TAG_SAVING_PROGRESS)
.build()
workRequest2 = OneTimeWorkRequest.Builder(Worker2::class.java)
.addTag(TAG_SAVING_PROGRESS)
.build()
findViewById<Button>(R.id.btn).setOnClickListener(View.OnClickListener { view ->
workManager?.
beginUniqueWork(TAG_SAVING_PROGRESS,ExistingWorkPolicy.REPLACE,workRequest1)
?.then(workRequest2)
?.enqueue()
})
progressBar = findViewById(R.id.progressBar)
workManager?.getWorkInfoByIdLiveData(workRequest1.id)
?.observe(this, Observer { workInfo: WorkInfo? ->
if (workInfo != null && workInfo.state == WorkInfo.State.SUCCEEDED) {
val progress = workInfo.outputData
val value = progress.getInt("progress", 0)
progressBar?.progress = value
}
})
workManager?.getWorkInfoByIdLiveData(workRequest2.id)
?.observe(this, Observer { workInfo: WorkInfo? ->
if (workInfo != null && workInfo.state == WorkInfo.State.SUCCEEDED) {
val progress = workInfo.outputData
val value = progress.getInt("progress", 0)
progressBar?.progress = value
}
})
The reason workManager's list of work infos is getting bigger each time the work is started even if the workers finished their work is because of
workManager.beginWith(OneTimeWorkRequest.from(firstWorker::class.java))
instead one need to use
workManager?.beginUniqueWork(TAG_SAVING_PROGRESS, ExistingWorkPolicy.REPLACE,OneTimeWorkRequest.from(firstWorker::class.java))
You can read more about it here

Kotlin coroutine in lifecycleScope doesn't block main thread

I am confused about coroutines in ViewModels.
My question is pretty simple: why does it seem like the following coroutine doesn't block UIThread? (UI is still smooth while coroutine is running)
My fragment right here:
class FragmentSeePaths : Fragment(R.layout.fragment_see_paths),
PathRecyclerAdapter.OnSetPathForWidgetListener {
private val pathViewModel: PathViewModel by activityViewModels()
private lateinit var binding: FragmentSeePathsBinding
private lateinit var listener: OnAddLineRequestListener
override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
...
}
private fun observeWidgetPath() {
pathViewModel.getUserWidgetPath().observe(viewLifecycleOwner, Observer {
if (it != null) {
lifecycleScope.launch {
val times = pathViewModel.fetchBusTime(it)
updateUI(it, times)
}
}
})
}
And here are the shotened ViewModel with the fetchBusTime method:
suspend fun fetchBusTime(path: Path): Pair<List<Time>?, List<Time>?> {
Log.v("fetchBusTimeUI", Thread.currentThread().name) // Main
// Some network requests made with Retrofit
val timesResponseStartPoint: GinkoTimesResponse? = repository.getTimes(
path.startingPoint.startName,
path.line.lineId,
path.isStartPointNaturalWay
)
val timesResponseEndPoint: GinkoTimesResponse? = repository.getTimes(
path.endingPoint.endName,
path.line.lineId,
path.isStartPointNaturalWay
)
return timesResponseStartPoint to timesResponseEndPoint
}

launch allows us to start a coroutine in the background and keep working in the meantime. Suspending functions may suspend the execution of the current coroutine without blocking the current thread. We can start a coroutine under any of the below dispatchers.
dipatcher.IO -> Network operation
dispatcher.Main -> On main thread
dispatcher.Default -> for CPU intensive operation
To explain you in detail, I am taking an example from the documentation :-
fun main() {
GlobalScope.launch { // launch new coroutine in background and continue
delay(1000L)
println("World!")
}
println("Hello,") // main thread continues here immediately
runBlocking { // but this expression blocks the main thread
delay(2000L) // ... while we delay for 2 seconds to keep JVM alive
}
}
The comments should speak for themselves. This will print “Hello,” immediately, and add “World!” after a second.
This is the same thing that will work with your code, the suspend function fetchBusTime() will be executed without blocking the thread and after the operation completed inside this method, it will execute updateUI(it, times).
For more details on this please go through this article here

Make part of coroutine continue past cancellation

I have a file managing class that can save a big file. The file manager class is an application singleton, so it outlives my UI classes. My Activity/Fragment can call the save suspend function of the file manager from a coroutine and then show success or failure in the UI. For example:
//In MyActivity:
private fun saveTheFile() = lifecycleScope.launch {
try {
myFileManager.saveBigFile()
myTextView.text = "Successfully saved file"
} catch (e: IOException) {
myTextView.text = "Failed to save file"
}
}
//In MyFileManager
suspend fun saveBigFile() {
//Set up the parameters
//...
withContext(Dispatchers.IO) {
//Save the file
//...
}
}
The problem with this approach is that I don't want the save operation to be aborted if the Activity is finished. If the activity is destroyed before the withContext block gets going, or if the withContext block has any suspension points in it, then saving will not be completed because the coroutine will be canceled.
What I want to happen is that the file is always saved. If the Activity is still around, then we can show UI updates on completion.
I thought one way to do it might be to start a new coroutineScope from the suspend function like this, but this scope still seems to get cancelled when its parent job is cancelled.
suspend fun saveBigFile() = coroutineScope {
//...
}
I thought another alternative might be to make this a regular function that updates some LiveData when it's finished. The Activity could observe the live data for the result, and since LiveData automatically removes lifecycle observers when they're destroyed, the Activity is not leaked to the FileManager. I'd like to avoid this pattern if the something less convoluted like the above can be done instead.
//In MyActivity:
private fun saveTheFile() {
val result = myFileManager.saveBigFile()
result.observe(this#MyActivity) {
myTextView.text = when (it) {
true -> "Successfully saved file"
else -> "Failed to save file"
}
}
}
//In MyFileManager
fun saveBigFile(): LiveData<Boolean> {
//Set up the parameters
//...
val liveData = MutableLiveData<Boolean>()
MainScope().launch {
val success = withContext(Dispatchers.IO) {
//Save the file
//...
}
liveData.value = success
}
return liveData
}

You can wrap the bit that you don't want to be cancelled with NonCancellable.
// May cancel here.
withContext(Dispatchers.IO + NonCancellable) {
// Will complete, even if cancelled.
}
// May cancel here.

If you have code whose lifetime is scoped to the lifetime of the whole application, then this is a use case for the GlobalScope. However, just saying GlobalScope.launch is not a good strategy because you could launch several concurrent file operations that may be in conflict (this depends on your app's details). The recommended way is to use a globally-scoped actor, in the role of an executor service.
Basically, you can say
#ObsoleteCoroutinesApi
val executor = GlobalScope.actor<() -> Unit>(Dispatchers.IO) {
for (task in channel) {
task()
}
}
And use it like this:
private fun saveTheFile() = lifecycleScope.launch {
executor.send {
try {
myFileManager.saveBigFile()
withContext(Main) {
myTextView.text = "Successfully saved file"
}
} catch (e: IOException) {
withContext(Main) {
myTextView.text = "Failed to save file"
}
}
}
}
Note that this is still not a great solution, it retains myTextView beyond its lifetime. Decoupling the UI notifications from the view is another topic, though.
actor is labeled as "obsolete coroutines API", but that's just an advance notice that it will be replaced with a more powerful alternative in a future version of Kotlin. It doesn't mean it's broken or unsupported.

I tried this, and it appears to do what I described that I wanted. The FileManager class has its own scope, though I suppose it could also be GlobalScope since it's a singleton class.
We launch a new job in its own scope from the coroutine. This is done from a separate function to remove any ambiguity about the scope of the job. I use async
for this other job so I can bubble up exceptions that the UI should respond to.
Then after launch, we await the async job back in the original scope. await() suspends until the job is completed and passes along any throws (in my case I want IOExceptions to bubble up for the UI to show an error message). So if the original scope is cancelled, its coroutine never waits for the result, but the launched job keeps rolling along until it completes normally. Any exceptions that we want to ensure are always handled should be handled within the async function. Otherwise, they won't bubble up if the original job is cancelled.
//In MyActivity:
private fun saveTheFile() = lifecycleScope.launch {
try {
myFileManager.saveBigFile()
myTextView.text = "Successfully saved file"
} catch (e: IOException) {
myTextView.text = "Failed to save file"
}
}
class MyFileManager private constructor(app: Application):
CoroutineScope by MainScope() {
suspend fun saveBigFile() {
//Set up the parameters
//...
val deferred = saveBigFileAsync()
deferred.await()
}
private fun saveBigFileAsync() = async(Dispatchers.IO) {
//Save the file
//...
}
}

Unit testing coroutines on UI thread

I'm using coroutines to do an asynchronous call on pull to refresh like so:
class DataFragment : Fragment(), SwipeRefreshLayout.OnRefreshListener {
// other functions here
override fun onRefresh() {
loadDataAsync()
}
private fun loadDataAsync() = async(UI) {
swipeRefreshLayout?.isRefreshing = true
progressLayout?.showContent()
val data = async(CommonPool) {
service?.getData() // suspending function
}.await()
when {
data == null -> showError()
data.isEmpty() -> progressLayout?.showEmpty(null, parentActivity?.getString(R.string.no_data), null)
else -> {
dataAdapter?.updateData(data)
dataAdapter?.notifyDataSetChanged()
progressLayout?.showContent()
}
}
swipeRefreshLayout?.isRefreshing = false
}
}
Everything here works fine when I actually put it on a device. My error, empty, and data states are all handled well and the performance is good. However, I'm also trying to unit test it with Spek. My Spek test looks like this:
#RunWith(JUnitPlatform::class)
class DataFragmentTest : Spek({
describe("The DataFragment") {
var uut: DataFragment? = null
beforeEachTest {
uut = DataFragment()
}
// test other functions
describe("when onRefresh") {
beforeEachTest {
uut?.swipeRefreshLayout = mock()
uut?.onRefresh()
}
it("sets swipeRefreshLayout.isRefreshing to true") {
verify(uut?.swipeRefreshLayout)?.isRefreshing = true // says no interaction with mock
}
}
}
}
The test is failing because it says that there was no interaction with the uut?.swipeRefreshLayout mock. After some experimenting, it seems this is because I'm using the UI context via async(UI). If I make it just be a regular async, I can get the test to pass but then the app crashes because I'm modifying views outside of the UI thread.
Any ideas why this might be occurring? Also, if anyone has any better suggestions for doing this which will make it more testable, I'm all ears.
Thanks.
EDIT: Forgot to mention that I also tried wrapping the verify and the uut?.onRefresh() in a runBlocking, but I still had no success.

If you want to make things clean and consider using MVP architecture in the future you should understand that CourutineContext is external dependency, that should be injected via DI, or passed to your presenter. More details on topic.
The answer for your question is simple, you should use only Unconfined CourutineContext for your tests. (more)
To make things simple create an object e.g. Injection with:
package com.example
object Injection {
val uiContext : CourutineContext = UI
val bgContext : CourutineContext = CommonPool
}
and in test package create absolutely the same object but change to:
package com.example
object Injection {
val uiContext : CourutineContext = Unconfined
val bgContext : CourutineContext = Unconfined
}
and inside your class it will be something like:
val data = async(Injection.bgContext) {service?.getData()}.await()