Develop Reference

How to avoid concurrency issues with Kotlin coroutines?

I am going to implement a chat feature in the android application. In order to do that, I fetch chat messages every five seconds from the server by a coroutine flow. The problem is when I want to send a message sometimes the server receives two concurrent requests and it returns an error. How should I make sure that these requests run sequentially in my chat repository? Here is my chat repository:
class ChatRepositoryImpl #Inject constructor(
private val api: ApolloApi,
private val checkTokenIsSetDataStore: CheckTokenIsSetDataStore
) : ChatRepository {
override fun chatMessages(
lastIndex: Int,
limit: Int,
offset: Int,
channelId: Int,
): Flow<Resource<ChatMessages>> = flow {
var token = ""
checkTokenIsSetDataStore.get.first {
token = it
true
}
while (true) {
val response = ChatMessagesQuery(
lastIndex = Input.fromNullable(lastIndex),
limit = Input.fromNullable(limit),
offset = Input.fromNullable(offset),
channelId
).let {
api.getApolloClient(token)
.query(it)
.await()
}
response.data?.let {
emit(
Resource.Success<ChatMessages>(
it.chatMessages
)
)
}
if (response.data == null)
emit(Resource.Error<ChatMessages>(message = response.errors?.get(0)?.message))
delay(5000L)
}
}.flowOn(Dispatchers.IO)
override fun chatSendText(channelId: Int, text: String): Flow<Resource<ChatSendText>> = flow {
var token = ""
checkTokenIsSetDataStore.get.first {
token = it
true
}
val response = ChatSendTextMutation(
channelId = channelId,
text = text
).let {
api.getApolloClient(token)
.mutate(it)
.await()
}
response.data?.let {
return#flow emit(
Resource.Success<ChatSendText>(
it.chatSendText
)
)
}
return#flow emit(Resource.Error<ChatSendText>(message = response.errors?.get(0)?.message))
}.flowOn(Dispatchers.IO)
}

One way to limit concurrency is to use utils like Mutex or Semaphore. We can very easily solve your problem with mutex:
class ChatRepositoryImpl ... {
private val apolloMutex = Mutex()
override fun chatMessages(...) {
...
apolloMutex.withLock {
api.getApolloClient(token)
.query(it)
.await()
}
...
}
override fun chatSendText(...) {
...
apolloMutex.withLock {
api.getApolloClient(token)
.mutate(it)
.await()
}
...
}
However, this problem should not be really fixed on the client side, but on the server side. Your attempted solution doesn't protect you against concurrent requests entirely. If for some reasons two instances of the application has the same token or if the user attempts to manipulate your application, it could still send concurrent requests.
If you can't easily fix the problem properly, you can apply the same fix on the server side that you intend to apply on the client side. Just handle requests or part of requests sequentially. It is more error-proof and also more performant, because this way only part of the whole request time has to be done sequentially.

Concurrency problem with Coroutines and Socket Android

I have a chat application which of course works with Sockets. So i have build a SocketManager where i have the callbacks and the sendMethod from the implementation 'com.neovisionaries:nv-websocket-client:2.14' library
override fun sendMessage(text: String) {
println("## SEND: $text")
webSocket?.let {
it.sendText(text)
}
}
override fun onTextMessage(websocket: WebSocket?, message: String?) {
super.onTextMessage(websocket, message)
println("## RECEIVED: Something received")
try {
flowSocketHandler.webSocketEventResolver(s, message) {
sendMessage(it)
}
} catch (e: Exception) {
e.printStackTrace()
}
}
When a new text comes from Socket the FlowSocketHandler resolves the type of message and calls the proper handler to handle the message. For example, FileHandler for File messages, MEssageHandler for simple messages, VideoHandler for video call messages.
private val scope = CoroutineScope(Dispatchers.Main + SupervisorJob())
override fun webSocketEventResolver(server: Server, message: String, socketCallback: (message: String) -> Unit) {
scope.launch(Dispatchers.IO) {
try {
val json = JSONObject(message)
when(Enums.SocketResponses.toSocketEvent(json)) {
Enums.SocketResponses.MESSAGE_RECEIVED -> messagesHandler.onMessageReceived(server, json.fromJson(), true, socketCallback)
Enums.SocketResponses.FILE_PART_RECEIVED -> filesHandler.onFilePartReceived(server, json.fromJson(), socketCallback)
else -> {}
}
} catch (e: Exception) {
e.printStackTrace()
}
}
}
Also, the philopsophy of downloading an incoming File message, is that
I have an incoming Text message that says "Hey, you have to download a file that consists of 1000 parts" which calls the MESSAGE_RECEIVED
I send 1000 socket messages requesting each part accordingly
I receive each part from Socket and handle it which calls the FILE_PART_RECEIVED
As you can see above, all this is taking place in a scope CoroutineScope
All this is taking place inside the FilesHandler
class FilesHandlerImpl(private val appContext: Context): FilesHandler, KoinComponent {
private val scope = CoroutineScope(Dispatchers.Main + SupervisorJob())
override suspend fun onFileHeaderReceived(
server: Server,
receiver: ReceiveNewMessage,
isFromSocket: Boolean,
socketCallback: ((message: String) -> Unit)?
) {
scope.launch(Dispatchers.IO) {
// Do some checks and start send requests for each part
println("## Time header message ${receiver.fileHash}")
pendingList.forEach { p ->
socketCallback?.invoke(requestFilePart(p))
}
// So here in the Log i see
// ## Send {json for each part}
}
}
// Take the part and create a file
override suspend fun onFilePartReceived(
server: Server,
receiver: FilePartRcv,
socketCallback: (message: String) -> Unit
) {
scope.launch(Dispatchers.IO) {
println("## File part received ${receiver.filePart.segment}")
filesRepository.createAndWriteFilePart(server, receiver)
filesRepository.updateFilePartStatus(server, receiver, FILE_PART_RECEIVED)
if (complete) {
// Do stuff
}
}
// So here in every part we receive we see in the Log
// ## File part received 1
// ## File part received 4
// ## File part received 2
// ## File part received 6
// ## File part received 9
......
}
}
The problem is the Following.
While the client sends requests for the File parts (So Log is full of ## SEND {...}) meanwhile the Socket callback onTextMessage is called so i see also ## RECEIVED: Something received.
Those callbacks are the incoming File Parts. So i should also see the ## File part received X. But i don't see any of those UNTIL the Send loop finishes. Then suddenly i see all the ## File part received X.
What i did, is to remove the scope from onFileHeaderReceived and onFilePartReceived and seems to play a bit smoother.
Can anybody explain me why is that happening?

scope.launch starts a job asynchronously. it returns a Job that you can hold onto and cancel on await.
If you just want to ensure the job happens on the IO Dispatcher, then you probably want withContext instead. This will be blocking so operations will happen sequentially.
withContext(Dispatchers.IO) {
...
}
Dispatchers.Main will typically be single threaded also, so forming a queue as you suggest. You could use Dispatchers.IO or Default to have concurrency.
https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-main.html
A coroutine dispatcher that is confined to the Main thread operating with UI objects. Usually such dispatchers are single-threaded.
Also suspend isn't needed is it just calls scope.launch, since that isn't a suspending function.

Android WorkManager - Can I pass input data to a Worker that runs periodically?

In my app I start a WebSocketWorker tasks that runs periodically every 15 minutes. As the name implies, it contains a WebSocket for listening to a socket in the background:
// MainApplication.kt
override fun onCreate() {
super.onCreate()
if (BuildConfig.DEBUG) {
Timber.plant(DebugTree())
}
val work = PeriodicWorkRequestBuilder<WebSocketWorker>(15, TimeUnit.MINUTES).build()
workManager.enqueueUniquePeriodicWork("UniqueWebSocketWorker", ExistingPeriodicWorkPolicy.KEEP, work)
}
The WebSocketWorker contains the following logic:
#HiltWorker
class WebSocketWorker #AssistedInject constructor(
#Assisted appContext: Context,
#Assisted workerParams: WorkerParameters
) : CoroutineWorker(appContext, workerParams) {
inner class MyWebSocketListener : WebSocketListener() {
override fun onMessage(webSocket: WebSocket, text: String) {
Timber.d("The message sent is %s", text)
// do sth. with the message
}
override fun onFailure(webSocket: WebSocket, t: Throwable, response: Response?) {
t.localizedMessage?.let { Timber.e("onFailure: %s", it) }
response?.message?.let { Timber.e("onFailure: %s", it) }
}
}
override suspend fun doWork(): Result {
try{
// code to be executed
val request = Request.Builder().url("ws://***.***.**.***:8000/ws/chat/lobby/").build()
val myWebSocketListener = MyWebSocketListener()
val client = OkHttpClient()
client.newWebSocket(request, myWebSocketListener)
return Result.success()
}
catch (throwable:Throwable){
Timber.e("There is a failure")
Timber.e("throwable.localizedMessage: %s", throwable.localizedMessage)
// clean up and log
return Result.failure()
}
}
}
As you can see, in the Worker class I set the WebSocket and everything is fine. Listening to the socket works.
Now, I also want to add the "sending of messages" functionality to my app. How can I reuse the websocket created in WebSocketWorker? Can I pass input data to the WebSocketWorker that runs in the background ?
Let's say I have a EditText for typing the message and a Button to send the message with a setOnClickListener attached like this:
binding.sendButton.setOnClickListener {
// get message
val message = binding.chatMessageEditText.text.toString()
// check if not empty
if(message.isNotEmpty()) {
// HOW CAN I REUSE THE WEBSOCKET RUNNING PERIODICALLY IN THE BACKGROUND?
// CAN I PASS THE MESSAGE TO THAT WEBSOCKET ?
// OR SHOULD I CREATE A DIFFERENT WORKER FOR SENDING MESSAGES (e.g.: a OneTimeRequest<SendMessageWorker> for sending messages ?
}
}
From the documentation, I know that you need to build Data objects for passing inputs and so on but there was no example which showcased how to pass input to a worker running periodically in the background.

My experience is saying that you can. Basically you "can't" interact with the worker object via the API. It is really annoying.
For example, with the JS you have the option to get a job and check the parameters of the job. There is no such option with the work. For example, I want to check what is the current state of the restrictions - what is satisfied, what is not. Nothing like this. You can just check states, cancel and that is almost all.
My suggestions is that it is because the WorkManager is a "facade/adapter" over other libraries like JS. It has it's own DB to restore JS jobs on device restart and stuff like this, but beside that if you want to interact with the internals I guess it was just too complicated for them to do so they just skipped.
You can just inject some other object and every time the work can ask it for it's data. I don't see other option.

NsdManager discovers, immediately loses and re-discovers same service after WiFi is re-enabled on Android phone

In my Android app I have to show a list of available services on the network published by another machine (RPi 3B con Raspbian Stretch) using avahi 0.6.32 (Bonjour/zeroconf daemon for Linux). I obtain the list on Android phone using NsdManager.
But, during testing, I'm getting a strange behavior: when I switch WiFi off and back on in the phone, most of the times the services are discovered, then immediately lost and then rediscovered (instead of just discovered once) and all of this in less than a second.
This causes the list of services to briefly appear on screen, then disappear and finally reappear almost immediately, but it's still very noticeable. And it forces the services to be discovered and resolved twice. As I expect to have lots of phones connecting to several services in the same LAN, I want to avoid overloading the network.
I'm not sure if I'm doing something wrong or it's just the way NsdManager works on Android. To reduce possible sources of problem, I commented out the lines that resolve the services (leaving only the log messages) but the problem persisted (more then half of the times).
How can I solve it?
Sample extract from Logcat:
2019-09-26 04:33:50.262 27300-27420/com.example.myapp D/NsdHelper$initializeDiscoveryListener: Service discovery success: name: MyService-1490247, type: _mytype._tcp., host: null, port: 0, txtRecord:
2019-09-26 04:33:50.879 27300-27420/com.example.myapp D/NsdHelper$initializeDiscoveryListener: Service lost: name: MyService-1490247, type: _mytype._tcp., host: null, port: 0, txtRecord:
2019-09-26 04:33:50.970 27300-27420/com.example.myapp D/NsdHelper$initializeDiscoveryListener: Service discovery success: name: MyService-1490247, type: _mytype._tcp., host: null, port: 0, txtRecord:
I'm testing on a Samsung Note 8 with Android O. I tried with 2 different WiFi routers and the behavior is the same.
I'm using the following NsdHelper class (in Kotlin):
import android.content.Context
import android.net.nsd.NsdManager
import android.net.nsd.NsdServiceInfo
import timber.log.Timber
import java.util.*
import java.util.concurrent.ConcurrentLinkedQueue
import java.util.concurrent.atomic.AtomicBoolean
import kotlin.collections.ArrayList
abstract class NsdHelper(val context: Context) {
// Declare DNS-SD related variables for service discovery
val nsdManager: NsdManager? = context.getSystemService(Context.NSD_SERVICE) as NsdManager?
private var discoveryListener: NsdManager.DiscoveryListener? = null
private var resolveListener: NsdManager.ResolveListener? = null
private var resolveListenerBusy = AtomicBoolean(false)
private var pendingNsdServices = ConcurrentLinkedQueue<NsdServiceInfo>()
var resolvedNsdServices: MutableList<NsdServiceInfo> =
Collections.synchronizedList(ArrayList<NsdServiceInfo>())
companion object {
// Type of services to look for
const val NSD_SERVICE_TYPE: String = "_mytype._tcp."
// Services' Names must start with this
const val NSD_SERVICE_NAME: String = "MyService-"
}
// Initialize Listeners
fun initializeNsd() {
// Initialize only resolve listener
initializeResolveListener()
}
// Instantiate DNS-SD discovery listener
// used to discover available Sonata audio servers on the same network
private fun initializeDiscoveryListener() {
Timber.d("Initialize DiscoveryListener")
// Instantiate a new DiscoveryListener
discoveryListener = object : NsdManager.DiscoveryListener {
override fun onDiscoveryStarted(regType: String) {
// Called as soon as service discovery begins.
Timber.d("Service discovery started: $regType")
}
override fun onServiceFound(service: NsdServiceInfo) {
// A service was found! Do something with it
Timber.d("Service discovery success: $service")
if ( service.serviceType == NSD_SERVICE_TYPE &&
service.serviceName.startsWith(NSD_SERVICE_NAME) ) {
// Both service type and service name are the ones we want
// If the resolver is free, resolve the service to get all the details
if (resolveListenerBusy.compareAndSet(false, true)) {
nsdManager?.resolveService(service, resolveListener)
} else {
// Resolver was busy. Add the service to the list of pending services
pendingNsdServices.add(service)
}
} else {
// Not our service. Log message but do nothing else
Timber.d("Not our Service - Name: ${service.serviceName}, Type: ${service.serviceType}")
}
}
override fun onServiceLost(service: NsdServiceInfo) {
Timber.d("Service lost: $service")
// If the lost service was in the queue of pending services, remove it
synchronized(pendingNsdServices) {
val iterator = pendingNsdServices.iterator()
while (iterator.hasNext()) {
if (iterator.next().serviceName == service.serviceName) iterator.remove()
}
}
// If the lost service was in the list of resolved services, remove it
synchronized(resolvedNsdServices) {
val iterator = resolvedNsdServices.iterator()
while (iterator.hasNext()) {
if (iterator.next().serviceName == service.serviceName) iterator.remove()
}
}
// Do the rest of the processing for the lost service
onNsdServiceLost(service)
}
override fun onDiscoveryStopped(serviceType: String) {
Timber.d("Discovery stopped: $serviceType")
}
override fun onStartDiscoveryFailed(serviceType: String, errorCode: Int) {
Timber.e("Start Discovery failed: Error code: $errorCode")
stopDiscovery()
}
override fun onStopDiscoveryFailed(serviceType: String, errorCode: Int) {
Timber.e("Stop Discovery failed: Error code: $errorCode")
nsdManager?.stopServiceDiscovery(this)
}
}
}
// Instantiate DNS-SD resolve listener to get extra information about the service
private fun initializeResolveListener() {
Timber.d("Initialize ResolveListener")
resolveListener = object : NsdManager.ResolveListener {
override fun onServiceResolved(service: NsdServiceInfo) {
Timber.d("Service Resolve Succeeded: $service")
// Register the newly resolved service into our list of resolved services
resolvedNsdServices.add(service)
// Process the newly resolved service
onNsdServiceResolved(service)
// Process the next service waiting to be resolved
resolveNextInQueue()
}
override fun onResolveFailed(serviceInfo: NsdServiceInfo, errorCode: Int) {
// Called when the resolve fails. Use the error code to debug.
Timber.e("Resolve failed: $serviceInfo - Error code: $errorCode")
// Process the next service waiting to be resolved
resolveNextInQueue()
}
}
}
// Start discovering services on the network
fun discoverServices() {
// Cancel any existing discovery request
stopDiscovery()
initializeDiscoveryListener()
// Start looking for available audio channels in the network
nsdManager?.discoverServices(NSD_SERVICE_TYPE, NsdManager.PROTOCOL_DNS_SD, discoveryListener)
}
// Stop DNS-SD service discovery
fun stopDiscovery() {
if (discoveryListener != null) {
Timber.d("stopDiscovery() called")
try {
nsdManager?.stopServiceDiscovery(discoveryListener)
} finally {
}
discoveryListener = null
}
}
// Resolve next NSD service pending resolution
private fun resolveNextInQueue() {
// Get the next NSD service waiting to be resolved from the queue
val nextNsdService = pendingNsdServices.poll()
if (nextNsdService != null) {
// There was one. Send to be resolved.
nsdManager?.resolveService(nextNsdService, resolveListener)
} else {
// There was no pending service. Release the flag
resolveListenerBusy.set(false)
}
}
// Function to be overriden with custom logic for new service resolved
abstract fun onNsdServiceResolved(service: NsdServiceInfo)
// Function to be overriden with custom logic for service lost
abstract fun onNsdServiceLost(service: NsdServiceInfo)
}
On the init block of the View Model, I start the service discovery:
class MyViewModel(application: Application) : AndroidViewModel(application) {
// Get application context
private val myAppContext: Context = getApplication<Application>().applicationContext
// Declare NsdHelper object for service discovery
private val nsdHelper: NsdHelper? = object : NsdHelper(myAppContext) {
override fun onNsdServiceResolved(service: NsdServiceInfo) {
// A new network service is available
// Update list of available services
updateServicesList()
}
override fun onNsdServiceLost(service: NsdServiceInfo) {
// A network service is no longer available
// Update list of available services
updateServicesList()
}
}
// Block that is run when the view model is created
init {
Timber.d("init block called")
// Initialize DNS-SD service discovery
nsdHelper?.initializeNsd()
// Start looking for available audio channels in the network
nsdHelper?.discoverServices()
}
// Called when the view model is destroyed
override fun onCleared() {
Timber.d("onCleared called")
nsdHelper?.stopDiscovery()
super.onCleared()
}
private fun updateServicesList() {
// Put the logic here to show the services on screen
return
}
}
Note: Timber is a logging utility, almost a direct replacement for standard Log commands, but easier to use.

Kotlin callback not called within async function

I am attempting to subscribe to multiple characteristics of a BLE peripheral within Android API 28.
Due to the asynchronous nature of the BLE API I need to make the function that subscribes to each characteristic (gatt.writeDescriptor()) block; otherwise the BLE API will attempt to subscribe to multiple characteristics at once, despite the fact that only one descriptor can be written at a time: meaning that only one characteristic is ever subscribed to.
The blocking is achieved by overriding the onServicesDiscovered callback and calling an asynchronous function to loop through and subscribe to characteristics. This is blocked with a simple Boolean value (canContinue). Unfortunately, the callback function onDescriptorWrite is never called.
See the code below:
override fun onDescriptorWrite(gatt: BluetoothGatt, descriptor: BluetoothGattDescriptor, status: Int) {
canContinue = true
}
override fun onServicesDiscovered(gatt: BluetoothGatt, status: Int) {
runBlocking {
loopAsync(gatt)
}
}
private suspend fun loopAsync(gatt: BluetoothGatt) {
coroutineScope {
async {
gatt.services.forEach { gattService ->
gattService.characteristics.forEach { gattChar ->
CHAR_LIST.forEach {
if (gattChar.uuid.toString().contains(it)) {
canContinue = false
gatt.setCharacteristicNotification(gattChar, true)
val descriptor = gattChar.getDescriptor(UUID.fromString(BleNamesResolver.CLIENT_CHARACTERISTIC_CONFIG))
descriptor.value = BluetoothGattDescriptor.ENABLE_NOTIFICATION_VALUE
val write = Runnable {
gatt.writeDescriptor(descriptor)
}
//private val mainHandler = Handler(Looper.getMainLooper())
//mainHandler.post(write)
//runOnUiThread(write)
gatt.writeDescriptor(descriptor)
}
while (!canContinue)
}
}
}
}
}
}
It was suggested in a related post that I run the gatt.writeDescriptor() function in the main thread. As you can see in the code above I have tried this to no avail using both runOnUiThread() and creating a Handler object following suggestions from this question.
The callback gets called if I call gatt.writeDescriptor() from a synchronous function, I have no idea why it doesn't get called from an asynchronous function.
EDIT: It appears that the while(!canContinue); loop is actually blocking the callback. If I comment this line out, the callback triggers but then I face the same issue as before. How can I block this function?
Any suggestions are most welcome! Forgive my ignorance, but I am very much used to working on embedded systems, Android is very much a new world to me!
Thanks,
Adam

I posted some notes in the comments but I figured it would be better to format it as an answer.
Even though you already fixed your issue I'd suggest running the actual coroutine asynchronously and inside of it wait for the write notification using channels
private var channel: Channel<Boolean> = Channel()
override fun onDescriptorWrite(gatt: BluetoothGatt, descriptor: BluetoothGattDescriptor, status: Int) {
GlobalScope.async {
channel.send(true)
}
}
override fun onServicesDiscovered(gatt: BluetoothGatt, status: Int) {
GlobalScope.async {
loopAsync(gatt)
}
}
private suspend fun loopAsync(gatt: BluetoothGatt) {
gatt.services.forEach { gattService ->
gattService.characteristics.forEach { gattChar ->
CHAR_LIST.forEach {
if (gattChar.uuid.toString().contains(it)) {
gatt.setCharacteristicNotification(gattChar, true)
val descriptor = gattChar.getDescriptor(UUID.fromString(BleNamesResolver.CLIENT_CHARACTERISTIC_CONFIG))
descriptor.value = BluetoothGattDescriptor.ENABLE_NOTIFICATION_VALUE
gatt.writeDescriptor(descriptor)
channel.receive()
}
}
}
}
}

So I actually figured out the answer myself.
The while(!canContinue); loop was actually blocking the callback as it was running in the main thread and took priority over the callback required to set the canContinue variable.
This was solved simply by calling both the gatt.writeDescriptor() function and the while loop from within the main thread:
val subscribe = Runnable {
gatt.writeDescriptor(descriptor)
while (!canContinue);
}
runOnUiThread(subscribe)