Develop Reference

Koltin Flow is repeating execution using flatMapMerge

I was trying to implement an approach to fetch products from two Data sources (Room & FirebaseFirestore) using Flows.
It was working fine until I noticed that the debugger was returning to the same break point infinitely. When the execution of "ViewmMdel.insertProducts(products)" ends, the debugger returns to Repository.getProducts(//) & repeats.
I changed the approach using only suspending functions & coroutines & works fine but I am curious about how I must to use Flows to implement this approach.
Maybe is only that flatMapMerge is in preview version.
Thanks in advance :D
This one is the implementation:
ViewModel:
fun getProductNames(companyName: String) {
viewModelScope.launch {
repository.getProducts(companyName).catch {
_event.value = AddSaleEvents.ShouldShowLoading(false)
_event.value = AddSaleEvents.ProductsFailureResponse(it.message.toString())
}.collect { products ->
productsList = products
if (products != emptyList<Product>()) {
_event.value = AddSaleEvents.ShouldShowLoading(false)
_event.value = AddSaleEvents.ProductsSuccessfulResponse(products)
insertProducts(products)
} else {
_event.value = AddSaleEvents.ShouldShowLoading(false)
_event.value = AddSaleEvents.ProductsSuccessfulResponse(products)
}
}
}
}
Repository:
#OptIn(FlowPreview::class)
override suspend fun getProducts(compnayName: String): Flow<List<Product>> {
return localDataSource.getProducts().flatMapMerge { list -> // LINE RUNNING INFINITELY
getProductsFromFirebase(list, compnayName)
}.flowOn(Dispatchers.IO).catch {
Log.d("Error", it.message.toString())
}
}
private fun getProductsFromFirebase(products: List<Product>, compnayName: String) = flow {
if (products.isEmpty()) {
remoteDataSource.getProducts(compnayName).collect {
emit(it)
}
} else {
emit(products)
}
}
LocalDataSource with Room:
override suspend fun getProducts(): Flow<List<Product>> = saleDao.getProducts()
Firebase Data Source:
override suspend fun getProducts(company: String): Flow<List<Product>> = flow {
val response = fireStore.collection("products").whereEqualTo("company", company).get()
response.await()
if (response.isSuccessful && !response.result.isEmpty) {
emit(response.result.toObjects(FirebaseProduct::class.java).toEntity())
}
}.catch {
Log.d("Error", it.message.toString())
}
How can I chain the response of a flow to trigger another one inside the MVVM Architecture + Clean Architecture?
6 if it is possible, I want to understand the reason the code is repeating infinitely.

Looks like insertProducts(products) triggers room's DAO.
So localDataSource.getProducts() is a observable read query
Observable queries are read operations that emit new values whenever there are changes to any of the tables that are referenced by the query.
Try to change LocalDataSource
interface SaleDao {
// fun getProducts(): Flow<List<Product>>
suspend fun getProducts(): List<Product>
}

State flow is not collecting emitted items

Imagine following scenario:
I open Search View and SearchViewModel is initialized
class SearchViewModel(
usecase: Usecase
) : ViewModel() {
init {
viewModelScope.launch {
usecase.initialize()
}
}
fun search(query: String) = viewModelScope.launch {
usecase.search(query)
}
}
User start typing characters calling search
class UseCase(
private val dataSource: DataSource
private val store: Store
) {
private val searchQueryEmitter = MutableStateFlow<String>("") // 2 change to MutableSharedFlow
private val searchQuery = searchQueryEmitter
.mapLatest { query -> dataSource.search(query) }
.onEach { store.update(it) }
.launchIn(CoroutineScope(Dispatchers.Default)) // 1 comment
suspend fun search(query: String) {
searchQueryEmitter.emit(query)
}
override suspend fun initialize() {
// searchQuery.launchIn(CoroutineScope(Dispatchers.Default)) // > Emit only initial value
// searchQuery.collect() // > Emit only initial value
}
}
Flow emits first item "" and next items according to query value
PROBLEM:
I don't understand if we comment launchIn (1), and call it later in initialize() method, then exactly the same searchQuery.launchIn(...) or searchQuery.collect() cause issue - flow emits only first item "", but calling search with query doesn't trigger emission of next items.
If we change StateFlow to SharedFlow no items will be emitted in any case.

The problem was in ViewModel. I was using two different instances of usecase. The one passed in parameter was used in init while the second one came from import
so the desired case with
override suspend fun initialize() {
searchQuery.collect()
}
is working right now

Kotlin Flow: emitAll is never collected

I am trying to write a UnitTest for the kotlin-version of networkBoundResource that can be found on serveral sources with several features
Here is my version of it with marker-comments for the following question.
inline fun <ResultType, RequestType> networkBoundResource(
...
coroutineDispatcher: CoroutineDispatcher
) = flow {
emit(Resource.loading(null)) // emit works!
val data = queryDatabase().firstOrNull()
val flow = if (shouldFetch(data)) {
emit(Resource.loading(data)) // emit works!
try {
saveFetchResult(fetch())
query().map { Resource.success(it) }
} catch (throwable: Throwable) {
onFetchFailed(throwable)
query().map { Resource.error(throwable.toString(), it) }
}
} else {
query().map { Resource.success(it) }
}
emitAll(flow) // emitAll does not work!
}.catch { exception ->
emit(Resource.error("An error occurred while fetching data! $exception", null))
}.flowOn(coroutineDispatcher)
This is one of my UnitTests for this code. The code is edited a bit to focus on my question:
#get:Rule
val testCoroutineRule = TestCoroutineRule()
private val coroutineDispatcher = TestCoroutineDispatcher()
#Test
fun networkBoundResource_noCachedData_shouldMakeNetworkCallAndStoreUserInDatabase() = testCoroutineRule.runBlockingTest {
...
// When getAuthToken is called
val result = networkBoundResource(..., coroutineDispatcher).toList()
result.forEach {
println(it)
}
}
The problem is that println(it) is only printing the Resource.loading(null) emissions. But if you have a look at the last line of the flow {} block, you will see that there should be another emission of the val flow. But this emission never arrives in my UnitTest. Why?

I'm not too sure of the complete behaviour, but essentially you want to get a resource, and current flow is all lumped into the FlowCollector<T> which makes it harder to reason and test.
I have never used or seen the Google code before and if I'm honest only glanced at it. My main take away was it had poor encapsulation and seems to break separations of concern - it manages the resource state, and handles all io work one one class. I'd prefer to have 2 different classes to separate that logic and allows for easier testing.
As simple pseudo code I would do something like this :
class ResourceRepository {
suspend fun get(r : Request) : Resource {
// abstract implementation details network request and io
// - this function should only fulfill the request
// can now be mocked for testing
delay(3_000)
return Resource.success(Any())
}
}
data class Request(val a : String)
sealed class Resource {
companion object {
val loading : Resource get() = Loading
fun success(a : Any) : Resource = Success(a)
fun error(t: Throwable) : Resource = Error(t)
}
object Loading : Resource()
data class Success(val a : Any) : Resource()
data class Error(val t : Throwable) : Resource()
}
fun resourceFromRequest(r : Request) : Flow<Resource> =
flow { emit(resourceRepository.get(r)) }
.onStart { emit(Resource.loading) }
.catch { emit(Resource.error(it)) }
This allows you to massively simplify the actual testing of the resourceFromRequest() function as you only have to mock the repository and one method. This allows you to abstract and deal with the networking and io work elsewhere, independently which again can be tested in isolation.

As #MarkKeen suggested, I now created my own implementation and it works quite well. Compared to the code that is going around on SO, this version now injects the coroutineDispatcher for easier testing, it lets flow take care of error handling, it does not contain nested flows and is imho easier to read and understand, too. There is still the side-effect of storing updated data to the database, but I am too tired now to tackle this.
import kotlinx.coroutines.CoroutineDispatcher
import kotlinx.coroutines.flow.*
inline fun <ResultType, RequestType> networkBoundResource(
crossinline query: () -> Flow<ResultType?>,
crossinline fetch: suspend () -> RequestType,
crossinline saveFetchResult: suspend (RequestType) -> Unit,
crossinline shouldFetch: (ResultType?) -> Boolean = { true },
coroutineDispatcher: CoroutineDispatcher
) = flow<Resource<ResultType>> {
// check for data in database
val data = query().firstOrNull()
if (data != null) {
// data is not null -> update loading status
emit(Resource.loading(data))
}
if (shouldFetch(data)) {
// Need to fetch data -> call backend
val fetchResult = fetch()
// got data from backend, store it in database
saveFetchResult(fetchResult)
}
// load updated data from database (must not return null anymore)
val updatedData = query().first()
// emit updated data
emit(Resource.success(updatedData))
}.onStart {
emit(Resource.loading(null))
}.catch { exception ->
emit(Resource.error("An error occurred while fetching data! $exception", null))
}.flowOn(coroutineDispatcher)
One possible UnitTest for this inline fun, which is used in an AuthRepsitory:
#ExperimentalCoroutinesApi
class AuthRepositoryTest {
companion object {
const val FAKE_ID_TOKEN = "FAkE_ID_TOKEN"
}
#get:Rule
val testCoroutineRule = TestCoroutineRule()
private val coroutineDispatcher = TestCoroutineDispatcher()
private val userDaoFake = spyk<UserDaoFake>()
private val mockApiService = mockk<MyApi>()
private val sut = AuthRepository(
userDaoFake, mockApiService, coroutineDispatcher
)
#Before
fun beforeEachTest() {
userDaoFake.clear()
}
#Test
fun getAuthToken_noCachedData_shouldMakeNetworkCallAndStoreUserInDatabase() = testCoroutineRule.runBlockingTest {
// Given an empty database
coEvery { mockApiService.getUser(any()) } returns NetworkResponse.Success(UserFakes.getNetworkUser(), null, HttpURLConnection.HTTP_OK)
// When getAuthToken is called
val result = sut.getAuthToken(FAKE_ID_TOKEN).toList()
coVerifyOrder {
// Then first try to fetch data from the DB
userDaoFake.get()
// Then fetch the User from the API
mockApiService.getUser(FAKE_ID_TOKEN)
// Then insert the user into the DB
userDaoFake.insert(any())
// Finally return the inserted user from the DB
userDaoFake.get()
}
assertThat(result).containsExactly(
Resource.loading(null),
Resource.success(UserFakes.getAppUser())
).inOrder()
}
}

How To Test PagingData From Paging 3

My ViewModel has a method which returns a flow of PagingData. In my app, the data is fetched from the remote server, which is then saved to Room (the single source of truth):
fun getChocolates(): Flow<PagingData<Chocolate>> {
val pagingSourceFactory = { dao().getChocolateListData() }
return Pager(
config = PagingConfig(
pageSize = NETWORK_PAGE_SIZE,
maxSize = MAX_MEMORY_SIZE,
enablePlaceholders = false
),
remoteMediator = ChocolateRemoteMediator(
api,
dao
),
pagingSourceFactory = pagingSourceFactory
).flow
}
How do I test this method? I want to test if the returning flow contains the correct data.
What I've tried so far:
#InternalCoroutinesApi
#Test
fun getChocolateListReturnsCorrectData() = runBlockingTest {
val chocolateListDao: ChocolateListDao by inject()
val chocolatesRepository: ChocolatesRepository by inject()
val chocolateListAdapter: ChocolateListAdapter by inject()
// 1
val chocolate1 = Chocolate(
name = "Dove"
)
val chocolate2 = Chocolate(
name = "Hershey's"
)
// 2
// You need to launch here because submitData suspends forever while PagingData is alive
val job = launch {
chocolatesRepository.getChocolateListStream().collectLatest {
chocolateListAdapter.submitData(it)
}
}
// Do some stuff to trigger loads
chocolateListDao.saveChocolate(chocolate1, chocolate2)
// How to read from adapter state, there is also .peek() and .itemCount
assertEquals(listOf(chocolate1, chocolate2).toMutableList(), chocolateListAdapter.snapshot())
// We need to cancel the launched job as coroutines.test framework checks for leaky jobs
job.cancel()
}
I'm wondering if I'm on the right track. Any help would be greatly appreciated!

I found using Turbine from cashapp would be much much easier.(JakeWharton comes to rescue again :P)
testImplementation "app.cash.turbine:turbine:0.2.1"
According to your code I think your test case should looks like:
#ExperimentalTime
#ExperimentalCoroutinesApi
#Test
fun `test if receive paged chocolate data`() = runBlockingTest {
val expected = listOf(
Chocolate(name = "Dove"),
Chocolate(name = "Hershey's")
)
coEvery {
dao().getChocolateListData()
}.returns(
listOf(
Chocolate(name = "Dove"),
Chocolate(name = "Hershey's")
)
)
launchTest {
viewModel.getChocolates().test(
timeout = Duration.ZERO,
validate = {
val collectedData = expectItem().collectData()
assertEquals(expected, collectedData)
expectComplete()
})
}
}
I also prepare a base ViewModelTest class for taking care of much of setup and tearDown tasks:
abstract class BaseViewModelTest {
#get:Rule
open val instantTaskExecutorRule = InstantTaskExecutorRule()
#get:Rule
open val testCoroutineRule = CoroutineTestRule()
#MockK
protected lateinit var owner: LifecycleOwner
private lateinit var lifecycle: LifecycleRegistry
#Before
open fun setup() {
MockKAnnotations.init(this)
lifecycle = LifecycleRegistry(owner)
every { owner.lifecycle } returns lifecycle
}
#After
fun tearDown() {
clearAllMocks()
}
protected fun initCoroutine(vm: BaseViewModel) {
vm.apply {
setViewModelScope(testCoroutineRule.testCoroutineScope)
setCoroutineContext(testCoroutineRule.testCoroutineDispatcher)
}
}
#ExperimentalCoroutinesApi
protected fun runBlockingTest(block: suspend TestCoroutineScope.() -> Unit) =
testCoroutineRule.runBlockingTest(block)
protected fun launchTest(block: suspend TestCoroutineScope.() -> Unit) =
testCoroutineRule.testCoroutineScope.launch(testCoroutineRule.testCoroutineDispatcher) { block }
}
As for extension function collectData() that's borrowed from answer from another post (Thanks #Farid!!)
And a slide show introducing turbine

There's basically two approaches to this depending on if you want pre-transformation or post-transformation data.
If you want to just assert the repository end, that your query is correct - you can just query PagingSource directly, this is pre-transform though so any mapping you do or filtering you do to PagingData in ViewModel won't be accounted for here. However, it's more "pure" if you want to test the query directly.
#Test
fun repo() = runBlockingTest {
val pagingSource = MyPagingSource()
val loadResult = pagingSource.load(...)
assertEquals(
expected = LoadResult.Page(...),
actual = loadResult,
)
}
The other way if you care about transforms, you need to load data from PagingData into a presenter API.
#Test
fun ui() = runBlockingTest {
val viewModel = ... // Some AndroidX Test rules can help you here, but also some people choose to do it manually.
val adapter = MyAdapter(..)
// You need to launch here because submitData suspends forever while PagingData is alive
val job = launch {
viewModel.flow.collectLatest {
adapter.submitData(it)
}
}
... // Do some stuff to trigger loads
advanceUntilIdle() // Let test dispatcher resolve everything
// How to read from adapter state, there is also .peek() and .itemCount
assertEquals(..., adapter.snapshot())
// We need to cancel the launched job as coroutines.test framework checks for leaky jobs
job.cancel()
}

How to enqueue sequential coroutines blocks

What I'm trying to do
I have an app that's using Room with Coroutines to save search queries in the database. It's also possible to add search suggestions and later on I retrieve this data to show them on a list. I've also made it possible to "pin" some of those suggestions.
My data structure is something like this:
#Entity(
tableName = "SEARCH_HISTORY",
indices = [Index(value = ["text"], unique = true)]
)
data class Suggestion(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "suggestion_id")
val suggestionId: Long = 0L,
val text: String,
val type: SuggestionType,
#ColumnInfo(name = "insert_date")
val insertDate: Calendar
)
enum class SuggestionType(val value: Int) {
PINNED(0), HISTORY(1), SUGGESTION(2)
}
I have made the "text" field unique to avoid repeated suggestions with different states/types. E.g.: A suggestion that's a pinned item and a previously queried text.
My Coroutine setup looks like this:
private val parentJob: Job = Job()
private val IO: CoroutineContext
get() = parentJob + Dispatchers.IO
private val MAIN: CoroutineContext
get() = parentJob + Dispatchers.Main
private val COMPUTATION: CoroutineContext
get() = parentJob + Dispatchers.Default
And my DAOs are basically like this:
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insert(obj: Suggestion): Long
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insert(objList: List<Suggestion>): List<Long>
I also have the following public functions to insert the data into the database:
fun saveQueryToDb(query: String, insertDate: Calendar) {
if (query.isBlank()) {
return
}
val suggestion = Suggestion(
text = query,
insertDate = insertDate,
type = SuggestionType.HISTORY
)
CoroutineScope(IO).launch {
suggestionDAO.insert(suggestion)
}
}
fun addPin(pin: String) {
if (pin.isBlank()) {
return
}
val suggestion = Suggestion(
text = pin,
insertDate = Calendar.getInstance(),
type = SuggestionType.PINNED
)
CoroutineScope(IO).launch {
suggestionDAO.insert(suggestion)
}
}
fun addSuggestions(suggestions: List<String>) {
addItems(suggestions, SuggestionType.SUGGESTION)
}
private fun addItems(items: List<String>, suggestionType: SuggestionType) {
if (items.isEmpty()) {
return
}
CoroutineScope(COMPUTATION).launch {
val insertDate = Calendar.getInstance()
val filteredList = items.filterNot { it.isBlank() }
val suggestionList = filteredList.map { History(text = it, insertDate = insertDate, suggestionType = suggestionType) }
withContext(IO) {
suggestionDAO.insert(suggestionList)
}
}
}
There are also some other methods, but let's focus on the ones above.
EDIT: All of the methods above are part of a lib that I made, they're are not made suspend because I don't want to force a particular type of programming to the user, like forcing to use Rx or Coroutines when using the lib.
The problem
Let's say I try to add a list of suggestions using the addSuggestions() method stated above, and that I also try to add a pinned suggestion using the addPin() method. The pinned text is also present in the suggestion list.
val list = getSuggestions() // Getting a list somewhere
addSuggestions(list)
addPin(list.first())
When I try to do this, sometimes the pin is added first and then it's overwritten by the suggestion present in the list, which makes me think I might've been dealing with some sort of race condition. Since the addSuggestions() method has more data to handle, and both methods will run in parallel, I believe the addPin() method is completing first.
Now, my Coroutines knowledge is pretty limited and I'd like to know if there's a way to enqueue those method calls and make sure they'll execute in the exact same order I invoked them, that must be strongly guaranteed to avoid overriding data and getting funky results later on. How can I achieve such behavior?

I'd follow the Go language slogan "Don't communicate by sharing memory; share memory by communicating", that means instead of maintaining atomic variables or jobs and trying to synchronize between them, model your operations as messages and use Coroutines actors to handle them.
sealed class Message {
data AddSuggestions(val suggestions: List<String>) : Message()
data AddPin(val pin: String) : Message()
}
And in your class
private val parentScope = CoroutineScope(Job())
private val actor = parentScope.actor<Message>(Dispatchers.IO) {
for (msg in channel) {
when (msg) {
is Message.AddSuggestions -> TODO("Map to the Suggestion and do suggestionDAO.insert(suggestions)")
is Message.AddPin -> TODO("Map to the Pin and do suggestionDAO.insert(pin)")
}
}
}
fun addSuggestions(suggestions: List<String>) {
actor.offer(Message.AddSuggestions(suggestions))
}
fun addPin(pin: String) {
actor.offer(Message.AddPin(pin))
}
By using actors you'll be able to queue messages and they will be processed in FIFO order.

By default when you call .launch{}, it launches a new coroutine without blocking the current thread and returns a reference to the coroutine as a Job. The coroutine is canceled when the resulting job is canceled.
It doesn't care or wait for other parts of your code it just runs.
But you can pass a parameter to basically tell it to run immediately or wait for other Coroutine to finish(LAZY).
For Example:
val work_1 = CoroutineScope(IO).launch( start = CoroutineStart.LAZY ){
//do dome work
}
val work_2 = CoroutineScope(IO).launch( start = CoroutineStart.LAZY ){
//do dome work
work_1.join()
}
val work_3 = CoroutineScope(IO).launch( ) {
//do dome work
work_2.join()
}
When you execute the above code first work_3 will finish and invoke work_2 when inturn invoke Work_1 and so on,
The summary of coroutine start options is:
DEFAULT -- immediately schedules coroutine for execution according to its context
LAZY -- starts coroutine lazily, only when it is needed
ATOMIC -- atomically (in a non-cancellable way) schedules coroutine for execution according to its context
UNDISPATCHED -- immediately executes coroutine until its first suspension point in the current thread.
So by default when you call .launch{} start = CoroutineStart.DEFAULT is passed because it is default parameter.

Don't launch coroutines from your database or repository. Use suspending functions and then switch dispatchers like:
suspend fun addPin(pin: String) {
...
withContext(Dispatchers.IO) {
suggestionDAO.insert(suggestion)
}
}
Then from your ViewModel (or Activity/Fragment) make the call:
fun addSuggestionsAndPinFirst(suggestions: List<Suggestion>) {
myCoroutineScope.launch {
repository.addSuggestions(suggestions)
repository.addPin(suggestions.first())
}
}
Why do you have a separate addPin() function anyways? You can just modify a suggestion and then store it:
fun pinAndStoreSuggestion(suggestion: Suggestion) {
myCoroutineScope.launch {
repository.storeSuggestion(suggestion.copy(type = SuggestionType.PINNED)
}
}
Also be careful using a Job like that. If any coroutine fails all your coroutines will be cancelled. Use a SupervisorJob instead. Read more on that here.
Disclaimer: I do not approve of the solution below. I'd rather use an old-fashioned ExecutorService and submit() my Runnable's
So if you really want to synchronize your coroutines in a way that the first function called is also the first one to write to your database. (I'm not sure it is guaranteed since your DAO functions are also suspending and Room uses it's own threads too). Try something like the following unit test:
class TestCoroutineSynchronization {
private val jobId = AtomicInteger(0)
private val jobToRun = AtomicInteger(0)
private val jobMap = mutableMapOf<Int, () -> Unit>()
#Test
fun testCoroutines() = runBlocking {
first()
second()
delay(2000) // delay so our coroutines finish
}
private fun first() {
val jobId = jobId.getAndIncrement()
CoroutineScope(SupervisorJob() + Dispatchers.Default).launch {
delay(1000) // intentionally delay your first coroutine
withContext(Dispatchers.IO) {
submitAndTryRunNextJob(jobId) { println(1) }
}
}
}
private fun second() {
val jobId = jobId.getAndIncrement()
CoroutineScope(SupervisorJob()).launch(Dispatchers.IO) {
submitAndTryRunNextJob(jobId) { println(2) }
}
}
private fun submitAndTryRunNextJob(jobId: Int, action: () -> Unit) {
synchronized(jobMap) {
jobMap[jobId] = action
tryRunNextJob()
}
}
private fun tryRunNextJob() {
var action = jobMap.remove(jobToRun.get())
while (action != null) {
action()
action = jobMap.remove(jobToRun.incrementAndGet())
}
}
}
So what I do on each call is increment a value (jobId) that is later used to prioritize what action to run first. Since you are using suspending function you probably need to add that modifier to the action submitted too (e.g. suspend () -> Unit).