What I'm trying to do
I have an app that's using Room with Coroutines to save search queries in the database. It's also possible to add search suggestions and later on I retrieve this data to show them on a list. I've also made it possible to "pin" some of those suggestions.
My data structure is something like this:
#Entity(
tableName = "SEARCH_HISTORY",
indices = [Index(value = ["text"], unique = true)]
)
data class Suggestion(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "suggestion_id")
val suggestionId: Long = 0L,
val text: String,
val type: SuggestionType,
#ColumnInfo(name = "insert_date")
val insertDate: Calendar
)
enum class SuggestionType(val value: Int) {
PINNED(0), HISTORY(1), SUGGESTION(2)
}
I have made the "text" field unique to avoid repeated suggestions with different states/types. E.g.: A suggestion that's a pinned item and a previously queried text.
My Coroutine setup looks like this:
private val parentJob: Job = Job()
private val IO: CoroutineContext
get() = parentJob + Dispatchers.IO
private val MAIN: CoroutineContext
get() = parentJob + Dispatchers.Main
private val COMPUTATION: CoroutineContext
get() = parentJob + Dispatchers.Default
And my DAOs are basically like this:
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insert(obj: Suggestion): Long
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insert(objList: List<Suggestion>): List<Long>
I also have the following public functions to insert the data into the database:
fun saveQueryToDb(query: String, insertDate: Calendar) {
if (query.isBlank()) {
return
}
val suggestion = Suggestion(
text = query,
insertDate = insertDate,
type = SuggestionType.HISTORY
)
CoroutineScope(IO).launch {
suggestionDAO.insert(suggestion)
}
}
fun addPin(pin: String) {
if (pin.isBlank()) {
return
}
val suggestion = Suggestion(
text = pin,
insertDate = Calendar.getInstance(),
type = SuggestionType.PINNED
)
CoroutineScope(IO).launch {
suggestionDAO.insert(suggestion)
}
}
fun addSuggestions(suggestions: List<String>) {
addItems(suggestions, SuggestionType.SUGGESTION)
}
private fun addItems(items: List<String>, suggestionType: SuggestionType) {
if (items.isEmpty()) {
return
}
CoroutineScope(COMPUTATION).launch {
val insertDate = Calendar.getInstance()
val filteredList = items.filterNot { it.isBlank() }
val suggestionList = filteredList.map { History(text = it, insertDate = insertDate, suggestionType = suggestionType) }
withContext(IO) {
suggestionDAO.insert(suggestionList)
}
}
}
There are also some other methods, but let's focus on the ones above.
EDIT: All of the methods above are part of a lib that I made, they're are not made suspend because I don't want to force a particular type of programming to the user, like forcing to use Rx or Coroutines when using the lib.
The problem
Let's say I try to add a list of suggestions using the addSuggestions() method stated above, and that I also try to add a pinned suggestion using the addPin() method. The pinned text is also present in the suggestion list.
val list = getSuggestions() // Getting a list somewhere
addSuggestions(list)
addPin(list.first())
When I try to do this, sometimes the pin is added first and then it's overwritten by the suggestion present in the list, which makes me think I might've been dealing with some sort of race condition. Since the addSuggestions() method has more data to handle, and both methods will run in parallel, I believe the addPin() method is completing first.
Now, my Coroutines knowledge is pretty limited and I'd like to know if there's a way to enqueue those method calls and make sure they'll execute in the exact same order I invoked them, that must be strongly guaranteed to avoid overriding data and getting funky results later on. How can I achieve such behavior?
I'd follow the Go language slogan "Don't communicate by sharing memory; share memory by communicating", that means instead of maintaining atomic variables or jobs and trying to synchronize between them, model your operations as messages and use Coroutines actors to handle them.
sealed class Message {
data AddSuggestions(val suggestions: List<String>) : Message()
data AddPin(val pin: String) : Message()
}
And in your class
private val parentScope = CoroutineScope(Job())
private val actor = parentScope.actor<Message>(Dispatchers.IO) {
for (msg in channel) {
when (msg) {
is Message.AddSuggestions -> TODO("Map to the Suggestion and do suggestionDAO.insert(suggestions)")
is Message.AddPin -> TODO("Map to the Pin and do suggestionDAO.insert(pin)")
}
}
}
fun addSuggestions(suggestions: List<String>) {
actor.offer(Message.AddSuggestions(suggestions))
}
fun addPin(pin: String) {
actor.offer(Message.AddPin(pin))
}
By using actors you'll be able to queue messages and they will be processed in FIFO order.
By default when you call .launch{}, it launches a new coroutine without blocking the current thread and returns a reference to the coroutine as a Job. The coroutine is canceled when the resulting job is canceled.
It doesn't care or wait for other parts of your code it just runs.
But you can pass a parameter to basically tell it to run immediately or wait for other Coroutine to finish(LAZY).
For Example:
val work_1 = CoroutineScope(IO).launch( start = CoroutineStart.LAZY ){
//do dome work
}
val work_2 = CoroutineScope(IO).launch( start = CoroutineStart.LAZY ){
//do dome work
work_1.join()
}
val work_3 = CoroutineScope(IO).launch( ) {
//do dome work
work_2.join()
}
When you execute the above code first work_3 will finish and invoke work_2 when inturn invoke Work_1 and so on,
The summary of coroutine start options is:
DEFAULT -- immediately schedules coroutine for execution according to its context
LAZY -- starts coroutine lazily, only when it is needed
ATOMIC -- atomically (in a non-cancellable way) schedules coroutine for execution according to its context
UNDISPATCHED -- immediately executes coroutine until its first suspension point in the current thread.
So by default when you call .launch{} start = CoroutineStart.DEFAULT is passed because it is default parameter.
Don't launch coroutines from your database or repository. Use suspending functions and then switch dispatchers like:
suspend fun addPin(pin: String) {
...
withContext(Dispatchers.IO) {
suggestionDAO.insert(suggestion)
}
}
Then from your ViewModel (or Activity/Fragment) make the call:
fun addSuggestionsAndPinFirst(suggestions: List<Suggestion>) {
myCoroutineScope.launch {
repository.addSuggestions(suggestions)
repository.addPin(suggestions.first())
}
}
Why do you have a separate addPin() function anyways? You can just modify a suggestion and then store it:
fun pinAndStoreSuggestion(suggestion: Suggestion) {
myCoroutineScope.launch {
repository.storeSuggestion(suggestion.copy(type = SuggestionType.PINNED)
}
}
Also be careful using a Job like that. If any coroutine fails all your coroutines will be cancelled. Use a SupervisorJob instead. Read more on that here.
Disclaimer: I do not approve of the solution below. I'd rather use an old-fashioned ExecutorService and submit() my Runnable's
So if you really want to synchronize your coroutines in a way that the first function called is also the first one to write to your database. (I'm not sure it is guaranteed since your DAO functions are also suspending and Room uses it's own threads too). Try something like the following unit test:
class TestCoroutineSynchronization {
private val jobId = AtomicInteger(0)
private val jobToRun = AtomicInteger(0)
private val jobMap = mutableMapOf<Int, () -> Unit>()
#Test
fun testCoroutines() = runBlocking {
first()
second()
delay(2000) // delay so our coroutines finish
}
private fun first() {
val jobId = jobId.getAndIncrement()
CoroutineScope(SupervisorJob() + Dispatchers.Default).launch {
delay(1000) // intentionally delay your first coroutine
withContext(Dispatchers.IO) {
submitAndTryRunNextJob(jobId) { println(1) }
}
}
}
private fun second() {
val jobId = jobId.getAndIncrement()
CoroutineScope(SupervisorJob()).launch(Dispatchers.IO) {
submitAndTryRunNextJob(jobId) { println(2) }
}
}
private fun submitAndTryRunNextJob(jobId: Int, action: () -> Unit) {
synchronized(jobMap) {
jobMap[jobId] = action
tryRunNextJob()
}
}
private fun tryRunNextJob() {
var action = jobMap.remove(jobToRun.get())
while (action != null) {
action()
action = jobMap.remove(jobToRun.incrementAndGet())
}
}
}
So what I do on each call is increment a value (jobId) that is later used to prioritize what action to run first. Since you are using suspending function you probably need to add that modifier to the action submitted too (e.g. suspend () -> Unit).
Related
I have a common situation of getting data. I use the Kotlin Coroutines.
1 variant:
class SomeViewModel(
private val gettingData: GetDataUseCase
) : ViewModel() {
lateinit var data: List<String>
init {
viewModelScope.launch {
data = gettingData.get()
}
}
}
2 variant:
class SomeViewModel(
private val gettingData: GetDataUseCase
) : ViewModel() {
val data = MutableStateFlow<List<String>?>(null)
init {
viewModelScope.launch {
data.emit(gettingData.get())
}
}
}
How can I initialize a data field not delayed, but immediately, with the viewModelScope but without a lateinit or nullble field? And without LiveData, my progect uses Coroutine Flow
I can't return a result of viewModelScope job in .run{} or by lazy {}.
I cant return a result drom fun:
val data: List<String> = getData()
fun getData(): List<String> {
viewModelScope.launch {
data = gettingData.get()
}
return ???
}
Also I can't make suspend fun getData() because I can't create coroutineScope in initialisation'
You're describing an impossibility. Presumably, gettingData.get() is defined as a suspend function, meaning the result literally cannot be retrieved immediately. Since it takes a while to retrieve, you cannot have an immediate value.
This is why apps and websites have loading indicators in their UI.
If you're using Flows, you can use a Flow with a nullable type (like in your option 2 above), and in your Activity/Fragment, in the collector, you show either a loading indicator or your data depending on whether it is null.
Your code 2 can be simplified using the flow builder and stateIn with a null default value:
class SomeViewModel(
private val gettingData: GetDataUseCase
) : ViewModel() {
val data = flow<List<String>?> { emit(gettingData.get()) }
.stateIn(viewModelScope, SharingStarted.Eagerly, null)
}
In your Activity or Fragment:
viewLifecycleOwner.lifecycleScope.launch {
viewModel.data
.flowWithLifecycle(viewLifecycleOwner.lifecycle, Lifecycle.State.STARTED)
.collect { list ->
if(list == null) {
// Show loading indicator in UI
} else {
// Show the data
}
}
}
If your data loads pretty quickly, instead of making the type nullable, you can just make the default value emptyList(). Then your collector can just not do anything when the list is empty. This works if the data loads quickly enough that the user isn't going to wonder if something is wrong because the screen is blank for so long.
You have to use SharedFlow with replay 1 (to store last value and replay it for a new subscriber) to implement it.
My sample:
interface DataSource {
suspend fun getData(): Int
}
class DataViewModel(dataSource: DataSource): ViewModel() {
val dataField =
flow<Int> {
emit(dataSource.getData())
}.shareIn(viewModelScope, SharingStarted.WhileSubscribed(1000), 1)
}
I'm trying to show a user information in DetailActivity. So, I request a data and get a data for the user from server. but in this case, the return type is Flow<User>. Let me show you the following code.
ServiceApi.kt
#GET("endpoint")
suspend fun getUser(#Query("id") id: Int): Response<User>
Repository.kt
fun getUser(id: Int): Flow<User> = flow<User> {
val userResponse = api.getUser(id = id)
if (userResponse.isSuccessful) {
val user = userResponse.body()
emit(user)
}
}
.flowOn(Dispatchers.IO)
.catch { // send error }
DetailViewModel.kt
class DetailViewModel(
private val repository : Repository
) {
val uiState: StateFlow<User> = repository.getUser(id = 369).stateIn(
scope = viewModelScope,
started = SharingStarted.WhileSubscribed(5000),
initialValue = User() // empty user
)
}
DetailActivity.kt
class DetailActivity: AppCompatActivity() {
....
initObersevers() {
lifecycleScope.launch {
// i used the `flowWithLifecycle` because the data is just a single object.
viewModel.uiState.flowWithLifecycle(lifecycle, Lifecycle.State.STARTED).collect { state ->
// show data
}
}
}
...
}
But, all of sudden, I just realized that this process is just an one-shot operation and thought i can use suspend function and return User in Repository.kt.
So, i changed the Repository.kt.
Repository.kt(changed)
suspend fun getUser(id: Int): User {
val userResponse = api.getUser(id = id)
return if(userResponse.isSuccessful) {
response.body()
} else {
User() // empty user
}
}
And in DetailViewModel, i want to convert the User into StateFlow<User> because of observing from DetailActivity and I'm going to use it the same way as before by using flowWithLifecycle.
the concept is... i thought it's just one single data and i dind't need to use Flow in Repository. because it's not several items like List.
is this way correct or not??
Yeap, this one-time flow doesn't make any sense - it emits only once and that's it.
You've got two different ways. First - is to create a state flow in your repo and emit there any values each time you're doing your GET request. This flow will be exposed to the use case and VM levels. I would say that it leads to more difficult error handling (I'm not fond of this way, but these things are always arguable, haha), but it also has some pros like caching, you can always show/get the previous results.
Second way is to leave your request as a simple suspend function which sends a request, returns the result of it back to your VM (skipping error handling here to be simple):
val userFlow: Flow<User>
get() = _userFlow
private val _userFlow = MutableStateFlow(User())
fun getUser() = launch(viewModelScope) {
_userFlow.value = repository.getUser()
}
This kind of implementation doesn't provide any cache out of scope of this VM's lifecycle, but it's easy to test and use.
So it's not like there is only one "the-coolest-way-to-do-it", it's rather a question what suits you more for your needs.
I want to debounce the items sent to a shared flow, and consume them after that. Something like this:
private var flow = MutableSharedFlow()
suspend fun search(query: String): Flow<Result> {
flow.emit(query)
return flow.debounce(1000).map{ executeSearch(it) }
}
The event that initiates the search is a user writing on a field. For each character, the search function is called. So I want to get a debounced result, to avoid many queries to the server.
It looks like the debounce operator returns a different flow instance each time, so that all the queries end up invoking the executeSearch() function, without dropping any of them as you could expect by using a debounce operator. How can I achieve a functionality like this, so that a client can invoke a function that returns a flow with debounced results?
You can try something like this:
private var flow = MutableSharedFlow()
init {
flow.debounce(1000)
.collect {
val result = executeSearch(it)
// Process the result (maybe send to the UI)
}
}
suspend fun search(query: String) {
flow.emit(query)
}
With two flows you could do it like this. One backing flow takes all the search inputs, and the second is a debounce version of it that runs the query. The search function doesn’t return a flow because the Flow is already available as a property and we aren’t creating new ones for each input.
private val searchInput = MutableSharedFlow<String>()
val searchResults = searchInput.debounce(1000)
.map { executeSearch(it) }
.shareIn(viewModelScope, SharingStarted.Eagerly)
fun submitSearchInput(query: String) {
searchInput.tryEmit(query)
}
You could alternatively do it with jobs that you extinguish when new inputs come in:
private val searchJob: Job? = null
private val _searchResults = MutableSharedFlow<SearchResultType>()
val searchResults = _searchResults.asSharedFlow()
fun submitSearchInput(query: String) {
searchJob?.cancel()
searchJob = viewModelScope.launch {
delay(1000)
_searchResults.emit(executeSearch(query))
}
}
I saw this but I'm not sure how to implement it or if this is the same issue, I have a mediator live data that updates when either of its 2 source live datas update or when the underlying data (Room db) updates, it seems to work fine but if the data updates a lot it refreshes a lot in quick succession and I get an error
Cannot run invalidation tracker. Is the db closed?
Cannot access database on the main thread since it may potentially lock the UI for a long period of time
this doesn't happen everytime, only when there are a lot of updates to the database in very quick succession heres the problem part of the view model,
var search: MutableLiveData<String> = getSearchState()
val filters: MutableLiveData<MutableSet<String>> = getCurrentFiltersState()
val searchPokemon: LiveData<PagingData<PokemonWithTypesAndSpeciesForList>>
val isFiltersLayoutExpanded: MutableLiveData<Boolean> = getFiltersLayoutExpanded()
init {
val combinedValues =
MediatorLiveData<Pair<String?, MutableSet<String>?>?>().apply {
addSource(search) {
value = Pair(it, filters.value)
}
addSource(filters) {
value = Pair(search.value, it)
}
}
searchPokemon = Transformations.switchMap(combinedValues) { pair ->
val search = pair?.first
val filters = pair?.second
if (search != null && filters != null) {
searchAndFilterPokemonPager(search, filters.toList())
} else null
}.distinctUntilChanged()
}
#SuppressLint("DefaultLocale")
private fun searchAndFilterPokemonPager(search: String, filters: List<String>): LiveData<PagingData<PokemonWithTypesAndSpeciesForList>> {
return Pager(
config = PagingConfig(
pageSize = 20,
enablePlaceholders = false,
maxSize = 60
)
) {
if (filters.isEmpty()){
searchPokemonForPaging(search)
} else {
searchAndFilterPokemonForPaging(search, filters)
}
}.liveData.cachedIn(viewModelScope)
}
#SuppressLint("DefaultLocale")
private fun getAllPokemonForPaging(): PagingSource<Int, PokemonWithTypesAndSpecies> {
return repository.getAllPokemonWithTypesAndSpeciesForPaging()
}
#SuppressLint("DefaultLocale")
private fun searchPokemonForPaging(search: String): PagingSource<Int, PokemonWithTypesAndSpeciesForList> {
return repository.searchPokemonWithTypesAndSpeciesForPaging(search)
}
#SuppressLint("DefaultLocale")
private fun searchAndFilterPokemonForPaging(search: String, filters: List<String>): PagingSource<Int, PokemonWithTypesAndSpeciesForList> {
return repository.searchAndFilterPokemonWithTypesAndSpeciesForPaging(search, filters)
}
the error is actually thrown from the function searchPokemonForPaging
for instance it happens when the app starts which does about 300 writes but if I force the calls off the main thread by making everything suspend and use runBlocking to return the Pager it does work and I don't get the error anymore but it obviously blocks the ui, so is there a way to maybe make the switchmap asynchronous or make the searchAndFilterPokemonPager method return a pager asynchronously? i know the second is technically possible (return from async) but maybe there is a way for coroutines to solve this
thanks for any help
You can simplify combining using combineTuple (which is available as a library that I wrote for this specific purpose) (optional)
Afterwards, you can use the liveData { coroutine builder to move execution to background thread
Now your code will look like
val search: MutableLiveData<String> = getSearchState()
val filters: MutableLiveData<Set<String>> = getCurrentFiltersState()
val searchPokemon: LiveData<PagingData<PokemonWithTypesAndSpeciesForList>>
val isFiltersLayoutExpanded: MutableLiveData<Boolean> = getFiltersLayoutExpanded()
init {
searchPokemon = combineTuple(search, filters).switchMap { (search, filters) ->
liveData {
val search = search ?: return#liveData
val filters = filters ?: return#liveData
withContext(Dispatchers.IO) {
emit(searchAndFilterPokemonPager(search, filters.toList()))
}
}
}.distinctUntilChanged()
}
I am currently trying to get data out of my Room Database without using a ViewModel. This is because I am working on a NotificationHandler which can be triggered at any point by an Alarm Manager.
Below is my code so far. This code below starts with a call to sortNotification from another class. sortNotification then calls launchAsyncTask which inturn goes off to the database by calling getQuotesFromDatabase. I then wait for the results (I believe), assign the data from the database to listOfQuotes variable, then call displayNotification to use it. My issue is, listOfQuotes is always null when I am trying to use it displayNotification.
Now I know the database has content as when I open my application and go to an Activity which has a ViewModel, the data is retrieved successfully. I think my issue is likely to be with the async task not completing properly or with my coroutineScope. I just need listOfQuotes to have data when the code gets into displayNotification. Any help would be greatly appreciated. Thanks in advance.
private var job = Job()
private val ioScope = CoroutineScope(Dispatchers.IO + job)
private lateinit var listOfQuotes: LiveData<List<DefaultQuote>>
fun sortNotification() {
launchAsyncTask()
}
private fun launchAsyncTask() = CoroutineScope(Dispatchers.IO).launch {
val asyncTask = ioScope.async {
getQuotesFromDatabase()
}
listOfQuotes = asyncTask.await()
displayNotification()
}
private suspend fun getQuotesFromDatabase(): LiveData<List<DefaultQuote>> {
return withContext(Dispatchers.IO) {
val defaultQuoteDao = QuoteDatabase.getDatabase(context, this).defaultQuoteDao()
val defaultQuoteRepository = DefaultQuoteRepository(defaultQuoteDao)
defaultQuoteRepository.allQuotes
}
}
private fun displayNotification() {
val quote = listOfQuotes.value?.let {
val size = it.size
val randomIndex = (0..size).random()
it[randomIndex]
} ?: throw NullPointerException("Quotes not found")
// ... then do notification stuff
I have also added in the code from my DAO:
#Dao
interface DefaultQuoteDao {
#Query("SELECT * FROM $DEFAULT_TABLE_NAME")
fun getAllQuotes(): LiveData<List<DefaultQuote>>
}
And the code from my repository:
class DefaultQuoteRepository(private val defaultQuoteDao: DefaultQuoteDao) {
val allQuotes: LiveData<List<DefaultQuote>> = defaultQuoteDao.getAllQuotes()
}
And the code for QuoteDatabase.getDatabase(Context, CoroutineScope):
fun getDatabase(context: Context, scope: CoroutineScope): QuoteDatabase {
val tempInstance = INSTANCE
if (tempInstance != null) {
return tempInstance
}
synchronized(this) {
val instance = Room.databaseBuilder(
context.applicationContext,
QuoteDatabase::class.java,
DATABASE_NAME
)
.fallbackToDestructiveMigration()
.addCallback(QuoteDatabaseCallback(scope))
.build()
INSTANCE = instance
return instance
}
}
The specific problem is you are never observing the value of the listOfQuotes LiveData. This is required to initiate the fetch from the database.
Overall you're doing this in a strange way. You should use either coroutines or LiveData. Both of them allow you to observe data in the database, but you don't need both. That would be like wrapping an async call inside and async call then having to unwrap them both. You should either:
Remove coroutines and synchronously return the LiveData and observe it.
Use Flow to return Flow<List<DefaultQuote>> from your dao function getAllQuotes
I recommend 2. if you expect your application to become medium large or complex. Flow allows you to map or combine data in a more succinct and flexible manner.
Then, your function sortNotification would become:
// Ideally this should be injected into the class, but for a Service that's a little difficult.
// At a minimum you should initialize it once for the class
val defaultQuoteRepository: DefaultQuoteRepository by lazy {
DefaultQuoteRepository(QuoteDatabase.getDatabase(context, this).defaultQuoteDao())
}
fun sortNotification() {
defaultQuoteRepository.allQuotes
.map { listOfQuotes ->
listOfQuotes.random()
}
.flowOn(Dispatchers.IO)
.onEach { randomQuote ->
displayNotification(randomQuote)
}
// This makes the above onEach lambda run on the main thread so you're safe to show notifications.
// Ideally you should have a specific scope defined but tbh if you're not testing it's not that important
.launchIn(GlobalScope)
}