Develop Reference

In-memory caching on repository level for Kotlin Flows on Android

Suppose you have a list of users downloaded from a remote data source in your Android application, and for some reason you do not have a local DB. This list of users is then used throughout your entire application in multiple ViewModels to make other network requests, so you would surely like to have it cached for as long as the app lives and re-fetch it only on demand. This necessarily means you want to cache it inside the Data Layer, which is a Repository in my case, to then get it from your ViewModels.
It is easy to do in a state holder like a ViewModel - just make a StateFlow or whatever. But what if we want a Flow of List<User> (that is cached in RAM after every API request) available inside a repository to then collect from it from the UI Layer? What is the most testable, stable and right way of achieving this?
My initial idea led to this:
class UsersRepository #Inject constructor(
private val usersApi: UsersApi,
private val handler: ResponseHandler
) {
private val _usersFlow = MutableStateFlow<Resource<List<UserResponse>>>(Resource.Empty)
val usersFlow = _usersFlow.asStateFlow()
suspend fun fetchUserList() = withContext(Dispatchers.IO) {
_usersFlow.emit(Resource.Loading)
_usersFlow.emit(
handler {
usersApi.getUsers()
}
)
}
}
Where ResponseHandler is:
class ResponseHandler {
suspend operator fun <T> invoke(block: suspend () -> T) = try {
Resource.Success(block())
} catch (e: Exception) {
Log.e(javaClass.name, e.toString())
val errorCode = when (e) {
is HttpException -> e.code()
is SocketTimeoutException -> ErrorCodes.SocketTimeOut.code
is UnknownHostException -> ErrorCodes.UnknownHost.code
else -> Int.MAX_VALUE
}
Resource.Error(getErrorMessage(errorCode))
}
}
But while researching I found random guy on the internet telling that it is wrong:
Currently StateFlow is hot in nature so it’s not recommended to use in repository. For cold and reactive stream, you can use flow, channelFlow or callbackFlow in repository.
Is he right? If he is, how exactly do cold flows help in this situation, and how do we properly manage them?
If it helps, my UI Layer is written solely with Jetpack Compose

In the official "Guide to app architecture" from Google for Android:
About the source of true: ✅ The repository can contain an in-memory-cache.
The source of truth can be a data source—for example, the database—or even an in-memory cache that the repository might contain. Repositories combine different data sources and solve any potential conflicts between the data sources to update the single source of truth regularly or due to a user input event.
About the lifecycle: ✅ You can scope an instance of your repository to the Application class (but take care).
If a class contains in-memory data—for example, a cache—you might want
to reuse the same instance of that class for a specific period of
time. This is also referred to as the lifecycle of the class instance.
If the class's responsibility is crucial for the whole application,
you can scope an instance of that class to the Application class. This
makes it so the instance follows the application's lifecycle.
About the implementation: I recommend you to check the link directly.
class NewsRepository(
private val newsRemoteDataSource: NewsRemoteDataSource
) {
// Mutex to make writes to cached values thread-safe.
private val latestNewsMutex = Mutex()
// Cache of the latest news got from the network.
private var latestNews: List<ArticleHeadline> = emptyList()
suspend fun getLatestNews(refresh: Boolean = false): List<ArticleHeadline> {
if (refresh || latestNews.isEmpty()) {
val networkResult = newsRemoteDataSource.fetchLatestNews()
// Thread-safe write to latestNews
latestNewsMutex.withLock {
this.latestNews = networkResult
}
}
return latestNewsMutex.withLock { this.latestNews }
}
}
You should read the following page, I think it will answer a lot of your questions : https://developer.android.com/topic/architecture/data-layer

To make this work as a cache you will have to use this repository as a singleton. This effectively create a huge memory leak since you have no control over this memory. You cannot free it, you cannot bypass cache if you want (i mean you can, but it requires additional code outside the flow), you don't have any control over eviction. It's very dumb cache which acts like a memory leak. Not worth it.
Cold flow don't "help" in caching per se. They just give you control over each request that comes from the client. There you can check some outside memory cache if the entry is cached. If yes - is it correct or should be evicted? If it is evicted you can just a normal request. And all this is a single flow that gets disposed right after, so no memory leaks. The only part that have to be singleton is the cache. Although you can implement it as disk cache, it will be faster than network anyway

I want to use Kotlin Flow to update my UI when SharedPreferences have changed

Ok so I want to start using Kotlin-Flow like all the cool kids are doing. It seems like what I want to do meets this reactive pattern. So I receive a Firebase message in the background
...
override fun onMessageReceived(remoteMessage: RemoteMessage) {
super.onMessageReceived(remoteMessage)
val msg = gson.fromJson(remoteMessage.data["data"], MyMessage::class.java)
// persist to SharedPreferences
val flow = flow<MyMessage> { emit(msg) }
and I have a dashboard UI that simply would refresh a banner with this message. Not sure how to observe or collect this message from my DashboardViewModel. Examples and tutorials all seem to emit and collect in the same class. Sounds like I need more direction and more experience here but not much luck finding more real world examples.

Have a look at the Kotlin docs for it: https://kotlinlang.org/docs/flow.html#flows
The basic idea is you create a Flow, and it can produce values over time. You run collect() on that in a coroutine, which allows you to asynchronously handle those updates as they come in.
Generally that flow does a bunch of work internally, and just emits values as it produces them. You could use this within a class as a kind of worker task, but a lot of the time you'd expose flows as a data source, for other components to observe. So you'll see, for example, repositories that return a Flow when you try to get a thing - it's basically "ok we don't have that yet, but it'll come through here".
I'm not an expert on them, and I know there are some caveats about the different builders and flow types, and how you emit to them - it's not always as simple as "create a flow, hand back a reference to it, emit data to it when it comes in". There's actually a callbackFlow builder specially designed around interfacing callbacks with the flow pattern, that's probably worth checking out:
https://developer.android.com/kotlin/flow#callback
The example is about Firebase specifically too - it looks like the idea is broadly that the user requests some data, and you return a flow which internally does a Firebase request and provides a callback. When it gets the data, it uses offer (a special version of emit that handles the callback coming through on a different coroutine context) to output data to the observer. But it's the same general idea - all the work the flow does is encapsulated within it. It's like a task that runs on its own, producing values and outputting them.
Hope that helps! I think once you get the general idea, it's easier to follow the examples, and then understand what the more specialised things like StateFlow and SharedFlow are there for. This might be some helpful reading (from the Android devs):
Lessons learnt using Coroutines Flow - some general "how to use it" ideas, section 4 is about callbacks again and the example might be helpful
Migrating from LiveData to Kotlin’s Flow - some basic patterns you might already be using, especially around UI and LiveData
edit- while I was finding those I saw a new Dev Summit video about Flows and it's pretty good! It's a nice overview of how they work and how to implement them in your app (especially for UI stuff where there are some things to consider): https://youtu.be/fSB6_KE95bU

flow<MyMessage> { emit(msg) } could just be flowOf(msg), but it's weird to wrap a single item in a Flow. If you're making a manual request for a single thing, this is more appropriately handled with a suspend function that returns that thing. You can convert the async callback code to a suspend function with suspendCoroutine(), but Firebase already provides suspend functions you can use instead of callbacks. If you were making repeated requests for data that changes over time, a Flow would be appropriate, but you need to do it higher up by converting the async code using callbackFlow.
In this case, it looks like you are using FirebaseMessagingService, which is an Android Service, and it directly acts as a callback using this onMessageReceived function.
What you possibly could do (and I haven't tried this before), is adapt a local BroadcastReceiver into a Flow you can use from elsewhere in your app. The FirebaseMessangingService can rebroadcast local Intents that can be picked up by such a Flow. So, you could have a function like this that creates a Flow out of a local broadcast.
fun localBroadcastFlow(context: Context, action: String) = callbackFlow {
val receiver = object : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
intent.extras?.run(::trySend)
}
}
LocalBroadcastManager.getInstance(context).registerReceiver(receiver, IntentFilter(action))
awaitClose { LocalBroadcastManager.getInstance(context).unregisterReceiver(receiver) }
}
Then in your service, you could expose the flow through a companion object, mapping to your data class type.
class MyMessageService: FirebaseMessagingService() {
companion object {
private const val MESSAGE_ACTION = "mypackage.MyMessageService.MyMessage"
private const val DATA_KEY = "MyMessage key"
private val gson: Gson = TODO()
fun messages(context: Context): Flow<MyMessage> =
localBroadcastFlow(context, MESSAGE_ACTION)
.mapNotNull { bundle ->
val messageData = bundle.getString(DATA_KEY) ?: return#mapNotNull null
gson.fromJson(messageData, MyMessage::class.java)
}
}
override fun onMessageReceived(remoteMessage: RemoteMessage) {
val intent = Intent(MESSAGE_ACTION)
intent.putExtra(DATA_KEY, remoteMessage.data["data"])
LocalBroadcastManager.getInstance(applicationContext).sendBroadcast(intent)
}
}
And then in your Fragment or Activity, you can collect from MyMessageService.messages().
Note that LocalBroadcastManager is recently deprecated because it promotes the practice of exposing data to all layers of your app. I don't really understand why this should be considered always bad. Any broadcast from the system is visible to all layers of your app. Any http address is visible to all layers of your app, etc. They suggest exposing an observable or LiveData as an alternative, but that would still expose the data to all layers of your app.

I created a class that helps me persist my data but also added an observable flow that emits the current message received.
class MessagePersistence(
private val gson: Gson,
context: Context
) {
private val sharedPreferences = context.getSharedPreferences(
"Messaging", MODE_PRIVATE
)
private val _MyMessageFlow = MutableStateFlow<Message?>(null)
var myMessageFlow: StateFlow<Message?> = __MyMessageFlow
data class Message(
val msg: String
)
var message: Message?
get() = sharedPreferences
.getString("MyMessages", null)
?.let { gson.fromJson(it, Message::class.java) }
set(value) = sharedPreferences
.edit()
.putString("MyMessages", value?.let(gson::toJson))
.apply()
_MyMessageFlow.value = message
myMessageFlow = _MyMessageFlow
}
In my viewModel I inject this class through its constructor and define it as
class MyViewModel(
private val messagePersistence: MessagePersistence
) : ViewModel() {
val myMessage = messagePersistence.myMessageFlow
...
}
then in my fragment I can collect it using an observer.
class MyFragment : Fragment() {
...
viewModel.myMessage.observe(viewLifecycleOwner.lifecycleScope) {
'update the UI with new message
}

Dao in Usecase. MVVM or Clean Architecture anti-pattern?

In our "SearchUsecase" we have access to "ShowFtsDao" directly.
Does it violate the Clean Architecture principles? Does it violate the MVVM architecture?
Assuming our intention is to develop a well-built, standard structure, is there anything wrong with this piece of code?
class SearchUsecase #Inject constructor(
private val searchRepository: SearchRepository,
private val showFtsDao: ShowFtsDao,
private val dispatchers: AppCoroutineDispatchers
) : SuspendingWorkInteractor<SearchShows.Params, List<ShowDetailed>>() {
override suspend fun doWork(params: Params): List<ShowDetailed> {
return withContext(dispatchers.io) {
val remoteResults = searchRepository.search(params.query)
if (remoteResults.isNotEmpty()) {
remoteResults
} else {
when {
params.query.isNotBlank() -> showFtsDao.search("*$params.query*")
else -> emptyList()
}
}
}
}
data class Params(val query: String)
}

I believe your use case handles more logic than it needs to.
As a simple explanation I like to think about the components this way:
Sources: RemoteSource (networking), LocalSource (db), optionally MemorySource are an abstraction over your database and networking api and they do the IO thread switching & data mapping (which comes in handy on big projects, where the backend is not exactly mobile driven)
Repository: communicates with the sources, he is responsible for deciding where do you get the data from. I believe in your case if the RemoteSource returns empty data, then you get it from the LocalSource. (you can expose of course different methods like get() or fetch(), where the consumer specifies if it wants the latest data and based on that the repository calls the correct Source.
UseCases: Talk with multiple repositories and combine their data.

Yes, it does.
Because your domain module has access to the data module and actually you've violated the dependency rule.
That rule specifies that something declared in an outer circle must
not be mentioned in the code by an inner circle.
Domain layer must contain interfaces for details (Repositories) which are implemented in the data layer,
and then, they could be injected into the UseCases (DIP).

Map multiple suspend functions to single LiveData

The company I just started working at uses a so called Navigator, which I for now interpreted as a stateless ViewModel. My Navigator receives some usecases, with each contains 1 suspend function. The result of any of those usecases could end up in a single LiveData. The Navigator has no coroutine scope, so I pass the responsibility of scoping suspending to the Fragment using fetchValue().
Most current code in project has LiveData in the data layer, which I tried not to. Because of that, their livedata is linked from view to dao.
My simplified classes:
class MyFeatureNavigator(
getUrl1: getUrl1UseCase,
getUrl1: getUrl1UseCase
) {
val url = MediatorLiveData<String>()
fun goToUrl1() {
url.fetchValue { getUrl1() }
}
fun goToUrl2() {
url.fetchValue { getUrl2() }
}
fun <T> MediatorLiveData<T>.fetchValue(provideValue: suspend () -> T) {
val liveData = liveData { emit(provideValue()) }
addSource(liveData) {
removeSource(liveData)
value = it
}
}
}
class MyFeatureFragment : Fragment {
val viewModel: MyFeatureViewModel by viewModel()
val navigator: MyFeatureNavigator by inject()
fun onViewCreated() {
button.setOnClickListener { navigator.goToUrl1() }
navigator.url.observe(viewLifecycleOwner, Observer { url ->
openUrl(url)
})
}
}
My two questions:
Is fetchValue() a good way to link a suspend function to LiveData? Could it leak? Any other concerns?
My main reason to only use coroutines (and flow) in the data layer, is 'because Google said so'. What's a better reason for this? And: what's the best trade off in being consistent with the project and current good coding practices?

Is fetchValue() a good way to link a suspend function to LiveData?
Could it leak? Any other concerns?
Generally it should work. You probably should remove the previous source of the MediatorLiveData before adding new one, otherwise if you get two calls to fetchValue in a row, the first url can be slower to fetch, so it will come later and win.
I don't see any other correctness concerns, but this code is pretty complicated, creates a couple of intermediate objects and generally difficult to read.
My main reason to only use coroutines (and flow) in the data layer,
is 'because Google said so'. What's a better reason for this?
Google has provided a lot of useful extensions to use coroutines in the UI layer, e.g. take a look at this page. So obviously they encourage people to use it.
Probably you mean the recommendation to use LiveData instead of the Flow in the UI layer. That's not a strict rule and it has one reason: LiveData is a value holder, it keeps its value and provides it immediately to new subscribers without doing any work. That's particularly useful in the UI/ViewModel layer - when a configuration change happens and activity/fragment is recreated, the newly created activity/fragment uses the same view model, subscribes to the same LiveData and receives the value at no cost.
At the same time Flow is 'cold' and if you expose a flow from your view model, each reconfiguration will trigger a new flow collection and the flow will be to execute from scratch.
So e.g. if you fetch data from db or network, LiveData will just provide the last value to new subscriber and Flow will execute the costly db/network operation again.
So as I said there is no strict rule, it depends on the particular use-case. Also I find it very useful to use Flow in view models - it provides a lot of operators and makes the code clean and concise. But than I convert it to a LiveData with help of extensions like asLiveData() and expose this LiveData to the UI. This way I get best from both words - LiveData catches value between reconfigurations and Flow makes the code of view models nice and clean.
Also you can use latest StateFlow and SharedFlow often they also can help to overcome the mentioned Flow issue in the UI layer.
Back to your code, I would implement it like this:
class MyFeatureNavigator(
getUrl1: getUrl1UseCase,
getUrl1: getUrl1UseCase
) {
private val currentUseCase = MutableStateFlow<UseCase?>(null)
val url = currentUseCase.filterNotNull().mapLatest { source -> source.getData()}.asLiveData()
fun goToUrl1() {
currentUseCase.value = getUrl1
}
fun goToUrl2() {
currentUseCase.value = getUrl2
}
}
This way there are no race conditions to care about and code is clean.
And: what's the best trade off in being consistent with the project
and current good coding practices?
That's an arguable question and it should be primarily team decision. In most projects I participated we adopted this rule: when fixing bugs, doing maintenance of existing code, one should follow the same style. When doing big refactoring/implementing new features one should use latest practices adopted by the team.

Android repository-pattern & RxJava: Use Flowable or Single?

Recently, I´ve read about how important it is to have a Single-Source-Of-Truth (SSOT) when designing an app´s backend (repository, not server-side-backend). https://developer.android.com/topic/libraries/architecture/guide.html
By developing a news-feed app (using the awesome https://newsapi.org/) I am trying to learn more about app architecture.
However, I am unsure of how to design the repository interface for my app.
Btw.: I am using MVVM for my presentation layer. The View subscribes to the ViewModel´s LiveData. The ViewModel subscribes to RxJava streams.
So I came up with 2 approaches:
Approach 1:
interface NewsFeedRepository {
fun loadFeed(): Flowable<List<Article>>
fun refreshFeed(): Completable
fun loadMore(): Completable
}
interface SearchArticleRepository {
fun searchArticles(sources: List<NewsSource>? = null, query: String? = null): Flowable<List<Article>>
fun moreArticles(): Completable
}
interface BookmarkRepository {
fun getBookmarkedArticles(): Flowable<List<Article>>
fun bookmarkArticle(id: String): Completable
}
This approach is primarily using Flowables which emit data if the corresponding data in the underlying SSOT (database) changes (e.g old data gets replaced with fresh data from API, more data was loaded from API, ...). However, I am unsure if using a Flowable for SearchArticleRepository#searchArticles(...) makes sense. As it is like some request/response thing, where maybe a Single might me be more intuitive.
Approach 2:
interface NewsFeedRepository {
fun loadFeed(): Single<List<Article>>
fun refreshFeed(): Single<List<Article>>
fun loadMore(): Single<List<Article>>
}
interface SearchArticleRepository {
fun searchArticles(sources: List<NewsSource>? = null, query: String? = null): Single<List<Article>>
fun moreArticles(): Single<List<Article>>​
}
interface BookmarkRepository {
fun getBookmarkedArticles(): Single<List<Article>>
fun bookmarkArticle(id: String): Single<Article> // Returns the article that was modified. Articles are immutable. ​
}
This approach is using Singles instead of Flowables. This seems very intuitive but if the data in the SSOT changes, no changes will be emitted. Instead, a call to the repository has to be made again. Another aspect to take into account is that the ViewModel may have to manage its own state.
Let´s take the FeedViewModel for example (pseudo-code).
class FeedViewModel : ViewModel() {
// Variables, Boilerplate, ...
val newsFeed: LiveData<List<Article>>
private val articles = mutableListOf<Article>()
fun loadNewsFeed() {
// ...
repository.loadFeed()
//...
// On success, clear the feed and append the loaded articles.
.subscribe({articles.clear(); articles.addAll(it)})
// ...
}
fun loadMore() {
// ...
repository.loadMore()
//...
// On success, append the newly loaded articles to the feed.
.subscribe({articles.addAll(it)})
// ...
}
}
So this might not be crucial for a smaller app like mine, but it definitely can get a problem for a larger app (see state management: http://hannesdorfmann.com/android/arch-components-purist).
Finally, I wanted to know which approach to take and why. Are there any best-practices? I know many of you have already done some larger software-projects/apps and it would be really awesome if some of you could share some knowledge with me and others.
Thanks a lot!

I'd rather go for the first approach using Observables instead of Flowables in your case:
interface NewsFeedRepository {
fun loadFeed(): Observable<List<Article>>
fun refreshFeed(): Completable
fun loadMore(): Completable
}
interface SearchArticleRepository {
fun searchArticles(sources: List<NewsSource>? = null, query: String? = null): Observable<List<Article>>
fun moreArticles(): Completable
}
interface BookmarkRepository {
fun getBookmarkedArticles(): Observable<List<Article>>
fun bookmarkArticle(id: String): Completable
}
I don't see any reason you should necessarily use Flowable for this purpose since you'll never have any OOME related issues checking your repository changes. In other words, for your use case IMHO backpressure is not necessary at all.
Check this official guide which gives us an advice of when to a Flowable over an Observable.
On the other hand, and not related to the question itself, I have serious doubts of what's the purpose of loadMore or moreArticles methods since they return a Completable. Without knowing the context, it may seem you could either refactor the method name by a better name or change the return type if they do what they seem to do by the name.

I believe the first approach is better, Your repo will update the data whenever the data is changed and your view model will be notified automatically and that's cool, while in your second approach you have to call the repo again and that's not really reactive programming.
Also, assume that the data can be changed by something rather than load more event from the view, like when new data added to the server, or some other part of the app changes the data, Now in the first approach again you get the data automatically while for the second your not even know about the changed data and you don't know when to call the method again.