I've got a flow from my repository that looks something like this:
val userListFlow: Flow<List<User>> = channelFlow<List<User>> {
source.setOnUserUpdatedListener { userList ->
trySend(userList)
}
awaitClose {
logger.info("waitClose")
source.setOnUserUpdatedListener(null)
}
}.stateIn(
scope = externalScope,
started = SharingStarted.WhileSubscribed(5000),
initialValue = emptyList()
)
suspend fun getUserThumbnail(user: User): File {
return getUserThumbFromCache(user) ?: run {
fetchUserThumbnailRemote()
}
}
private suspend fetchUserThumbnailRemote(user: User): Bitmap {
thumbnailService.getUserThumbnailBitmap(user.id)
}
fun getUserThumbFromCache(user: User) {
val thumbFile = getThumbFile(user)
return if (thumbFile.exists() && thumbFile.size() > 0) {
thumbFile
} else null
}
private fun getThumbFile(user: User): File {
return File(cacheDir, "${user.id}.jpg")
}
}
For each of these users I can call the suspend function to get a thumbnail for the user.
I don't want to wait for the thumbnail before showing the list of users though, I'd rather it show the users and then when the thumbnail is fetched, update the list.
However I'd like the list to be updated when a thumbnail is fetched..
From my ViewModel I have something like
data class UserWithThumb(user: User, thumb: File?)
val userLiveData = repo.userListFlow.map {
UserWithThumb(it, repo.getUserThumbFromCache(it))
}.asLiveData()
So then from my Fragment I do
viewModel.userLiveData.observe(viewLifecycleOwner) {
userListAdapter.submitList(it)
}
My thumbnails are all null though as I need to fetch them from remote. However if I call that function then that will delay my list from getting to the UI until the thumbnail is fetched. How can I get the thumbnail to the UI in a clean way? I realize that I need to have my livedata or flow update itself once the thumbnail is fetched but I have no idea how to hook that into my code. Any ideas would be appreciated.
I suppose one way to think about this is I'd like my upstream (repository) flow to contain the list of users but then I'd like to update the list given to the view not just when the upstream (repo) flow gets new data but when new thumbnails are downloaded as well..
What I understood from the question is, you have a list of UserWithThumb that is created once you set Users list and you want to show it to the UI immediately. In the background you want to fetch User thumbnails and once you receive them, you want to update the list again.
One way to achieve what you want is:
val userLiveData = flow {
repo.userListFlow.collect { users ->
val initialList = users.map { UserWithThumb(it, repo. getUserThumbFromCache(it)) }
emit(initialList)
coroutineScope {
val finalList = users.map {
async(Dispatchers.IO) { // fetch all thumbnails in parallel
UserWithThumb(it, repo. getUserThumbnail(it))
}
}.awaitAll() // wait until all thumbnails have been fetched
emit(finalList)
}
}
}.asLiveData()
Related
I'm trying to show a user information in DetailActivity. So, I request a data and get a data for the user from server. but in this case, the return type is Flow<User>. Let me show you the following code.
ServiceApi.kt
#GET("endpoint")
suspend fun getUser(#Query("id") id: Int): Response<User>
Repository.kt
fun getUser(id: Int): Flow<User> = flow<User> {
val userResponse = api.getUser(id = id)
if (userResponse.isSuccessful) {
val user = userResponse.body()
emit(user)
}
}
.flowOn(Dispatchers.IO)
.catch { // send error }
DetailViewModel.kt
class DetailViewModel(
private val repository : Repository
) {
val uiState: StateFlow<User> = repository.getUser(id = 369).stateIn(
scope = viewModelScope,
started = SharingStarted.WhileSubscribed(5000),
initialValue = User() // empty user
)
}
DetailActivity.kt
class DetailActivity: AppCompatActivity() {
....
initObersevers() {
lifecycleScope.launch {
// i used the `flowWithLifecycle` because the data is just a single object.
viewModel.uiState.flowWithLifecycle(lifecycle, Lifecycle.State.STARTED).collect { state ->
// show data
}
}
}
...
}
But, all of sudden, I just realized that this process is just an one-shot operation and thought i can use suspend function and return User in Repository.kt.
So, i changed the Repository.kt.
Repository.kt(changed)
suspend fun getUser(id: Int): User {
val userResponse = api.getUser(id = id)
return if(userResponse.isSuccessful) {
response.body()
} else {
User() // empty user
}
}
And in DetailViewModel, i want to convert the User into StateFlow<User> because of observing from DetailActivity and I'm going to use it the same way as before by using flowWithLifecycle.
the concept is... i thought it's just one single data and i dind't need to use Flow in Repository. because it's not several items like List.
is this way correct or not??
Yeap, this one-time flow doesn't make any sense - it emits only once and that's it.
You've got two different ways. First - is to create a state flow in your repo and emit there any values each time you're doing your GET request. This flow will be exposed to the use case and VM levels. I would say that it leads to more difficult error handling (I'm not fond of this way, but these things are always arguable, haha), but it also has some pros like caching, you can always show/get the previous results.
Second way is to leave your request as a simple suspend function which sends a request, returns the result of it back to your VM (skipping error handling here to be simple):
val userFlow: Flow<User>
get() = _userFlow
private val _userFlow = MutableStateFlow(User())
fun getUser() = launch(viewModelScope) {
_userFlow.value = repository.getUser()
}
This kind of implementation doesn't provide any cache out of scope of this VM's lifecycle, but it's easy to test and use.
So it's not like there is only one "the-coolest-way-to-do-it", it's rather a question what suits you more for your needs.
I'm investigating the use of Kotlin Flow within my current Android application
My application retrieves its data from a remote server via Retrofit API calls.
Some of these API's return 50,000 data items in 500 item pages.
Each API response contains an HTTP Link header containing the Next pages complete URL.
These calls can take up to 2 seconds to complete.
In an attempt to reduce the elapsed time I have employed a Kotlin Flow to concurrently process each page
of data while also making the next page API call.
My flow is defined as follows:
private val persistenceThreadPool = Executors.newFixedThreadPool(3).asCoroutineDispatcher()
private val internalWorkWorkState = MutableStateFlow<Response<List<MyPage>>?>(null)
private val workWorkState = internalWorkWorkState.asStateFlow()
private val myJob: Job
init {
myJob = GlobalScope.launch(persistenceThreadPool) {
workWorkState.collect { page ->
if (page == null) {
} else managePage(page!!)
}
}
}
My Recursive function is defined as follows that fetches all pages:-
private suspend fun managePages(accessToken: String, response: Response<List<MyPage>>) {
when {
result != null -> return
response.isSuccessful -> internalWorkWorkState.emit(response)
else -> {
manageError(response.errorBody())
result = Result.failure()
return
}
}
response.headers().filter { it.first == HTTP_HEADER_LINK && it.second.contains(REL_NEXT) }.forEach {
val parts = it.second.split(OPEN_ANGLE, CLOSE_ANGLE)
if (parts.size >= 2) {
managePages(accessToken, service.myApiCall(accessToken, parts[1]))
}
}
}
private suspend fun managePage(response: Response<List<MyPage>>) {
val pages = response.body()
pages?.let {
persistResponse(it)
}
}
private suspend fun persistResponse(myPage: List<MyPage>) {
val myPageDOs = ArrayList<MyPageDO>()
myPage.forEach { page ->
myPageDOs.add(page.mapDO())
}
database.myPageDAO().insertAsync(myPageDOs)
}
My numerous issues are
This code does not insert all data items that I retrieve
How do complete the flow when all data items have been retrieved
How do I complete the GlobalScope job once all the data items have been retrieved and persisted
UPDATE
By making the following changes I have managed to insert all the data
private val persistenceThreadPool = Executors.newFixedThreadPool(3).asCoroutineDispatcher()
private val completed = CompletableDeferred<Int>()
private val channel = Channel<Response<List<MyPage>>?>(UNLIMITED)
private val channelFlow = channel.consumeAsFlow().flowOn(persistenceThreadPool)
private val frank: Job
init {
frank = GlobalScope.launch(persistenceThreadPool) {
channelFlow.collect { page ->
if (page == null) {
completed.complete(totalItems)
} else managePage(page!!)
}
}
}
...
...
...
channel.send(null)
completed.await()
return result ?: Result.success(outputData)
I do not like having to rely on a CompletableDeferred, is there a better approach than this to know when the Flow has completed everything?
You are looking for the flow builder and Flow.buffer():
suspend fun getData(): Flow<Data> = flow {
var pageData: List<Data>
var pageUrl: String? = "bla"
while (pageUrl != null) {
TODO("fetch pageData from pageUrl and change pageUrl to the next page")
emitAll(pageData)
}
}
.flowOn(Dispatchers.IO /* no need for a thread pool executor, IO does it automatically */)
.buffer(3)
You can use it just like a normal Flow, iterate, etc. If you want to know the total length of the output, you should calculate it on the consumer with a mutable closure variable. Note you shouldn't need to use GlobalScope anywhere (ideally ever).
There are a few ways to achieve the desired behaviour. I would suggest to use coroutineScope which is designed specifically for parallel decomposition. It also provides good cancellation and error handling behaviour out of the box. In conjunction with Channel.close behaviour it makes the implementation pretty simple. Conceptually the implementation may look like this:
suspend fun fetchAllPages() {
coroutineScope {
val channel = Channel<MyPage>(Channel.UNLIMITED)
launch(Dispatchers.IO){ loadData(channel) }
launch(Dispatchers.IO){ processData(channel) }
}
}
suspend fun loadData(sendChannel: SendChannel<MyPage>){
while(hasMoreData()){
sendChannel.send(loadPage())
}
sendChannel.close()
}
suspend fun processData(channel: ReceiveChannel<MyPage>){
for(page in channel){
// process page
}
}
It works in the following way:
coroutineScope suspends until all children are finished. So you don't need CompletableDeferred anymore.
loadData() loads pages in cycle and posts them into the channel. It closes the channel as soon as all pages have been loaded.
processData fetches items from the channel one by one and process them. The cycle will finish as soon as all the items have been processed (and the channel has been closed).
In this implementation the producer coroutine works independently, with no back-pressure, so it can take a lot of memory if the processing is slow. Limit the buffer capacity to have the producer coroutine suspend when the buffer is full.
It might be also a good idea to use channels fan-out behaviour to launch multiple processors to speed up the computation.
In my app I have database which uses Room and a network service using retrofit. I have a requirement where if there is no data in local database I need to query the network and show a progress bar. If the network returns empty data then I need to show a empty view. One of the problem is that I need to ignore the empty data from the room and only consider empty data from the server so that when the user doesn't have any data he just sees a loading view and after the server returns empty data he will see empty view.
I have implemented this using a publish subject. Lce(loading content error) is wrapper object around data.
val recentPublish = PublishSubject.create<Lce<List<RecentMessage>>>()
fun loadRecentMessages() {
loadMessageFromDB()
loadRecentMessageFromServer()
}
private fun loadMessageFromDB() {
disposable = recentMessageDao.getRecentMessages() // this is a flowable
.subscribeOn(Schedulers.io())
.subscribe({
Timber.d("recent message from db size ${it.size}")
handleMessageFromDB(it)
}, {
it.printStackTrace()
Timber.e("error on flowable from db!")
})
}
protected fun handleMessageFromDB(messages: List<RecentMessage>) {
// only publish if the data is not empty
if (messages.isNotEmpty()) {
recentPublish.onNext(Lce.Content(messages))
}
}
private fun loadRecentMessageFromServer() {
recentPublish.onNext(Lce.Loading())
networkService.getLatestMessage() // this is a single
.subscribe({
val parsedMessages =
DtoConverter.convertRecentPrivateMessageResponse(it, user.id!!)
handleMessageFromServer(parsedMessages)
}, {
it.printStackTrace()
recentPublish.onNext(Lce.Error(it))
Timber.w("failed to load recent message for private chat from server")
})
}
private fun handleMessageFromServer(recentMessages: List<RecentMessage>) {
Timber.i("recent messages from server ${recentMessages.size}")
if (recentMessages.isEmpty()) {
recentPublish.onNext(Lce.Content(arrayListOf()))
} else {
recentMessageDao.saveAll(recentMessages)
}
}
In the above code I am only passing the empty data from server and ignoring the empty data from room. This solution works but I wonder if there is some better functional approach to solve this problem. I am a beginner to Rxjava and any help will be appreciated. Thank you.
After some research and the comment from #EpicPandaForce, I came up with this approach. I learned quite few things and it just clicked on me, about how to correctly use rxjava. Here is my approach, any comment will be appreciated.
fun getMessages(): Observable<Lce<List<RecentMessage>>> {
return Observable.mergeDelayError(getMessagesFromDB(), getMessagesFromNetwork()) // even if network fails, we still want to observe the DB
}
private fun getMessagesFromDB(): Observable<Lce.Content<List<RecentMessage>>> {
return recentMessageDao.getRecentMessages()
.filter {
it.isNotEmpty() // only forward the data from db if it's not empty
}.map {
Lce.Content(it)
}
}
private fun getMessagesFromNetwork(): Observable<Lce<List<RecentMessage>>> {
// first show a loading , then request for data
return Observable.concat(Observable.just(Lce.Loading()), profileService.getLatestMessage()
.flatMap {
processServerResponse(it) // store the data to db
}.onErrorReturn {
Lce.Error(it)
}.filter {
(it as Lce.Content).packet.isEmpty() // only forward data if it's empty
})
}
private fun processServerResponse(response: RecentMessageResponse): Observable<Lce<List<RecentMessage>>> {
return Observable.create {
val parsedMessages =
DtoConverter.convertRecentPrivateMessageResponse(response, user.id!!)
handleMessageFromServer(parsedMessages)
it.onComplete() // we use single source of truth so don't return anyting
}
}
So I'm pretty sure that I'm kind of at a loss here.
The expected behavior is:
Get data from API -> save it in the local DB -> load data from the
local DB and display it
First of all in my Fragment I call a function that does this:
mLiveData = viewModel.fetchAllCategories(getString(R.string.lang_code))
mCategoryLiveData!!.observe(this, Observer<Array<Category>> { it ->
if (it != null) {
this#CategoryListFragment.allCategories = SparseArray(it.size)
for (category in it) {
allCategories[category.id] = category
}
displayedCategory = allCategories[1]
this#CategoryListFragment.mLoadingCircle.visibility = View.GONE
this#CategoryListFragment.displayCategoryChildren()
}
})
fetchAllCategories calls a function in the ViewModel which calls this function:
fun getAllCategoriesFromAPI(language: String): Flowable<Array<Category>> {
return service.getAllCategories(language)
.doOnNext {
Log.e("Repository", "Fetched ${it.size} Categories from the API ")
storeCategoriesInDb(it)
}
}
However the function displayCategoryChildren() fires before the onNext finishes which results in an error since the data the app is supposed to get from the db is not saved there yet.
If it is in any way relevant I can also post the fuction in the ViewModel
I need to implement a search on a large data set that can take some time to complete on mobile devices. So I want to display each matching result as soon as it becomes available.
I need to fetch all available data from a data store that decides whether to get them from network or from the device. This call is an Observable. As soon as the data from that Observable becomes available I want to loop over it, apply a search predicate and notify any Observers for any match found.
So far my idea was to use a PublishSubject to subscribe to and call its onNext function every time the search finds a new match. However I can't seem to get the desired behavior to work.
I'm using MVVM + Android Databinding and want to display every matched entry in a RecyclerView so for every onNext event that is received by the observing viewModel I have to call notifyItemRangeInserted on the RecyclerView's adapter.
class MySearch(val dataStore: MyDataStore) {
private val searchSubject = PublishSubject.create<List<MyDto>>()
fun findEntries(query: String): Observable<List<MyDto>> {
return searchSubject.doOnSubscribe {
// dataStore.fetchAll returns an Observable<List<MyDto>>
dataStore.fetchAll.doOnNext {
myDtos -> if (query.isNotBlank()) {
search(query, myDtos)
} else {
searchSubject.onNext(myDtos)
}
}.subscribe(searchSubject)
}
}
private fun(query: String, data: List<MyDto>) {
data.forEach {
if (it.matches(query)) {
// in real life I cache a few results and don't send each single item
searchSubject.onNext(listOf(it))
}
}
}
fun MyDto.matches(query: String): Boolean // stub
}
-
class MyViewModel(val mySearch: MySearch, val viewNotifications: Observer<Pair<Int, Int>>): BaseObservable() {
var displayItems: List<MyItemViewModel> = listOf()
fun loadData(query: String): Subscription {
return mySearch.findEntries(query)
.observeOn(AndroidSchedulers.mainThread())
.doOnNext(this::onSearchResult)
.doOnCompleted(viewNotifications::onCompleted)
.doOnError(viewNotifications::onError)
.subscribe()
}
private fun onSearchResult(List<MyDto> data) {
val lastIndex = displayItems.lastIndex
displayItems = data.map { createItem(it) }
notifyChange()
viewNotifications.onNext(Pair(lastIndex, data.count()))
}
private fun createItem(dto: MyDto): MyItemViewModel // stub
}
The problem I have with the above code is that with an empty query MyViewModel::onSearchResult is called 3 times in a row and when the query is not empty MyViewModel::onSearchResult isn't called at all.
I suspect the problem lies somewhere in the way I have nested the Observables in findEntries or that I'm subscribing wrong / getting data from a wrong thread.
Does anyone have an idea about this?