Develop Reference

Combine data from Kotlin Flow/LiveData

I've got a flow from my repository that looks something like this:
val userListFlow: Flow<List<User>> = channelFlow<List<User>> {
source.setOnUserUpdatedListener { userList ->
trySend(userList)
}
awaitClose {
logger.info("waitClose")
source.setOnUserUpdatedListener(null)
}
}.stateIn(
scope = externalScope,
started = SharingStarted.WhileSubscribed(5000),
initialValue = emptyList()
)
suspend fun getUserThumbnail(user: User): File {
return getUserThumbFromCache(user) ?: run {
fetchUserThumbnailRemote()
}
}
private suspend fetchUserThumbnailRemote(user: User): Bitmap {
thumbnailService.getUserThumbnailBitmap(user.id)
}
fun getUserThumbFromCache(user: User) {
val thumbFile = getThumbFile(user)
return if (thumbFile.exists() && thumbFile.size() > 0) {
thumbFile
} else null
}
private fun getThumbFile(user: User): File {
return File(cacheDir, "${user.id}.jpg")
}
}
For each of these users I can call the suspend function to get a thumbnail for the user.
I don't want to wait for the thumbnail before showing the list of users though, I'd rather it show the users and then when the thumbnail is fetched, update the list.
However I'd like the list to be updated when a thumbnail is fetched..
From my ViewModel I have something like
data class UserWithThumb(user: User, thumb: File?)
val userLiveData = repo.userListFlow.map {
UserWithThumb(it, repo.getUserThumbFromCache(it))
}.asLiveData()
So then from my Fragment I do
viewModel.userLiveData.observe(viewLifecycleOwner) {
userListAdapter.submitList(it)
}
My thumbnails are all null though as I need to fetch them from remote. However if I call that function then that will delay my list from getting to the UI until the thumbnail is fetched. How can I get the thumbnail to the UI in a clean way? I realize that I need to have my livedata or flow update itself once the thumbnail is fetched but I have no idea how to hook that into my code. Any ideas would be appreciated.
I suppose one way to think about this is I'd like my upstream (repository) flow to contain the list of users but then I'd like to update the list given to the view not just when the upstream (repo) flow gets new data but when new thumbnails are downloaded as well..

What I understood from the question is, you have a list of UserWithThumb that is created once you set Users list and you want to show it to the UI immediately. In the background you want to fetch User thumbnails and once you receive them, you want to update the list again.
One way to achieve what you want is:
val userLiveData = flow {
repo.userListFlow.collect { users ->
val initialList = users.map { UserWithThumb(it, repo. getUserThumbFromCache(it)) }
emit(initialList)
coroutineScope {
val finalList = users.map {
async(Dispatchers.IO) { // fetch all thumbnails in parallel
UserWithThumb(it, repo. getUserThumbnail(it))
}
}.awaitAll() // wait until all thumbnails have been fetched
emit(finalList)
}
}
}.asLiveData()

Picasso Fetching Images in Activity, then show them into Fragment's ImageView

I am trying to fetch multiple images in an Activity with FragmentStateAdapter in it.
Then I need to show them into some Fragment's ImageView.
I want to preload them into the cache ( after I receive an API response with the info about images like imageID's)
Do I need to use do something else after .fetch() in Activity and do I need to create again same URL request and then load it into the right image view?
Currently, I am seeing images normally but I guess that they are not preloaded in Activity and I fetch them in the Fragment. I am not sure, how to check it?
Thank you for your help in advance!
class FavouriteActivity: - here I want to preload them:
#Subscribe
fun onCoolingImageInfoEvent(coolingEvent: FreezerImageEvent) {
viewModel.retrieveCoolingImage(coolingEvent.data)
val applianceID = viewModel.haID
viewModel.shownCoolingImages.value?.forEach {
picasso.load("https:/.../api/$applianceID/images/${it.imagekey}")
?.fetch() // does I need to do something else?
}
This is my adapter in which I have some fragments (for each image different one)
class FavouriteAdapter(
activity: BaseWearActivity,
private val viewModel: FavouriteViewModel
) : FragmentStateAdapter(activity) {
enum class FavouriteFragmentsEnum(
val position: Int,
val fragment: Fragment
) {
FAVOURITES(0, FavouritesFragment()),
COOLING_IMAGE(1, CoolingImageFragment(imageIndex = 0)),
COOLING_IMAGE_2(2, CoolingImageFragment(imageIndex = 1)),
//(...)
and Fragment code with images where I need to load already fetched images with Picaso
private fun initPicassoImage(coolingImageData: List<CoolingImageData>) {
applianceID = viewModel.haID
imageID = viewModel.getImageIDByIndex(imageIndex)
picasso.load("https:/.../api/$applianceID/images/$imageID")
?.into(current_image, object : Callback {
// (...)
} // does I need to do something else?
)
}

Your code seemed to me ok.
However, you are fetching images from your server. Picasso may invalidate the cache for that URL whenever your server change http headers such as etag, document size, etc in future for that URL.
You may use picasso.setIndicatorsEnabled(true) to see if an image loaded from cache. This adds a little indicator at top left of your image.
Color
Picasso loaded from
Red
Network
Green
Memory
Blue
Disk

I did it like that:
In Activity:
#Subscribe
fun onCoolingImageInfoEvent(coolingEvent: FreezerImageEvent?) {
(...)
viewModel.shownCoolingImages.forEach {
picasso
.load("https://(...)/$applianceID/images/${
viewModel.getImageIDByIndex(index)
}")
.priority(Picasso.Priority.HIGH)
?.fetch()
}
(...)
}
And in fragment:
private fun initPicassoImage() {
(...)
val imageUrl = "https://(...)/$applianceID/images/$imageID"
picasso
.load(imageUrl)
.networkPolicy(NetworkPolicy.OFFLINE)
.into(current_image, object : Callback {
override fun onSuccess() {
Log.d("PICASSO", "success load image from memory")
}
override fun onError(e: Exception?) {
//Try again online if cache failed
picasso
.load(imageUrl)
.into(current_image, object : Callback {
override fun onSuccess() {
Log.d("PICASSO", "load image from network")
}
override fun onError(e: Exception?) {
Log.e("Picasso", "Could not fetch image")
}
});
}
})
}
Take a Note: as #ocos said, you can check if it loads Image from Memory/Network.
However, you are fetching images from your server. Picasso may
invalidate the cache for that URL whenever your server change http headers such as etag, document size, etc in future for that URL.
You may use picasso.setIndicatorsEnabled(true) to see if an image
loaded from cache. This adds a little indicator at top left of your
image.
Color
Picasso loaded from
Red
Network
Green
Memory
Blue
Disk

Can we download multiple files in a Single Coroutine in Android Kotlin

Hi all Developers pls sort out my query as much as possible.
I'm working on Image Album application. Here while user click on particular album we should download all related images to that particular album.
For example album name is Tree so that album have multiple image url's array. So i should download all images from that album by array of url's
Ex : imageArray = arrayOf("url1","url2","url3","url4",....etc url(n))
i should put them in for loop or else recursion then i should download them upto (n) urls on completion of one by one.
i have written snippet for one file download here my doubt is how i can proceed to download multiple files.
should i use same coroutine for all files download or else one one coroutine for one one file
CoroutineScope(Dispatchers.IO).launch {
**//here itself i can run for loop or else any other robust/proper way to do this requirement.**
ktor.downloadFile(outputStream, url).collect {
withContext(Dispatchers.Main) {
when (it) {
is DownloadResult.Success -> {
**//on success of one file download should i call recursively to download one more file by this method - private fun downloadFile(context: Context, url: String, file: Uri)**
viewFile(file)
}
below is the code to download a single file
private fun downloadFile(context: Context, url: String, file: Uri) {
val ktor = HttpClient(Android)
contentResolver.openOutputStream(file)?.let { outputStream ->
CoroutineScope(Dispatchers.IO).launch {
ktor.downloadFile(outputStream, url).collect {
withContext(Dispatchers.Main) {
when (it) {
is DownloadResult.Success -> {
viewFile(file)
}
is DownloadResult.Error -> {
}
is DownloadResult.Progress -> {
txtProgress.text = "${it.progress}"
}
}
}
}
}
}
}
suspend fun HttpClient.downloadFile(file: OutputStream, url: String): Flow<DownloadResult> {
return flow {
try {
val response = call {
url(url)
method = HttpMethod.Get
}.response
val data = ByteArray(response.contentLength()!!.toInt())
var offset = 0
do {
val currentRead = response.content.readAvailable(data, offset, data.size)
offset += currentRead
val progress = (offset * 100f / data.size).roundToInt()
emit(DownloadResult.Progress(progress))
} while (currentRead > 0)
response.close()
if (response.status.isSuccess()) {
withContext(Dispatchers.IO) {
file.write(data)
}
emit(DownloadResult.Success)
} else {
emit(DownloadResult.Error("File not downloaded"))
}
} catch (e: TimeoutCancellationException) {
emit(DownloadResult.Error("Connection timed out", e))
} catch (t: Throwable) {
emit(DownloadResult.Error("Failed to connect"))
}
}
}
sealed class DownloadResult {
object Success : DownloadResult()
data class Error(val message: String, val cause: Exception? = null) : DownloadResult()
data class Progress(val progress: Int): DownloadResult()
}
Gradle Files i have used
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.3'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.3.3'
implementation "io.ktor:ktor-client-android:1.2.5"

Should be possible.
Create another function that will iterate through the list of files to be downloaded.
With the new function, use an async or launch coroutine function - this allows for more control over the flow of your logic, sequential behavior or asynchronous behavior respectively.
e.g. fun downloadBatch(list: List<Uri>) { GlobalScope.async(Dispatchers.IO) { //logic goes here }}
Note: GlobalScope is just for an easy example - not advisable to use in live production.
Inside the iterator/for-loop, call a function to download an individual file. This particular function should be appended with suspend at the beginning
e.g. suspend fun downloadFile(uri: Uri)
Note: the suspended function won't use any threading logic itself and depends on being nested within a functional Coroutine.
Continue to use rxJava, or try LiveData, to broadcast your files.

How to complete a Kotlin Flow in Android Worker

I'm investigating the use of Kotlin Flow within my current Android application
My application retrieves its data from a remote server via Retrofit API calls.
Some of these API's return 50,000 data items in 500 item pages.
Each API response contains an HTTP Link header containing the Next pages complete URL.
These calls can take up to 2 seconds to complete.
In an attempt to reduce the elapsed time I have employed a Kotlin Flow to concurrently process each page
of data while also making the next page API call.
My flow is defined as follows:
private val persistenceThreadPool = Executors.newFixedThreadPool(3).asCoroutineDispatcher()
private val internalWorkWorkState = MutableStateFlow<Response<List<MyPage>>?>(null)
private val workWorkState = internalWorkWorkState.asStateFlow()
private val myJob: Job
init {
myJob = GlobalScope.launch(persistenceThreadPool) {
workWorkState.collect { page ->
if (page == null) {
} else managePage(page!!)
}
}
}
My Recursive function is defined as follows that fetches all pages:-
private suspend fun managePages(accessToken: String, response: Response<List<MyPage>>) {
when {
result != null -> return
response.isSuccessful -> internalWorkWorkState.emit(response)
else -> {
manageError(response.errorBody())
result = Result.failure()
return
}
}
response.headers().filter { it.first == HTTP_HEADER_LINK && it.second.contains(REL_NEXT) }.forEach {
val parts = it.second.split(OPEN_ANGLE, CLOSE_ANGLE)
if (parts.size >= 2) {
managePages(accessToken, service.myApiCall(accessToken, parts[1]))
}
}
}
private suspend fun managePage(response: Response<List<MyPage>>) {
val pages = response.body()
pages?.let {
persistResponse(it)
}
}
private suspend fun persistResponse(myPage: List<MyPage>) {
val myPageDOs = ArrayList<MyPageDO>()
myPage.forEach { page ->
myPageDOs.add(page.mapDO())
}
database.myPageDAO().insertAsync(myPageDOs)
}
My numerous issues are
This code does not insert all data items that I retrieve
How do complete the flow when all data items have been retrieved
How do I complete the GlobalScope job once all the data items have been retrieved and persisted
UPDATE
By making the following changes I have managed to insert all the data
private val persistenceThreadPool = Executors.newFixedThreadPool(3).asCoroutineDispatcher()
private val completed = CompletableDeferred<Int>()
private val channel = Channel<Response<List<MyPage>>?>(UNLIMITED)
private val channelFlow = channel.consumeAsFlow().flowOn(persistenceThreadPool)
private val frank: Job
init {
frank = GlobalScope.launch(persistenceThreadPool) {
channelFlow.collect { page ->
if (page == null) {
completed.complete(totalItems)
} else managePage(page!!)
}
}
}
...
...
...
channel.send(null)
completed.await()
return result ?: Result.success(outputData)
I do not like having to rely on a CompletableDeferred, is there a better approach than this to know when the Flow has completed everything?

You are looking for the flow builder and Flow.buffer():
suspend fun getData(): Flow<Data> = flow {
var pageData: List<Data>
var pageUrl: String? = "bla"
while (pageUrl != null) {
TODO("fetch pageData from pageUrl and change pageUrl to the next page")
emitAll(pageData)
}
}
.flowOn(Dispatchers.IO /* no need for a thread pool executor, IO does it automatically */)
.buffer(3)
You can use it just like a normal Flow, iterate, etc. If you want to know the total length of the output, you should calculate it on the consumer with a mutable closure variable. Note you shouldn't need to use GlobalScope anywhere (ideally ever).

There are a few ways to achieve the desired behaviour. I would suggest to use coroutineScope which is designed specifically for parallel decomposition. It also provides good cancellation and error handling behaviour out of the box. In conjunction with Channel.close behaviour it makes the implementation pretty simple. Conceptually the implementation may look like this:
suspend fun fetchAllPages() {
coroutineScope {
val channel = Channel<MyPage>(Channel.UNLIMITED)
launch(Dispatchers.IO){ loadData(channel) }
launch(Dispatchers.IO){ processData(channel) }
}
}
suspend fun loadData(sendChannel: SendChannel<MyPage>){
while(hasMoreData()){
sendChannel.send(loadPage())
}
sendChannel.close()
}
suspend fun processData(channel: ReceiveChannel<MyPage>){
for(page in channel){
// process page
}
}
It works in the following way:
coroutineScope suspends until all children are finished. So you don't need CompletableDeferred anymore.
loadData() loads pages in cycle and posts them into the channel. It closes the channel as soon as all pages have been loaded.
processData fetches items from the channel one by one and process them. The cycle will finish as soon as all the items have been processed (and the channel has been closed).
In this implementation the producer coroutine works independently, with no back-pressure, so it can take a lot of memory if the processing is slow. Limit the buffer capacity to have the producer coroutine suspend when the buffer is full.
It might be also a good idea to use channels fan-out behaviour to launch multiple processors to speed up the computation.

Queue multiple async calls with callbacks

I need to fetch some images from the gallery, process them (resize, compress...) and save them to a certain path. However, i need to queue the calls because older devices won't be able to process multiple images at the same time.
I am using Glide, this is the code used for processing one image:
fun processImage(context: Context, sourcePath: String, destinationPath: String, quality: Int, width: Int, height: Int, deleteOriginal: Boolean, callback: ((success: Boolean) -> Unit)) {
val sourceFile = File(sourcePath)
val destinationFile = File(destinationPath)
GlideApp.with(context)
.asBitmap()
.load(sourceFile)
.into(object : SimpleTarget<Bitmap>(width, height) {
override fun onResourceReady(resource: Bitmap, transition: Transition<in Bitmap>?) {
try {
destinationFile.writeBytes(ImageUtilities.imageToByteArray(resource, quality, Bitmap.CompressFormat.JPEG, false))
if (deleteOriginal) {
val originalFile = File(sourcePath)
originalFile.delete()
}
callback.invoke(true)
} catch (ex: Exception) {
callback.invoke(false)
}
}
})
}
Now i am queuing the calls manually by calling processNextImage which calls itself recursively until all the images are processed:
private fun processImages(sourceImagePaths: List<String>) {
processNextImage(sourceImagePaths, 0)
}
private fun processNextImage(sourceImagePaths: List<String>, index: Int) {
val imagePath = sourceImagePaths[index]
val destination = FileUtilities.generateImagePath()
processImage(this, imagePath, destination, 90, 1000, 1000, false) {
processedImagePaths.add(destination)
if (index + 1 < sourceImagePaths.count())
processImage(sourceImagePaths, index + 1)
else
success()
}
}
However I don't think this is the best way to do it and I tried to look into Kotlin coroutines but all I found were examples when the queued code is already blocking, which doesn't fit my case because Glide already handles the resizing asynchronously and returns the result in a callback onResourceReady
Any ideas for a clean way to do this?

As described in the official documentation, there is a simple pattern to follow if you want to turn a callback-based API into one based on suspendable functions. I'll paraphrase that description here.
Your key tool is the function from the standard library called suspendCoroutine(). Assume that you have someLongComputation function with a callback that receives a Result object:
fun someLongComputation(params: Params, callback: (Result) -> Unit)
You can convert it into a suspending function with the following straightforward code:
suspend fun someLongComputation(params: Params): Result =
suspendCoroutine { cont ->
someLongComputation(params) { cont.resume(it) }
}
Note how the type of the object passed to the original callback became simply the return value of the suspendable function.
With this you can see the magic of coroutines happen right in front of you: even though it looks exactly like a blocking call, it isn't. The coroutine will get suspended behind the scenes and resume when the return value is ready — and how it will resume is totally under your control.

I was able to solve the issue using suspendCoroutine as suggested in Marko's comment, here is my code:
private fun processImages(sourceImagePaths: List<String>) {
async(UI) {
sourceImagePaths.forEach { path ->
processNextImage(path)?.let {
processedImagePaths.add(it)
}
}
if (processedImagePaths.isEmpty()) finishWithFailure() else finishWithSuccess()
}
}
private suspend fun processNextImage(sourceImagePath: String): String? = suspendCoroutine { cont ->
val destination = FileUtilities.generateImagePath()
processImage(this, sourceImagePath, destination, 90, 1000, 1000, false) { success ->
if (success)
cont.resume(destination)
else
cont.resume(null)
}
}
The method processImages iterates over the list of paths, and calls processNextImage for each path. Since processNextImage contains a suspendCoroutine, it will block the thread until cont.resume is called, which guarantees that the next image will not be processed before the current one is done.