For start I must say I am begginer in RxJava.
Data class:
#Entity(tableName = "google_book")
data class GoogleBook (
#PrimaryKey(autoGenerate = true) val id: Int=0,
val items: ArrayList<VolumeInfo>)
data class VolumeInfo(val volumeInfo: BookInfo){
data class BookInfo(val title: String, val publisher: String, val description: String, val imageLinks: ImageLinks?)
data class ImageLinks(val smallThumbnail: String?)
}
Function which helps me save data to database:
fun searchBooks(query: String) {
searchJob?.cancel()
searchJob = viewModelScope.launch {
val text = query.trim()
if (text.isNotEmpty()) {
bookRepository.getBooksFromApi(query)
.map { t ->
t.items.map {
it.volumeInfo.imageLinks?.smallThumbnail?.filter { x -> x != null }
}
t
}
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe { x ->
x?.let { googleBook ->
searchJob?.cancel()
searchJob = viewModelScope.launch {
bookRepository.deleteGoogleBook()
bookRepository.insertGoogleBook(googleBook)
}
} ?: kotlin.run {
Log.d(TAG, "observeTasks: Error")
}
}
}
}
}
As seen I want to filter list within GoogleBook object by image parameter but It doesnt work. I cannot add filtering for data class ImageLinks so I have no Idea how can I make it right
I am asking mostly about this part:
.map { t ->
t.items.map {
it.volumeInfo.imageLinks?.smallThumbnail?.filter { x -> x != null }
}
t
}
Thanks for reading
welcome to RxJava, you gonna love it.
As far as I can tell the issue with your filtering simply relies here:
.map { t ->
t.items.map {
it.volumeInfo.imageLinks?.smallThumbnail?.filter { x -> x != null })
} // this returns you a new list filtered list here, but does not modify the original one
t // but you return the same data object here, it is not modified at all
}
// also consider naming it bookInfo if it is actually a bookInfo
What you should do is make a copy of your object with the filtered elements, something like this:
fun filterGoogleBookBySmallThumbNail(googleBook: GoogleBook): GoogleBook {
val filteredItems = googleBook.items.filter { it.volumeInfo.imageLinks?.smallThumbnail == null }
return googleBook.copy(items = ArrayList(filteredItems)) // now a new googleBook item is created with the filtered elements
}
// snippet to adjust then
bookRepository.getBooksFromApi(query)
.map { googleBook -> filterGoogleBookBySmallThumbNail(googleBook) }
//...
Some additional notes / suggestions I have:
I don't see you actually disposing of the subscription of the Observable.
bookRepository.getBooksFromApi(query) If this line returns an Observable, even if you cancel the job, you will be still observing that Observable. If it returns a Single then you are in luck, because after one element it is disposed.
To properly dispose, in cancellation you would have to do something like this(still i would recommend the other two rather, just wanted to note the not disposing):
searchJob = viewModelScope.launch {
val text = query.trim()
if (text.isNotEmpty()) {
val disposable = bookRepository.getBooksFromApi(query)
//...
.subscribe { x ->
//...
}
try {
awaitCancellation() // this actually suspends the coroutine until it is cancelled
} catch (cancellableException: CancellationException) {
disposable.dispose() // this disposes the observable subscription
// that way the coroutine stays alive as long as it's not cancelled, and at that point it actually cleans up the Rx Subscription
}
Seems wasteful that you start a new coroutine job just to do actions
If you want to go the Rx way, you could make the
bookRepository.deleteGoogleBook() and bookRepository.insertGoogleBook(googleBook) Completable, and setup the observable as:
bookRepository.getBooksFromApi(query)
//..
.flatMap {
bookRepository.deleteGoogleBook().andThen(bookRepository.insertGoogleBook(it)).andThen(Observable.just(it))
}
//..subscribeOn
.subscribe()
Seems weird you are mixing coroutine and RX this way
if you don't want to go full Rx, you may consider converting your Observable into a kotlin coroutine Flow, that would be easier to handle with coroutine cancellations and calling suspend functions.
I hope it's helpful
Related
I am doing multiple network requests in parallel and monitoring the result using a Stateflow.
Each network request is done in a separate flow, and I use combine to push the latest status on my Stateflow. Here's my code:
Repo class:
fun networkRequest1(id: Int): Flow<Resource<List<Area>>> =
flow {
emit(Resource.Loading())
try {
val areas = retrofitInterface.getAreas(id)
emit(Resource.Success(areas))
} catch (throwable: Throwable) {
emit(
Resource.Error()
)
)
}
}
fun networkRequest2(id: Int): Flow<Resource<List<Area>>> = //same code as above for simplicity
fun networkRequest3(id: Int): Flow<Resource<List<Area>>> = //same code as above for simplicity
fun networkRequest4(id: Int): Flow<Resource<List<Area>>> = //same code as above for simplicity
ViewModel class:
val getDataCombinedStateFlow: StateFlow<Resource<HashMap<String, Resource<out List<Any>>>>?> =
getDataTrigger.flatMapLatest {
withContext(it) {
combine(
repo.networkRequest1(id: Int),
repo.networkRequest2(id: Int),
repo.networkRequest3(id: Int),
repo.networkRequest4(id: Int)
) { a,
b,
c,
d
->
hashMapOf(
Pair("1", a),
Pair("2",b),
Pair("3", c),
Pair("4", d),
)
}.flatMapLatest {
val progress = it
var isLoading = false
flow<Resource<HashMap<String, Resource<out List<Any>>>>?> {
emit(Resource.Loading())
progress.forEach { (t, u) ->
if (u is Resource.Error) {
emit(Resource.Error(error = u.error!!))
// I want to cancel here, as I no longer care if 1 request fails
return#flow
}
if (u is Resource.Loading) {
isLoading = true
}
}
if (isLoading) {
emit(Resource.Loading())
return#flow
}
if (!isLoading) {
emit(Resource.Success(progress))
}
}
}
}
}.stateIn(viewModelScope, SharingStarted.Lazily, null)
View class:
viewLifecycleOwner.lifecycleScope.launchWhenCreated() {
viewModel.getDataCombinedStateFlow.collect {
val result = it ?: return#collect
binding.loadingErrorState.apply {
if (result is Resource.Loading) {
//show smth
}
if (result is Resource.Error) {
//show error msg
}
if (result is Resource.Success) {
//done
}
}
}
}
I want to be able to cancel all work after a Resource.Error is emitted, as I no longer want to wait or do any related work for the response of other API calls in case one of them fails.
How can I achieve that?
I tried to cancel the collect, but the flows that build the Stateflow keep working and emmit results. I know that they won't be collected but still, I find this a waste of resources.
I think this whole situation is complicated by the fact that you have source flows just to precede what would otherwise be suspend functions with a Loading state. So then you're having to merge them and filter out various loading states, and your end result flow keeps repeatedly emitting a loading state until all the sources are ready.
If you instead have basic suspend functions for your network operations, for example:
suspend fun networkRequest1(id: Int): List<Area> =
retrofitInterface.getAreas(id)
Then your view model flow becomes simpler. It doesn't make sense to use a specific context just to call a flow builder function, so I left that part out. (I'm also confused as to why you have a flow of CoroutineContexts.)
I also think it's much cleaner if you break out the request call into a separate function.
private fun makeParallelRequests(id: Int): Map<String, Resource<out List<Any>> = coroutineScope {
val results = listOf(
async { networkRequest1(id) },
async { networkRequest2(id) },
async { networkRequest2(id) },
async { networkRequest4(id) }
).awaitAll()
.map { Resource.Success(it) }
listOf("1", "2", "3", "4").zip(results).toMap()
}
val dataCombinedStateFlow: StateFlow<Resource<Map<String, Resource<out List<Any>>>>?> =
getDataTrigger.flatMapLatest {
flow {
emit(Resource.Loading())
try {
val result = makeParallelRequests(id)
emit(Resource.Success(result))
catch (e: Throwable) {
emit(Resource.Error(e))
}
}
}
I agree with #Tenfour04 that those nested flows are overly complicated and there are several ways to simplify this (#Tenfour04's solution is a good one).
If you don't want to rewrite everything then you can fix that one line that breaks the structured concurrency:
.stateIn(viewModelScope, SharingStarted.Lazily, null)
With this the whole ViewModel flow is started in the ViewModel's scope while the view starts the collect from a separate scope (viewLifecycleOwner.lifecycleScope which would be the Fragment / Activity scope).
If you want to cancel the flow from the view, you need to use either the same scope or expose a cancel function that would cancel the ViewModel's scope.
If you want to cancel the flow from the ViewModel itself (at the return#flow statement) then you can simply add:
viewModelScope.cancel()
I want to get data from firestore, pretend that I have 10 documents in a collection. after that I must get data inside every document, so I save data in ArrayList. But FireBase never return all documents in a collection. Sometimes it returns only 5 ,6 docs in collection that has 10 docs.
my fireBaseUtil :
fun getDocumentByQueryAList( idQuery: List<String>, callBack: (ArrayList<DocumentSnapshot>) -> Unit) {
val listDocumentSnapshot = ArrayList<DocumentSnapshot>()
collRef = fireStore.collection("myCollection")
val size = idQuery.size
for (i in 0 until size) {
val query = collRef.whereEqualTo("fieldQuery", idQuery[i])
query.get().addOnSuccessListener { documents ->
for (document in documents) {
listDocumentSnapshot.add(document)
if (i == size - 1) {
callBack.invoke(listDocumentSnapshot)
}
}
}
}
}
I log out when size = 10 , but i = 8 it called invoke....
UserRepository:
FireBaseUtil.getDocumentByQueryAList{
// myList.addAll(listGettedByCallback)
}
->> when I want to have data in my list I call FireBaseUtil.getDocumentByQueryAList. I know firebase return value async but I dont know how to get all my doc then receiver.invoke("callbackValue").
Please tell me is there any solution. Thank in advance.
The problem you are experiencing is that you are expecting the queries to be run in order like so:
get idQuery[0], then add to list, then
get idQuery[1], then add to list, then
get idQuery[2], then add to list, then
...
get idQuery[8], then add to list, then
get idQuery[9], then add to list, then
invoke callback
But in reality, all of the following things happen in parallel.
get idQuery[0] (add to list when finished)
get idQuery[1] (add to list when finished)
get idQuery[2] (add to list when finished)
...
get idQuery[8] (add to list when finished)
get idQuery[9] (add to list and invoke callback when finished)
If the get idQuery[9] finishes before some of the others, you will be invoking your callback before the list is completely filled.
A primitive way to fix this would be to count the number of finished get queries, and when all of them finish, then invoke the callback.
fun getDocumentByQueryAList( idQuery: List<String>, callBack: (ArrayList<DocumentSnapshot>) -> Unit) {
val listDocumentSnapshot = ArrayList<DocumentSnapshot>()
collRef = fireStore.collection("myCollection")
val size = idQuery.size
val finishedCount = 0
for (i in 0 until size) {
val query = collRef.whereEqualTo("fieldQuery", idQuery[i])
query.get().addOnSuccessListener { documents ->
for (document in documents) {
listDocumentSnapshot.add(document)
}
if (++finishedCount == size) { // ++finishedCount will add 1 to finishedCount, and return the new value
// all tasks done
callBack.invoke(listDocumentSnapshot)
}
}
}
}
However, this will run into issues where the callback is never invoked if any of the queries fail. You could use a addOnFailureListener or addOnCompleteListener to handle these failed tasks.
The more correct and proper way to do what you are expecting is to make use of Tasks.whenAll, which is used in a similar fashion to how you see JavaScript answers using Promise.all. I'm still new to Kotlin syntax myself, so expect the following block to potentially throw errors.
fun getDocumentByQueryAList( idQueryList: List<String>, callBack: (ArrayList<DocumentSnapshot>) -> Unit) {
val listDocumentSnapshot = ArrayList<DocumentSnapshot>()
collRef = fireStore.collection("myCollection")
val getTasks = new ArrayList<Task<Void>>();
for (idQuery in idQueryList) {
val query = collRef.whereEqualTo("fieldQuery", idQuery)
getTasks.add(
query.get()
.onSuccessTask { documents ->
// if query succeeds, add matching documents to list
for (document in documents) {
listDocumentSnapshot.add(document)
}
}
)
}
Tasks.whenAll(getTasks)
.addOnSuccessListener { results ->
callback.invoke(listDocumentSnapshot)
}
.addOnFailureListener { errors ->
// one or more get queries failed
// do something
}
}
Instead of using the callback, you could return a Task instead, where the last bit would be:
return Tasks.whenAll(getTasks)
.onSuccessTask { results ->
return listDocumentSnapshot
}
This would allow you to use the following along with other Task and Tasks methods.
getDocumentByQueryAList(idQueryList)
.addOnSuccessListener { /* ... */ }
.addOnFailureListener { /* ... */ }
Use smth like this using RxJava:
override fun getAllDocs(): Single<List<MyClass>> {
return Single.create { emitter ->
db.collection("myCollection").get()
.addOnSuccessListener { documents ->
val list = mutableListOf<MyClass>()
documents.forEach { document ->
list.add(mapDocumentToMyClass(document))}
emitter.onSuccess(list)
}
.addOnFailureListener { exception ->
emitter.onError(exception)
}
}
}
private fun mapDocumentToMyClass(documentSnapshot: QueryDocumentSnapshot) =
MyClass(
documentSnapshot.get(ID).toString(),
documentSnapshot.get(SMTH).toString(),
// some extra fields
null
)
Or smth like this using coroutine:
override suspend fun getAllDocs(): List<MyClass> {
return try {
val snapshot =
db.collection("myCollection")
.get()
.await()
snapshot.map {
mapDocumentToMyClass(it)
}
} catch (e: Exception) {
throw e
}
}
This functions helps you to get all data from one doc
I have a flow like this:
fun createRawDataFlow() = callbackFlow<String> {
SensorProProvider.getInstance(this#MainActivity).registerDataCallback { sensorProDeviceInfo, bytes ->
val dataString = bytes.map { it.toString() }.reduce { acc, s -> "$acc, $s" }
val hexString = HEXUtils.byteToHex(bytes)
Log.e("onDataReceived", "deviceInfo: ${sensorProDeviceInfo.deviceIdentify}, dataSize:${bytes.size}, data:$dataString")
offer(hexString)
}
awaitClose { }
}
GlobalScope.launch(Dispatchers.IO) {
createRawDataFlow()
.map {
Log.e("onDataReceived", "map2: ${Thread.currentThread().name}")
// what I want is collecting 10 emits of sensor's data, and return a list of them
// arraylistOf<String>(/* 10 hexStrings here*/)
}
.flowOn(Dispatchers.IO)
.collect {
Log.e("onDataReceived", "thread: ${Thread.currentThread().name}, hexData:$it")
}
}
Just like the comment in the code. I wanna collect 10 hex strings from the flow because of these strings come from the same period of time, and then pack them in an array list for return. How can I achieve this? Is there any operator similar to map to do this? Btw, forgive my poor English.
If you need a batch collection, and you do not wanna cancel the original flow, you could adjust your emitting flow function in a way that it holds a cache for the values.
/*
* Returns a list of at least [batchSize] integers.
*/
fun aFlow(cacheSize: Int = 10): Flow<List<Int>> {
var counter: Int = 0
val cache: MutableList<Int> = mutableListOf()
return flow {
while(currentCoroutineContext().isActive) {
cache.add(counter++)
if (cache.size >= cacheSize) {
emit(cache)
cache.clear()
}
delay(500L) // the delay is just to simulate incoming sensor data
}
}
}
Generic Solution
To make this a bit more generic I created a generic extension function on flow you can apply to any Flow you wanna have a batch list returned.
Consider we have an infiniteFlow of integers:
fun infiniteFlow(): Flow<Int> {
var counter: Int = 0
return flow {
while (currentCoroutineContext().isActive) {
emit(counter++)
delay(250L) // the delay is just to simulate incoming sensor data
}
}
}
And this batch extension function:
/**
* Maps the Flow<T> to Flow<List<T>>. The list size is at least [batchSize]
*/
fun <T> Flow<T>.batch(batchSize: Int = 10): Flow<List<T>> {
val cache: MutableList<T> = mutableListOf()
return map {
cache.apply { add(it) }
}.filter { it.size >= batchSize }
.map {
mutableListOf<T>().apply { // copy the list and clears the cache
addAll(cache)
cache.clear()
}
}
}
Note: This is just an example. It is not optimized or tested for edge-cases!
You can then use this function like:
infiniteFlow().batch(batchSize = 12).collect { println(it) }
First you apply take
fun <T> Flow<T>.take(count: Int): Flow<T>
Returns a flow that contains first count elements. When count elements are consumed, the original flow is cancelled. Throws IllegalArgumentException if count is not positive.
Then you apply toCollection
suspend fun <T, C : MutableCollection<in T>> Flow<T>.toCollection(destination: C): C
Collects given flow into a destination
Between take and toCollection you can place other operations if you need. This is how it looks all together:
val hexList: List<String> = createRawDataFlow().take(10) // take first 10 items
.map { ... } // do some (optional) mapping
.toCollection(mutableListOf())
// after the flow is cancelled (all items consumed) you have the elements in hexList
Documentation
Flow
I am making a network repository that supports multiple data retrieval configs, therefore I want to separate those configs' logic into functions.
However, I have a config that fetches the data continuously at specified intervals. Everything is fine when I emit those values to the original Flow. But when I take the logic into another function and return another Flow through it, it stops caring about its coroutine scope. Even after the scope's cancelation, it keeps on fetching the data.
TLDR: Suspend function returning a flow runs forever when currentCoroutineContext is used to control its loop's termination.
What am I doing wrong here?
Here's the simplified version of my code:
Fragment calling the viewmodels function that basically calls the getData()
lifecycleScope.launch {
viewModel.getLatestDataList()
}
Repository
suspend fun getData(config: MyConfig): Flow<List<Data>>
{
return flow {
when (config)
{
CONTINUOUS ->
{
//It worked fine when fetchContinuously was ingrained to here and emitted directly to the current flow
//And now it keeps on running eternally
fetchContinuously().collect { updatedList ->
emit(updatedList)
}
}
}
}
}
//Note logic of this function is greatly reduced to keep the focus on the problem
private suspend fun fetchContinuously(): Flow<List<Data>>
{
return flow {
while (currentCoroutineContext().isActive)
{
val updatedList = fetchDataListOverNetwork().await()
if (updatedList != null)
{
emit(updatedList)
}
delay(refreshIntervalInMs)
}
Timber.i("Context is no longer active - terminating the continuous-fetch coroutine")
}
}
private suspend fun fetchDataListOverNetwork(): Deferred<List<Data>?> =
withContext(Dispatchers.IO) {
return#withContext async {
var list: List<Data>? = null
try
{
val response = apiService.getDataList().execute()
if (response.isSuccessful && response.body() != null)
{
list = response.body()!!.list
}
else
{
Timber.w("Failed to fetch data from the network database. Error body: ${response.errorBody()}, Response body: ${response.body()}")
}
}
catch (e: Exception)
{
Timber.w("Exception while trying to fetch data from the network database. Stacktrace: ${e.printStackTrace()}")
}
finally
{
return#async list
}
list //IDE is not smart enough to realize we are already returning no matter what inside of the finally block; therefore, this needs to stay here
}
}
I am not sure whether this is a solution to your problem, but you do not need to have a suspending function that returns a Flow. The lambda you are passing is a suspending function itself:
fun <T> flow(block: suspend FlowCollector<T>.() -> Unit): Flow<T> (source)
Here is an example of a flow that repeats a (GraphQl) query (simplified - without type parameters) I am using:
override fun query(query: Query,
updateIntervalMillis: Long): Flow<Result<T>> {
return flow {
// this ensures at least one query
val result: Result<T> = execute(query)
emit(result)
while (coroutineContext[Job]?.isActive == true && updateIntervalMillis > 0) {
delay(updateIntervalMillis)
val otherResult: Result<T> = execute(query)
emit(otherResult)
}
}
}
I'm not that good at Flow but I think the problem is that you are delaying only the getData() flow instead of delaying both of them.
Try adding this:
suspend fun getData(config: MyConfig): Flow<List<Data>>
{
return flow {
when (config)
{
CONTINUOUS ->
{
fetchContinuously().collect { updatedList ->
emit(updatedList)
delay(refreshIntervalInMs)
}
}
}
}
}
Take note of the delay(refreshIntervalInMs).
Usually I'm returning from my dao suspend function:
#Dao
interface DataDao {
#Query("SELECT * FROM data")
fun getAllData(): List<Data>
}
And handle the call within the repository:
class DataRepository(
private val dataDao: DataDao
) {
fun getAllData(): Flow<DataState> = flow {
val cacheResult = safeDatabaseCall(dispatcher = Dispatchers.IO) { dataDao.getAllData() }
//handle cacheResult, convert to DataState, emit DataState values
}.flowOn(Dispatchers.IO)
}
With generic fun:
suspend fun <T> safeDatabaseCall(
dispatcher: CoroutineDispatcher,
cacheCall: suspend () -> T?
): CacheResult<T?> {
return withContext(dispatcher) {
try {
withTimeout(10000L) {
CacheResult.Success(cacheCall.invoke())
}
} catch (t: Throwable) {
when (t) {
is TimeoutCancellationException -> {
CacheResult.Error("Timeout error")
}
else -> {
CacheResult.Error("Unknown error")
}
}
}
}
}
The problem is that I want return fun getAllData(): Flow<List<Data>> instead of fun getAllData(): List<Data> In order to get immediate updates, But if I'm returning Flow from the Dao, I can't handle the call with safe call and catch errors.
I thought about collecting the data, but if i'm collecting the data the call already done without error handling
Basically I need the cache result return CacheResult<Data> and not CacheResult<Flow<Data>>
How can I solve the problem And make a generic safeDatabaseCall while returning Flow from Dao?
So if I understand correctly you just want to handle the query and return of information safely in a flow. My only question is around the types. I can sorta assume Data DataState and CacheResult are not the same types so I use a "magic" function that converts the intermediary values to the correct one. You will need to adjust accordingly
class DataRepository(
private val dataDao: DataDao
) {
fun getAllData(): Flow<DataState> = flow {
val result = safeDatabaseCall(dispatcher = Dispatchers.IO) {
dataDao.getAllData()
}
// Emit the result
emit(result)
}.catch { t : Throwable ->
// Do our transformation like before
val result = when (t) {
is TimeoutCancellationException -> {
CacheResult.Error("Timeout error")
}
else -> {
CacheResult.Error("Unknown error")
}
}
// And because catch is actually extending a FlowCollector
// We can emit the result in the stream
emit(result)
}.map { cacheResult ->
convertToDataOrDataState(cacheResult)
}
You shouldn't need flowOn with a dispatcher here since the work inside this flow doesn't require thread dispatching
to Dispatcher.IO. The code we are putting in our flow, is purely exception handling and invoking a function. The only place that seems to require any manual dispatch changing is, safeDatabaseCall(). I am not familiar with this function but if it does exist and takes a dispatcher for the result of actualing making the db calls on an IO thread, then all should be good without flowOn. Otherwise you will be switching dispatchers from original dispatcher -> IO and then to IO again. It's not much but the extra no-op context switch doesn't add anything other than confusion later on.
The flow itself traps any upstream issues and you then make them part of the resulting flow