Develop Reference

Collect flow but only any new values, not the currently existing value

Currently struggling with this one, and so far no combination of SharedFlow and StateFlow have worked.
I have a flow that might have already started with a value, or not.
Using that flow I want to collect any new values that are emitted after I start collecting.
At this moment all my attempts have always failed, no matter what I try it always gets the current value as soon as I start collecting.
An example of what I am trying to achieve:
Given a Flow (could be any type, Int is just for simplification)
with the following timeline: value 4 is emitted | value 2 is emitted | value 10 is emitted
I want to be able to do the following:
If I start collecting after value 4 has already been emitted, I want to only receive anything after that, in this case it would collect 2 and 10 once emitted
If I start collecting after value 2 then it would only receive the 10
If I start collecting before 4 then it would receive 4, 2 and 10
Tried SharedFlow and Stateflow, tried with replay = 0 and WhileSubscribed, no combination I could find would do what I am looking for.
The only workaround so far that I found was to locally register the time I start my .collect{ } and compare with the start time of the item I receive in the collect. In this case I have the object I am using has a specific origin time, but this workaround will not work for everything like the example above with Integers.
EDIT: Adding implementation example as requested for SharedFlow
This is tied to a Room database call that returns a Flow<MyObject>
MyFragment.kt
lifecycleScope.launch(Dispatchers.IO) {
viewModel.getMyObjectFlow.shareIn(
viewModel.viewModelScope, // also tried with fragment lifecyclescope
SharingStarted.WhileSubscribed(), // also tried with the other 2 options
replay = 0,
).collect{
...
}
}

You have a misconception of how flows work. They always emit only after you start collecting. They emit on-demand. Let's get this example:
val flow1 = flow {
println("Emitting 1")
emit(1)
delay(10.seconds)
println("Emitting 2")
emit(2)
}
delay(5.seconds)
println("Start collecting")
flow1.collect {
println("Collected: $it")
}
The output is:
Start collecting
Emitting 1
Collected: 1
not:
Emitting 1
Start collecting
Collected: 1
This is because flow starts emitting only after you start collecting it. Otherwise, it would have nowhere to emit.
Of course, there are flows which emit from some kind of a cache, queue or a buffer. For example shared flows do this. In that case it looks like you collect after emitting. But this is not really the case. Technically speaking, it works like this:
val buffer = listOf(1 , 2, 3)
val flow1 = flow {
buffer.forEach {
println("Emitting $it")
emit(it)
}
}
It still emits after you start collecting, but it just emits from the cache. Of course, the item was added to the cache before you started collecting, but this is entirely abstracted from you. You can't know why a flow emitted an item. From the collector perspective it always emitted just now, not in the past. Similarly, you can't know if a webserver read the data from the DB or a cache - this is abstracted from you.
Summing up: it is not possible to collect only new items from just any flow in an universal way. Flows in general don't understand the concept of "new items". They just emit, but you don't know why they do this. Maybe they somehow generate items on-the-fly, maybe they passively observe external events or maybe they re-transmit some items that they collected from another flow. You don't know that.
While developing your solution, you need to understand what was the source of items and develop your code accordingly. For example, if the source is a regular cold flow, then it never starts doing anything before you start collecting. If the source is a state flow, you can just drop the first item. If it is a shared flow or a flow with some replay buffer, then the situation is more complicated.
One possible approach would be to start collecting earlier than we need, initially ignore all collected items and at some point in time start processing them. But this is still far from perfect and it may not work as we expect.

It doesn't make sense to use shareIn at the use site like that. You're creating a shared Flow that cannot be shared because you don't store the reference for other classes to access and use.
Anyway, the problem is that you are creating the SharedFlow at the use site, so your shared flow only begins collecting from upstream when the fragment calls this code. If the upstream flow is cold, then you will be getting the first value emitted by the cold flow.
The SharedFlow should be created in the ViewModel and put in a property so each Fragment can collect from the same instance. You'll want to use SharingStarted.Eagerly to prevent the cold upstream flow from restarting from the beginning when there are new subscribers after a break.

Is there a way to create a StateFlow from a Flow

Can I create a StateFlow from a Flow so that I can get the .value of it? Is there a way to do it without using .collect?
Something like
val myStateFlow = StateFlow<MyDataType>(this.existingFlow)
So that later, when someone clicks a button, I can say what the last value of the flow is?
fun handleButton() {
val lastValue = myStateFlow.value
print(lastValue)
}
I would prefer not using collect, since I dont want the flow to flow until someone else decides to collect it.

There is a built-in function called stateIn.
Is there a way to do it without using .collect?
I would prefer not using collect, since I dont want the flow to flow until someone else decides to collect it.
If you use the non-suspending overload of stateIn, the flow collection doesn't have to start right away, but you have to provide an initial value for the state (because this call cannot wait for the first value of the flow if it doesn't suspend).
In that case, you can play with the started argument to make the actual collection lazy, but note that accessing myStateFlow.value won't trigger the collection of the flow and will just return the initial value over and over. Only terminal flow operators (like collect) on the StateFlow will actually trigger the underlying flow collection, which is probably not what you want (but maybe it is!).
Note that you have to have a coroutine running to get the values from the initial flow and set the state accordingly if you want to access values via myStateFlow.value. This is actually what stateIn does by default: it starts a coroutine that collects the flow to set the StateFlow's value - so it technically uses collect.

Transforming a flow of PagingData from paging3 into a StateFlow of PagingData

I am learning pagination using paging3 from the jetpack library. I am making a call to an api to receive a list of articles. I've noticed that the result received after making the call in the repository using the Pager is a flow of PagingData containing the desired result like so:
Flow<PagingData<Articles>>
When I receive this flow in my ViewModel I would like to convert it into a Stateflow. I have tried using the stateIn operator but it requires a default value which I think would be a StateFlow , this is where I am stuck. How can I convert the flow of PagingData into a Stateflow and is it advisable to do so?

I also have a question: why do you want to convert the Flow of PagingData into a Stateflow when you get the Article and show them on UI?
In my practice, no need to touch Stateflow. If I want to get Articles with Paging lib 3. Pay attention to Flow, ArticleRemoteMediator: RemoteMediator<Int, ArticleEntity> ...
I think this article can help you to archive your goal. (getting article with Paging 3) https://medium.com/nerd-for-tech/pagination-in-android-with-paging-3-retrofit-and-kotlin-flow-2c2454ff776e

I would say it depends on the usecase, while using regular flow every new subscriber triggers a new flow emission this can lead to resource wastage (unnecessary network requests) on configuration change eg screen ration, if you would desire such a behaviour its okey to use regular if not consider using StateFlow.
You can convert a regular flow emissions into StateFlow by using stateIn extension and set the initial state as empty as follows.
someFlow.stateIn(coroutineScope, SharingStarted.WhileSubscribed(), PagingData.empty())
According to the documentation PagingData.empty() immediately displays an empty list of items with dispatching any load state update to a presenter

BehaviorSubject vs PublishSubject

I'm trying to get my head around the golden rule (if any) about:
When to use BehaviorSubject ?
and
When to use PublishSubject ?
The difference between them is very clear
There are many kinds of subjects. For this specific requirement, a PublishSubject works well because we wish to continue the sequence from where it left off. So assuming events 1,2,3 were emitted in (B), after (A) connects back we only want to see 4, 5, 6. If we used a ReplaySubject we would see [1, 2, 3], 4, 5, 6; or if we used a BehaviorSubject we would see 3, 4, 5, 6 etc.
(source : How to think about Subjects in RxJava (Part 1))
I have seen that Subject's are used in two contexts (at least), UI context and listener context.
UI context (MVVM as example)
For example here a BehaviorSubject is used, and it's clear why they use Subject and not Observable but I have changed the BehaviorSubject to PublishSubject but the app behavior still the same.
Listener context
Why they make project field a BehaviorSubject and not PublishSubject ?

The main difference between PublishSubject and BehaviorSubject is that the latter one remembers the last emitted item. Because of that BehaviorSubject is really useful when you want to emit states.
Why they make project field a BehaviorSubject and not PublishSubject ?
Probably because they want to be able to retrieve the last emitted project with this method:
#Override public #NonNull Observable<Project> project() {
return this.project;
}

PublishSubject: Starts empty and only emits new elements to subscribers.
There is a possibility that one or more items may be lost between the time the Subject is created and the observer subscribes to it because PublishSubject starts emitting elements immediately upon creation.
BehaviorSubject: It needs an initial value and replays it or the latest element to new subscribers. As BehaviorSubject always emits the latest element, you can’t create one without giving a default initial value.
BehaviorSubject is helpful for depicting "values over time". For example, an event stream of birthdays is a Subject, but the stream of a person's age would be a BehaviorSubject.

Publish Subject: Here, if a student entered late into the classroom, he just wants to listen from that point of time when he entered the classroom. So, Publish will be the best for this use-case.
Behavior Subject: Here, if a student entered late into the classroom, he wants to listen the most recent things(not from the beginning) being taught by the professor so that he gets the idea of the context. So, here we will use Behavior.

The difference on BehaviourSubject and PublishSubject relies on how long they keep the data they captures, in instance the PublishSubject only keeps the data available at moment and keeps updating on every entry while BehaviourSubject keeps the last data inserted, so you may use for example to confirm password on a signup form and as an example for PublishSubject, performing a search and it has to update the data constantly in order to give accurate results and there's no too much necessity to compare data that are being inserted.
As reference i leave this two photos from http://reactivex.io/documentation/subject.html
PublishSubject
BehaviourSubject

Trigger Observable on Subject's onNext and Share Result

I have a button which when pressed should make the btnSubject's onNext fire and make an API call in an Observable created in my ViewModel like so:
val apiObservable =
btnSubject.flatMap{ apiService.getSomething()
.toResponseOrErrorObservable()
.subscribeOn(Schedulers.io())}
Then I can reuse this observable to create two more, which are then subscribed to from my view allowing me to keep the logic in my ViewModel like so:
val successObservable = apiObservable.filter{ it.isSuccess() }
val failureObservable = apiObservable.filter{ it.isFailure() }
So apiObservable is triggered by the btnSubject.onNext() being called.
The view is then updated because it's listening to the successObservable and failureObservable
Is this possible? Perhaps with a .share() on the apiObservable?
UPDATE
I added the share operator and all observables emitted items when first subscribing. Even the filters didn't stop it... I must be missing something obvious

There might be a few way to do that.
As you have written, using share() operator multiplies output to many Subscribers. However, you have to be careful, that you also have to call connect() to turn cold Observable into hot one. If calling also replay(), you woudln't need to call connect() many times.
(Source)
However, there is more simple solution: use Jake Wharton's library RxReplayingShare. The author of previous blog post suggests it in his next article