I am trying to develop a client for Hacker News using this API, just for learning how Android works, as a personal project. I tried following some tutorials, but I am stuck at a certain point.
I want to retrieve the top N stories' titles, upvotes, etc. This would be done, using this api by:
Making a request to the api to retrieve the ID's of top posts (500 of them, to be exact)
For each ID, make a request to the api's posts endpoint to retrieve the details.
It seems that I am stuck on how to create N different network requests for the posts that I want, retrieving them and putting them on a List, then displaying them on my Fragment.
I am trying to follow an MVVM pattern, with Repositories. The relevant files are here:
NewsApi.kt:
interface NewsApi {
#GET("topstories.json")
fun getTopStories() : Single<List<Int>>
#GET("item/{id}")
fun getItem(#Path("id") id: String): Single<News>
}
MainRepository.kt (I):
interface MainRepository {
fun getTopStoryIDs(): Single<List<Int>>
fun getStory(storyId: Int): Single<News>
fun getTop20Stories(): Single<List<News>>
}
The News object is a simple data class with all the JSON fields that are returned from item/{id}, so I am omitting it.
Here is my Repository, the implementation:
class DefaultMainRepository #Inject constructor(
private val api: NewsApi
) : MainRepository {
override fun getTopStoryIDs(): Single<List<Int>> {
return api.getTopStories()
}
override fun getStory(storyId: Int): Single<News> {
return api.getItem(storyId.toString())
}
override fun getTop20Stories(): Single<List<News>> {
TODO("HOW?")
}
}
The top questions I have are:
How can I make chained API calls in this way, using Retrofit / RxJava? I have reviewed previous answers using flatMap, but in my case, using a List of Int's, I do not actually know how to do that correctly.
Is this the right way to go about this? Should I just ditch the architectural choices I've made, and try to think in a wholly new way?
Say I can complete getTop20Stories (which, as the name implies, should retrieve 20 of the news, using the result from getTopStoryIDs, first 20 elements for the time should do the trick), how would I be able to retrieve data from it? Who should do the honors of retrieving the response? VM? Fragment?
Thanks in advance.
Single as a return type in your case will not be the best option because it is designed to only maintain single stream. concatMap or flatMap on Single will not either because it will try to map list of items to another list of items which is not the case
here.
Instead you could use Observable or map your Single to Observable by using toObservable() with concatMapIterable operator which maps your single item to sequence of items.
I used concatMap operator instead of flatMap because it maintains order of the list items so your data won't be mixed up.
getTopStoryIDs()
.map { it.take(20) }
.toObservable()
.concatMapIterable { it }
.concatMapSingle { singleId ->
api.getItem(singleId)
}
.toList()
.subscribe { items ->
//do something with your items
}
This code will work but it's not the best solution because you will make 20 or more api calls which will hurt your network data and device battery so I wouldn't use it if it is not completely necessary.
If you have any questions fill free to ask :)
You where on the right track with FlatMap.
Something like this should do the trick:
getTopStoryIDs().flatMap { storyId -> getStory(storyId) }
Related
On various occasions I've seen people use Kotlin Flows when retrieving a list of objects from a local database, but nowhere have I noticed any warning/error about using them when retrieving single objects (as opposed to a list), or even specification that Flows are to be used only on lists
My question is, will using Flows on single objects cause an error/problem? If not does it hinder performance? In case no, why is it generally not used then?
Sample code of what I mean:
Single object retrieval with Flows:
#Query("SELECT * FROM objects WHERE id = :id")
fun getObjectById(id: Int): Flow<Object>
Retrieving function in ViewModel:
fun objectRetrieval(id: Int) {
// ... More code
getObjectById(id)
.onEach { object: Object ->
// Operations
}
.launchIn(viewModelScope)
}
Thank you for your time!
I think there are multiple misconceptions here, both in the question and in comments/answer.
Flows are indeed related to sequences of values processed asynchronously. In simple words that means: if in the future we plan to receive some data multiple times, flows are just for this purpose. Examples are: user events, incoming messages in an instant messaging application, etc.
If we plan to receive the data only once, we can still use flow, but this is an overkill and it adds unnecessary complication. In these cases it is better to use a suspend function as it is easier to use and it is very clear about the fact the data is received only once.
However, it doesn't really matter, if the data we receive is a single object or a list of objects. As a matter of fact, list is an object as well:
suspend fun foo(): String - receive a single object once.
suspend fun foo(): List<String> - receive a list of objects, all at once.
fun foo(): Flow<String> - receive a single object multiple times.
fun foo(): Flow<List<String>> - receive a list of objects multiple times.
All of above cases make sense and can be used in different scenarios.
Now, going to your specific example, there is another misconception there. Your getObjectById() doesn't receive only a single value, but a sequence of values. By using Flow we say to Room that we want to observe the data for changes. First, it sends you the current data for the object, but if the object ever changes, you receive updated data again.
If you want to only get the current data and do not observe for changes, use a suspend function instead.
Yes you should flow single objects Singe Source of Truth Architecture
val numLightsObserver: Observer<Int> = Observer { i: Int ->
i.let { jx.setTitle(i.toString()) }
}
wordViewModel.numLights.observe(this, numLightsObserver)
Now I have a fully automated object for my UI what will I do with the number of lights. Above I convert it to a string for this demo commit.
here's a link to commit diff I had to modify 4 files to make it work
diff on github where I try it flow a single object
I am new with kotlin flow and I am working about this document. Kotlin Flows. In this code every five seconds datasource fetch data from api and emits it.
This is my example datasource class.
I am getting data and emitting it.
class RemoteDataSourceImpl #Inject constructor(
private val api:CryptoApi
): RemoteDataSource {
override suspend fun cryptoList(): Flow<List<CryptoCoinDto>> {
return flow {
while (true){
val data = api.getCoinList()
emit(data)
delay(5000L)
}
}
}
}
This is my example repository.
I am mapping data and saving it room database. I want to get data from room database and emit it because of single source of truth principle but I still have to return dataSource because if I open new flow{} I can't reach datasource's data. Of course I can fix the problem by using List instead of Flow<List> inside of RemoteDataSource class. But I want to understand this example. How can I apply here single source of truth.
class CoinRepositoryImpl #Inject constructor(
private val dataSource:RemoteDataSource,
private val dao: CryptoDao
):CoinRepository {
override fun getDataList(): Flow<List<CryptoCoin>> {
dataSource.cryptoList().map { dtoList ->
val entityList = dtoList.map { dto ->
dto.toCryptoEntity()
}
dao.insertAll(entityList)
}
return dataSource.cryptoList().map {
it.map { it.toCryptoCoin() }
}
}
This is actually more complicated than it seems. Flows were designed to support back-pressure which means that they usually only produce items on demand, when being consumed. They are passive, instead of pushing items, items are pulled from the flow.
(Disclaimer: this is all true for cold flows, not for hot flows. But cryptoList() is a cold flow.)
It was designed this way to greatly simplify cases when the consumer is slower than producer or nobody is consuming items at all. Then producer just stops producing and everything is fine.
In your case there are two consumers, so this is again more complicated. You need to decide what should happen if one consumer is slower than the other. For example, what should happen if nobody collects data from getDataList()? There are multiple options, each requires a little different approach:
Stop consuming the source flow and therefore stop updating the database.
Update the database all the time and queue items if nobody is collecting from getDataList(). What if there are more and more items in the queue?
Update the database all the time and discard items if nobody is collecting from getDataList().
Ad.1.
It can be done by using onEach():
return dataSource.cryptoList().onEach {
// update db
}.map {
it.map { it.toCryptoCoin() }
}
In this solution updating the database is a "side effect" of consuming the getDataList() flow.
Ad.2. and Ad.3.
In this case we can't passively wait until someone asks us for an item. We need to actively consume items from the source flow and push them to the downstream flow. So we need a hot flow: SharedFlow. Also, because we remain the active side in this case, we have to launch a coroutine that will do this in the background. So we need a CoroutineScope.
Solution depends on your specific needs: do you need a queue or not, what should happen if queue exceeded the size limit, etc., but it will be similar to:
return dataSource.cryptoList().onEach {
// update db
}.map {
it.map { it.toCryptoCoin() }
}.shareIn(scope, SharingStarted.Eagerly)
You can also read about buffer() and MutableSharedFlow - they could be useful to you.
I'm trying out the new coroutine's flow, my goal is to make a simple repository that can fetch data from a web api and save it to db, also return a flow from the db.
I'm using room and firebase as the web api, now everything seems pretty straight forward until i try to pass errors coming from the api to the ui.
Since i get a flow from the database which only contains the data and no state, what is the correct approach to give it a state (like loading, content, error) by combining it with the web api result?
Some of the code i wrote:
The DAO:
#Query("SELECT * FROM users")
fun getUsers(): Flow<List<UserPojo>>
The Repository:
val users: Flow<List<UserPojo>> = userDao.getUsers()
The Api call:
override fun downloadUsers(filters: UserListFilters, onResult: (result: FailableWrapper<MutableList<UserApiPojo>>) -> Unit) {
val data = Gson().toJson(filters)
functions.getHttpsCallable("users").call(data).addOnSuccessListener {
try {
val type = object : TypeToken<List<UserApiPojo>>() {}.type
val users = Gson().fromJson<List<UserApiPojo>>(it.data.toString(), type)
onResult.invoke(FailableWrapper(users.toMutableList(), null))
} catch (e: java.lang.Exception) {
onResult.invoke(FailableWrapper(null, "Error parsing data"))
}
}.addOnFailureListener {
onResult(FailableWrapper(null, it.localizedMessage))
}
}
I hope the question is clear enough
Thanks for the help
Edit: Since the question wasn't clear i'll try to clarify. My issue is that with the default flow emitted by room you only have the data, so if i were to subscribe to the flow i would only receive the data (eg. In this case i would only receive a list of users). What i need to achieve is some way to notify the state of the app, like loading or error. At the moment the only way i can think of is a "response" object that contains the state, but i can't seem to find a way to implement it.
Something like:
fun getUsers(): Flow<Lce<List<UserPojo>>>{
emit(Loading())
downloadFromApi()
if(downloadSuccessful)
return flowFromDatabase
else
emit(Error(throwable))
}
But the obvious issue i'm running into is that the flow from the database is of type Flow<List<UserPojo>>, i don't know how to "enrich it" with the state editing the flow, without losing the subscription from the database and without running a new network call every time the db is updated (by doing it in a map transformation).
Hope it's clearer
I believe this is more of an architecture question, but let me try to answer some of your questions first.
My issue is that with the default flow emitted by room you only have
the data, so if i were to subscribe to the flow i would only receive
the data
If there is an error with the Flow returned by Room, you can handle it via catch()
What i need to achieve is some way to notify the state of the app,
like loading or error.
I agree with you that having a State object is a good approach. In my mind, it is the ViewModel's responsibility to present the State object to the View. This State object should have a way to expose errors.
At the moment the only way i can think of is a "response" object that
contains the state, but i can't seem to find a way to implement it.
I have found that it is easier to have the State object that the ViewModel controls be responsible for errors instead of an object that bubbles up from the Service layer.
Now with these questions out of the way, let me try to propose one particular "solution" to your issue.
As you mention, it is common practice to have a Repository that handles retrieving data from multiple data sources. In this case, the Repository would take the DAO and an object that represents getting data from the network, let's call it Api. I am assuming that you are using FirebaseFirestore, so the class and method signature would look something like this:
class Api(private val firestore: FirebaseFirestore) {
fun getUsers() : Flow<List<UserApiPojo>
}
Now the question becomes how to turn a callback based API into a Flow. Luckily, we can use callbackFlow() for this. Then Api becomes:
class Api(private val firestore: FirebaseFirestore) {
fun getUsers() : Flow<List<UserApiPojo> = callbackFlow {
val data = Gson().toJson(filters)
functions.getHttpsCallable("users").call(data).addOnSuccessListener {
try {
val type = object : TypeToken<List<UserApiPojo>>() {}.type
val users = Gson().fromJson<List<UserApiPojo>>(it.data.toString(), type)
offer(users.toMutableList())
} catch (e: java.lang.Exception) {
cancel(CancellationException("API Error", e))
}
}.addOnFailureListener {
cancel(CancellationException("Failure", e))
}
}
}
As you can see, callbackFlow allows us to cancel the flow when something goes wrong and have someone donwnstream handle the error.
Moving to the Repository we would now like to do something like:
val users: Flow<List<User>> = Flow.concat(userDao.getUsers().toUsers(), api.getUsers().toUsers()).first()
There are a few caveats here. first() and concat() are operators you will have to come up with it seems. I did not see a version of first() that returns a Flow; it is a terminal operator (Rx used to have a version of first() that returned an Observable, Dan Lew uses it in this post). Flow.concat() does not seem to exist either. The goal of users is to return a Flow that emits the first value emitted by any of the source Flows. Also, note that I am mapping DAO users and Api users to a common User object.
We can now talk about the ViewModel. As I said before, the ViewModel should have something that holds State. This State should represent data, errors and loading states. One way that can be accomplished is with a data class.
data class State(val users: List<User>, val loading: Boolean, val serverError: Boolean)
Since we have access to the Repository the ViewModel can look like:
val state = repo.users.map {users -> State(users, false, false)}.catch {emit(State(emptyList(), false, true)}
Please keep in mind that this is a rough explanation to point you in a direction, there are many ways to accomplish state management and this is by no means a complete implementation. It may not even make sense to turn the API call into a Flow, for example.
The answer from Emmanuel is really close to answering what i need, i need some clarifications about some of it.
It may not even make sense to turn the API call into a Flow
You are totally right, in fact i only want to actually make it a coroutine, i don't really need it to be a flow.
If there is an error with the Flow returned by Room, you can handle it via catch()
Yes i discovered this after posting the question. But my problem is more something like:
I'd like to call a method, say "getData", this method should return the flow from db, start the network call to update the db (so that i'm going to be notified when it's done via the db flow) and somewhere in here, i would need to let the ui know if db or network errored, right?. Or should i maybe do a separate "getDbFlow" and "updateData" and get the errors separately for each one?
val users: Flow> = Flow.concat(userDao.getUsers().toUsers(), api.getUsers().toUsers()).first()
This is a good idea, but i'd like to keep the db as the single source of truth, and never return to the ui any data directly from the network
I just wanted to know and how to implement multiple API call observer in a single view(Act or Fragment)
So my question brief is I have a multiple API's calls in a single Activity like fetch user detail, fetch banner image list and products list via ViewModel and Livedata,
So is it compulsory to observe three livedata depend on three API calls?
I want to observe only a single livedata for three different APIs.
like below example, I have three live data for three APIs
var userLiveData = MutableLiveData<UserDetails>()
var bannerLiveData = MutableLiveData<BannerDetails>()
var productLiveData = MutableLiveData<Product>()
Call rest API and observe in Act like below
userLiveData.observe(this, Observer { data ->
})
bannerLiveData.observe(this, Observer { data ->
})
productLiveData.observe(this, Observer { data ->
})
it is ok no issue but I think it is not good practice to observe every API call, I need to combine all three API calls to one so is there any solution for that?
I read about MediatorLiveData but cant help!
Forgive me for my bad English.
If anyone knows anything please help me.
Recently, I´ve read about how important it is to have a Single-Source-Of-Truth (SSOT) when designing an app´s backend (repository, not server-side-backend). https://developer.android.com/topic/libraries/architecture/guide.html
By developing a news-feed app (using the awesome https://newsapi.org/) I am trying to learn more about app architecture.
However, I am unsure of how to design the repository interface for my app.
Btw.: I am using MVVM for my presentation layer. The View subscribes to the ViewModel´s LiveData. The ViewModel subscribes to RxJava streams.
So I came up with 2 approaches:
Approach 1:
interface NewsFeedRepository {
fun loadFeed(): Flowable<List<Article>>
fun refreshFeed(): Completable
fun loadMore(): Completable
}
interface SearchArticleRepository {
fun searchArticles(sources: List<NewsSource>? = null, query: String? = null): Flowable<List<Article>>
fun moreArticles(): Completable
}
interface BookmarkRepository {
fun getBookmarkedArticles(): Flowable<List<Article>>
fun bookmarkArticle(id: String): Completable
}
This approach is primarily using Flowables which emit data if the corresponding data in the underlying SSOT (database) changes (e.g old data gets replaced with fresh data from API, more data was loaded from API, ...). However, I am unsure if using a Flowable for SearchArticleRepository#searchArticles(...) makes sense. As it is like some request/response thing, where maybe a Single might me be more intuitive.
Approach 2:
interface NewsFeedRepository {
fun loadFeed(): Single<List<Article>>
fun refreshFeed(): Single<List<Article>>
fun loadMore(): Single<List<Article>>
}
interface SearchArticleRepository {
fun searchArticles(sources: List<NewsSource>? = null, query: String? = null): Single<List<Article>>
fun moreArticles(): Single<List<Article>>
}
interface BookmarkRepository {
fun getBookmarkedArticles(): Single<List<Article>>
fun bookmarkArticle(id: String): Single<Article> // Returns the article that was modified. Articles are immutable.
}
This approach is using Singles instead of Flowables. This seems very intuitive but if the data in the SSOT changes, no changes will be emitted. Instead, a call to the repository has to be made again. Another aspect to take into account is that the ViewModel may have to manage its own state.
Let´s take the FeedViewModel for example (pseudo-code).
class FeedViewModel : ViewModel() {
// Variables, Boilerplate, ...
val newsFeed: LiveData<List<Article>>
private val articles = mutableListOf<Article>()
fun loadNewsFeed() {
// ...
repository.loadFeed()
//...
// On success, clear the feed and append the loaded articles.
.subscribe({articles.clear(); articles.addAll(it)})
// ...
}
fun loadMore() {
// ...
repository.loadMore()
//...
// On success, append the newly loaded articles to the feed.
.subscribe({articles.addAll(it)})
// ...
}
}
So this might not be crucial for a smaller app like mine, but it definitely can get a problem for a larger app (see state management: http://hannesdorfmann.com/android/arch-components-purist).
Finally, I wanted to know which approach to take and why. Are there any best-practices? I know many of you have already done some larger software-projects/apps and it would be really awesome if some of you could share some knowledge with me and others.
Thanks a lot!
I'd rather go for the first approach using Observables instead of Flowables in your case:
interface NewsFeedRepository {
fun loadFeed(): Observable<List<Article>>
fun refreshFeed(): Completable
fun loadMore(): Completable
}
interface SearchArticleRepository {
fun searchArticles(sources: List<NewsSource>? = null, query: String? = null): Observable<List<Article>>
fun moreArticles(): Completable
}
interface BookmarkRepository {
fun getBookmarkedArticles(): Observable<List<Article>>
fun bookmarkArticle(id: String): Completable
}
I don't see any reason you should necessarily use Flowable for this purpose since you'll never have any OOME related issues checking your repository changes. In other words, for your use case IMHO backpressure is not necessary at all.
Check this official guide which gives us an advice of when to a Flowable over an Observable.
On the other hand, and not related to the question itself, I have serious doubts of what's the purpose of loadMore or moreArticles methods since they return a Completable. Without knowing the context, it may seem you could either refactor the method name by a better name or change the return type if they do what they seem to do by the name.
I believe the first approach is better, Your repo will update the data whenever the data is changed and your view model will be notified automatically and that's cool, while in your second approach you have to call the repo again and that's not really reactive programming.
Also, assume that the data can be changed by something rather than load more event from the view, like when new data added to the server, or some other part of the app changes the data, Now in the first approach again you get the data automatically while for the second your not even know about the changed data and you don't know when to call the method again.