How can you tell if paging is working properly? All the examples I've looked at involve using retrofit apiservice which appears to be returning pages of data, but I'm pulling down a single rss feed and parsing it into a giant List<POJO>. I suspect that my PagingSource is loading the entire list into one page, but I'm not sure how to tell.
My list has near 1000 items, so I assume it'd be good practice to implement some kind of paging/DiffUtil. I'm playing around in this with jetpack compose usingandroidx.paging:paging-compose:1.0.0-alpha12 which probably complicates things.
Can anyone give me some pointers?
class RssListSource(): PagingSource<Int, RssItem>() {
override suspend fun load(params: LoadParams<Int>): LoadResult<Int, RssItem> {
return try {
val nextPage = params.key ?: 1
val rssList: List<RssItem> = RssFeedFetcher().fetchRss()
LoadResult.Page(
data = rssList,
prevKey = if (nextPage == 1) null else nextPage - 1,
nextKey = nextPage.plus(1)
)
} catch (e: Exception){
LoadResult.Error(e)
}
}
}
class MainActivityViewModel: ViewModel() {
val rss: Flow<PagingData<RssItem>> = Pager(PagingConfig(pageSize = 10)){
RssListSource() // returned to LazyPagingItems list in #Composable
}.flow.cachedIn(viewModelScope)
}
Your data still needs a way to fetch pages of data. I would expect your RssFeedFetcher to use the page information and return a page accordingly.
You are probably correct that you are currently returning all items at once.
There's two main strategies here:
Add a long enough delay() to load() such that you have enough time to scroll to the end of the list before new page loads
class RssListSource(): PagingSource<Int, RssItem>() {
override suspend fun load(params: LoadParams<Int>): LoadResult<Int, RssItem> {
delay(5000)
...
}
Observe changes to LazyPagingItems.loadState and look for either PREPEND or APPEND switching between Loading and NotLoading.
In order for you to be able to implement pagination with the paging library you need to use a paginated API, that means that in your example, you'd need a way to fetch the RSS in a paginated fashion. The Paging library won't be able to make use of pagination if your data source does not provide a way to query for paginated data, unfortunately.
You could achieve what you want by implementing a middleware that fetches the RSS feed and splits it into pages for you to query from the Android app.
EDIT: Another approach could be to have a background task (using the Android WorkManager) to download the RSS feed and save it in a Room Database, then use the Paging library to load pages off the database. Here's a summary of how to show paginated data from a Room database: https://developer.android.com/topic/libraries/architecture/paging/v3-network-db
Related
I'm looking into using the paging 3 library for handling paging on my android app. One small hitch is that every single example I find assumes I'd always provide a page number to the API when my company uses skip/take for our APIs as does a few other APIs I use that's not under my control.
I see that paging 2 had something but it looks to be depreciated so I'm curious what their solution is for paging 3?
By skip/take do you mean you have a item-keyed source which wants a number of items to load and an offset as inputs?
In Paging2 there were explicit classes for each type of key, but in Paging3 you control how the key is interpreted directly, so you can implement a PagingSource using the offset as the key.
Naively, just to show nextKey calculation:
class MyPagingSource : PagingSource<Int, Item>() {
override suspend fun load(params: LoadParams) {
...
val data = api.load(offset = params.key, size = params.loadSize)
return LoadResult.Page(
...
prevKey = ...
nextKey = data.size() + params.key
)
}
...
}
Note: You will still need to implement prepend if you want to support that, error handling, and getRefreshKey
I am trying to develop a client for Hacker News using this API, just for learning how Android works, as a personal project. I tried following some tutorials, but I am stuck at a certain point.
I want to retrieve the top N stories' titles, upvotes, etc. This would be done, using this api by:
Making a request to the api to retrieve the ID's of top posts (500 of them, to be exact)
For each ID, make a request to the api's posts endpoint to retrieve the details.
It seems that I am stuck on how to create N different network requests for the posts that I want, retrieving them and putting them on a List, then displaying them on my Fragment.
I am trying to follow an MVVM pattern, with Repositories. The relevant files are here:
NewsApi.kt:
interface NewsApi {
#GET("topstories.json")
fun getTopStories() : Single<List<Int>>
#GET("item/{id}")
fun getItem(#Path("id") id: String): Single<News>
}
MainRepository.kt (I):
interface MainRepository {
fun getTopStoryIDs(): Single<List<Int>>
fun getStory(storyId: Int): Single<News>
fun getTop20Stories(): Single<List<News>>
}
The News object is a simple data class with all the JSON fields that are returned from item/{id}, so I am omitting it.
Here is my Repository, the implementation:
class DefaultMainRepository #Inject constructor(
private val api: NewsApi
) : MainRepository {
override fun getTopStoryIDs(): Single<List<Int>> {
return api.getTopStories()
}
override fun getStory(storyId: Int): Single<News> {
return api.getItem(storyId.toString())
}
override fun getTop20Stories(): Single<List<News>> {
TODO("HOW?")
}
}
The top questions I have are:
How can I make chained API calls in this way, using Retrofit / RxJava? I have reviewed previous answers using flatMap, but in my case, using a List of Int's, I do not actually know how to do that correctly.
Is this the right way to go about this? Should I just ditch the architectural choices I've made, and try to think in a wholly new way?
Say I can complete getTop20Stories (which, as the name implies, should retrieve 20 of the news, using the result from getTopStoryIDs, first 20 elements for the time should do the trick), how would I be able to retrieve data from it? Who should do the honors of retrieving the response? VM? Fragment?
Thanks in advance.
Single as a return type in your case will not be the best option because it is designed to only maintain single stream. concatMap or flatMap on Single will not either because it will try to map list of items to another list of items which is not the case
here.
Instead you could use Observable or map your Single to Observable by using toObservable() with concatMapIterable operator which maps your single item to sequence of items.
I used concatMap operator instead of flatMap because it maintains order of the list items so your data won't be mixed up.
getTopStoryIDs()
.map { it.take(20) }
.toObservable()
.concatMapIterable { it }
.concatMapSingle { singleId ->
api.getItem(singleId)
}
.toList()
.subscribe { items ->
//do something with your items
}
This code will work but it's not the best solution because you will make 20 or more api calls which will hurt your network data and device battery so I wouldn't use it if it is not completely necessary.
If you have any questions fill free to ask :)
You where on the right track with FlatMap.
Something like this should do the trick:
getTopStoryIDs().flatMap { storyId -> getStory(storyId) }
sopuse I have a function in my Room DAO like this:
#Query("SELECT * FROM cached_tbl ORDER BY id")
fun getAll(): Flowable<List<Item>>
this returns all the items in the database
but I don't want this, I want the data to be paginated and be emitted in little chunks. I want the data to be loaded from the database on demand, for example, 100 items per page. is there a way to do that?
Jetpack has a library for this called Paging which you might be interested in. The good thing about using Room is that it also has Paging integration, so setup will go something like:
Dao
#Query("SELECT * FROM cached_tbl ORDER BY id")
fun getAll(): PagingSource<Int, Item>
ViewModel
val pager = Pager(PagingConfig(pageSize)) { dao.getAll() }
.cachedIn(viewModelScope)
Activity
lifecycleScope.launch {
pager.flow.collectLatest {
PagingDataAdapter.submtiData(it)
}
}
You can read quite a bit more here: https://developer.android.com/topic/libraries/architecture/paging/v3-overview, including how to setup a layered source, transformations and more.
I was wondering can I use paging3 library for API's that does not support 'page=RANDOM_NUMBER' in their queries? For example I have an API in which I can add custom query like 'number=50' and it will display 50 items as a result. I'm confused that I wouldn't be able to use that library for my API without page=RANDOM_NUMBER query. Can someone give me an answer?
Paging3 supports arbitrary key types (you define both the key and how it is used). In order to load data incrementally, you need to be able specify "load after ___", otherwise it's not possible to continue loading data after the initial load. If this is something that is tracked independently, say a cookie or session token, then you can try keeping maxSize set to unbounded, and just use any non-null value for nextKey.
Edit: Since you mentioned you are in the item-keyed scenario, where your next load is based on the last item you loaded, you might do something like this:
class MyPagingSource : PagingSource<String, Item>(
val api: NetworkApi,
) {
override suspend fun load(params: LoadParams): LoadResult<String, Item> {
try {
val result = withContext(Dispatchers.IO) {
api.loadPage(after_id = params.key)
}
return LoadResult.Page(
data = result.items,
nextKey = result.items.lastOrNull().id,
)
} catch (exception: IOException) {
return LoadResult.Error(exception)
}
}
}
Basically whatever value you pass to nextKey will get passed to LoadParams.key when user near the bottom of the loaded data, and in the case where there are no more items or you get an empty response from network (Due to being at the end of the list), you can return null for nextKey to tell Paging there is no more to load in that direction.
Note that I haven't covered prepend / prevKey, but if it is unsupported in your case you can just pass null.
If you don't support prepend, you won't be able to resume loading from the middle of the list, so you need to return null in getRefreshKey() which tells Paging what key to use to resume loading from a scroll position in case of config change, etc.
So I have a RecyclerView with infinite scrolling. I first do a network call to my API and get a first page of 20 items.
In my ViewModel (code below), I have an observable that triggers the network call in my repository using the page number.
So, when the user scrolls to the bottom, the page number is incremented, and it triggers another network request.
Here's the code in my ViewModel:
private val scheduleObservable = Transformations.switchMap(scheduleParams) { params: Map<String, Any> ->
ScheduleRepository.schedule(params["organizationId"] as String, params["page"] as Int)
}
// This is the method I call in my Fragment to fetch another page
fun fetchSchedule(organizationId: String, page: Int) {
val params = mapOf(
"organizationId" to organizationId,
"page" to page
)
scheduleParams.value = params
}
fun scheduleObservable() : LiveData<Resource<Items>> {
return scheduleObservable
}
In my fragment, I observe scheduleObservable, and when it emits data, I append them to my RecyclerView's adapter:
viewModel.scheduleObservable().observe(this, Observer {
it?.data?.let {
if (!isAppending) {
adapter.replaceData(it)
} else {
adapter.addData(it)
}
}
})
The problem with my current implementation is that, on configuration change, I rebind my observer, and the observable emits the last fetched data. In my case, it will emit the last fetched page only.
When coming back from a configuration change, I would want to have the full list of items fetched to this point so I can repopulate the adapter with these.
I'm wondering what's the best way to solve this. Should I have two observables? Should I create a list variable in my ViewModel to store all the items fetched and use that list for my adapter?
I checked android-architecture-components on GitHub, but it's usually overkill compared for my needs (no database, no paging library, etc) and I get lost since I am still trying to wrap my head around architecture components.