I'm looking into using the paging 3 library for handling paging on my android app. One small hitch is that every single example I find assumes I'd always provide a page number to the API when my company uses skip/take for our APIs as does a few other APIs I use that's not under my control.
I see that paging 2 had something but it looks to be depreciated so I'm curious what their solution is for paging 3?
By skip/take do you mean you have a item-keyed source which wants a number of items to load and an offset as inputs?
In Paging2 there were explicit classes for each type of key, but in Paging3 you control how the key is interpreted directly, so you can implement a PagingSource using the offset as the key.
Naively, just to show nextKey calculation:
class MyPagingSource : PagingSource<Int, Item>() {
override suspend fun load(params: LoadParams) {
...
val data = api.load(offset = params.key, size = params.loadSize)
return LoadResult.Page(
...
prevKey = ...
nextKey = data.size() + params.key
)
}
...
}
Note: You will still need to implement prepend if you want to support that, error handling, and getRefreshKey
Related
I am trying to develop a client for Hacker News using this API, just for learning how Android works, as a personal project. I tried following some tutorials, but I am stuck at a certain point.
I want to retrieve the top N stories' titles, upvotes, etc. This would be done, using this api by:
Making a request to the api to retrieve the ID's of top posts (500 of them, to be exact)
For each ID, make a request to the api's posts endpoint to retrieve the details.
It seems that I am stuck on how to create N different network requests for the posts that I want, retrieving them and putting them on a List, then displaying them on my Fragment.
I am trying to follow an MVVM pattern, with Repositories. The relevant files are here:
NewsApi.kt:
interface NewsApi {
#GET("topstories.json")
fun getTopStories() : Single<List<Int>>
#GET("item/{id}")
fun getItem(#Path("id") id: String): Single<News>
}
MainRepository.kt (I):
interface MainRepository {
fun getTopStoryIDs(): Single<List<Int>>
fun getStory(storyId: Int): Single<News>
fun getTop20Stories(): Single<List<News>>
}
The News object is a simple data class with all the JSON fields that are returned from item/{id}, so I am omitting it.
Here is my Repository, the implementation:
class DefaultMainRepository #Inject constructor(
private val api: NewsApi
) : MainRepository {
override fun getTopStoryIDs(): Single<List<Int>> {
return api.getTopStories()
}
override fun getStory(storyId: Int): Single<News> {
return api.getItem(storyId.toString())
}
override fun getTop20Stories(): Single<List<News>> {
TODO("HOW?")
}
}
The top questions I have are:
How can I make chained API calls in this way, using Retrofit / RxJava? I have reviewed previous answers using flatMap, but in my case, using a List of Int's, I do not actually know how to do that correctly.
Is this the right way to go about this? Should I just ditch the architectural choices I've made, and try to think in a wholly new way?
Say I can complete getTop20Stories (which, as the name implies, should retrieve 20 of the news, using the result from getTopStoryIDs, first 20 elements for the time should do the trick), how would I be able to retrieve data from it? Who should do the honors of retrieving the response? VM? Fragment?
Thanks in advance.
Single as a return type in your case will not be the best option because it is designed to only maintain single stream. concatMap or flatMap on Single will not either because it will try to map list of items to another list of items which is not the case
here.
Instead you could use Observable or map your Single to Observable by using toObservable() with concatMapIterable operator which maps your single item to sequence of items.
I used concatMap operator instead of flatMap because it maintains order of the list items so your data won't be mixed up.
getTopStoryIDs()
.map { it.take(20) }
.toObservable()
.concatMapIterable { it }
.concatMapSingle { singleId ->
api.getItem(singleId)
}
.toList()
.subscribe { items ->
//do something with your items
}
This code will work but it's not the best solution because you will make 20 or more api calls which will hurt your network data and device battery so I wouldn't use it if it is not completely necessary.
If you have any questions fill free to ask :)
You where on the right track with FlatMap.
Something like this should do the trick:
getTopStoryIDs().flatMap { storyId -> getStory(storyId) }
How can you tell if paging is working properly? All the examples I've looked at involve using retrofit apiservice which appears to be returning pages of data, but I'm pulling down a single rss feed and parsing it into a giant List<POJO>. I suspect that my PagingSource is loading the entire list into one page, but I'm not sure how to tell.
My list has near 1000 items, so I assume it'd be good practice to implement some kind of paging/DiffUtil. I'm playing around in this with jetpack compose usingandroidx.paging:paging-compose:1.0.0-alpha12 which probably complicates things.
Can anyone give me some pointers?
class RssListSource(): PagingSource<Int, RssItem>() {
override suspend fun load(params: LoadParams<Int>): LoadResult<Int, RssItem> {
return try {
val nextPage = params.key ?: 1
val rssList: List<RssItem> = RssFeedFetcher().fetchRss()
LoadResult.Page(
data = rssList,
prevKey = if (nextPage == 1) null else nextPage - 1,
nextKey = nextPage.plus(1)
)
} catch (e: Exception){
LoadResult.Error(e)
}
}
}
class MainActivityViewModel: ViewModel() {
val rss: Flow<PagingData<RssItem>> = Pager(PagingConfig(pageSize = 10)){
RssListSource() // returned to LazyPagingItems list in #Composable
}.flow.cachedIn(viewModelScope)
}
Your data still needs a way to fetch pages of data. I would expect your RssFeedFetcher to use the page information and return a page accordingly.
You are probably correct that you are currently returning all items at once.
There's two main strategies here:
Add a long enough delay() to load() such that you have enough time to scroll to the end of the list before new page loads
class RssListSource(): PagingSource<Int, RssItem>() {
override suspend fun load(params: LoadParams<Int>): LoadResult<Int, RssItem> {
delay(5000)
...
}
Observe changes to LazyPagingItems.loadState and look for either PREPEND or APPEND switching between Loading and NotLoading.
In order for you to be able to implement pagination with the paging library you need to use a paginated API, that means that in your example, you'd need a way to fetch the RSS in a paginated fashion. The Paging library won't be able to make use of pagination if your data source does not provide a way to query for paginated data, unfortunately.
You could achieve what you want by implementing a middleware that fetches the RSS feed and splits it into pages for you to query from the Android app.
EDIT: Another approach could be to have a background task (using the Android WorkManager) to download the RSS feed and save it in a Room Database, then use the Paging library to load pages off the database. Here's a summary of how to show paginated data from a Room database: https://developer.android.com/topic/libraries/architecture/paging/v3-network-db
I was wondering can I use paging3 library for API's that does not support 'page=RANDOM_NUMBER' in their queries? For example I have an API in which I can add custom query like 'number=50' and it will display 50 items as a result. I'm confused that I wouldn't be able to use that library for my API without page=RANDOM_NUMBER query. Can someone give me an answer?
Paging3 supports arbitrary key types (you define both the key and how it is used). In order to load data incrementally, you need to be able specify "load after ___", otherwise it's not possible to continue loading data after the initial load. If this is something that is tracked independently, say a cookie or session token, then you can try keeping maxSize set to unbounded, and just use any non-null value for nextKey.
Edit: Since you mentioned you are in the item-keyed scenario, where your next load is based on the last item you loaded, you might do something like this:
class MyPagingSource : PagingSource<String, Item>(
val api: NetworkApi,
) {
override suspend fun load(params: LoadParams): LoadResult<String, Item> {
try {
val result = withContext(Dispatchers.IO) {
api.loadPage(after_id = params.key)
}
return LoadResult.Page(
data = result.items,
nextKey = result.items.lastOrNull().id,
)
} catch (exception: IOException) {
return LoadResult.Error(exception)
}
}
}
Basically whatever value you pass to nextKey will get passed to LoadParams.key when user near the bottom of the loaded data, and in the case where there are no more items or you get an empty response from network (Due to being at the end of the list), you can return null for nextKey to tell Paging there is no more to load in that direction.
Note that I haven't covered prepend / prevKey, but if it is unsupported in your case you can just pass null.
If you don't support prepend, you won't be able to resume loading from the middle of the list, so you need to return null in getRefreshKey() which tells Paging what key to use to resume loading from a scroll position in case of config change, etc.
I am loading posts from network and for this i'm using Paging 3, but now problem is that my list items contains Like/Dislike button, suppose Like is clicked then how can i update data for that item without reloading whole dataset?
i have read this android-pagedlist-updates, but it seems that this is for older Paging 2 or 1, So what is perfect way to to achieve this in Paging 3
In Paging3 you still need to rely on PagingSource.invalidate to submit updates, this isn't so much about immutability, but more about having a single source of truth.
In general, the correct way to do this is to update the backing dataset and call invalidate, which will trigger a REFRESH + DiffUtil that shouldn't cause any UI changes, but guarantees that if that page is dropped and refetched, the loaded pages will still be up-to-date. The easiest way to do this is to use a PagingSource implementation that already has self-invalidation built-in, like the one provided by Room, and just update the corresponding row onClick of the like / dislike button.
There is an open bug tracking the work to support highly frequent, granular updates to the list with a Flow<>, which you can follow here if this is your use case: https://issuetracker.google.com/160232968
I overcome this challenge with below mechanism. Maintain the internal Hashmap to hold key and object, keep this internal hashmap inside your pagedlist adapter. As the list scrolls , you will add remote like/dislike into internal hashmap as initial status by using its something unique key, since the key is unique, you will not going to duplicate and then you refer this internal hashmap for your update UI.
onClick listener of like and dislike will update this internal hashmap. again internal hashmap is reference for UI update.
Solution is simple - collecting helpful data on another internal hashmap for later manipulation.
I found a work-around with which u can achieve this, giving some of the background of the code-base that I am working with:
I have a PagingDataAdapter (Pagination-3.0 adapter) as a recycler view adapter.
I have a viewmodel for fragment
I have a repository which returns flow of PaginationData
And exposing this flow as liveData to fragment
Code for repository:
override fun getPetsList(): Flow<PagingData<Pets>> {
return Pager(
config = PagingConfig(
pageSize = 15,
enablePlaceholders = false,
prefetchDistance = 4
),
pagingSourceFactory = {
PetDataSource(petService = petService)
}
).flow
}
Code for viewmodel:
// create a hashmap that stores the (key, value) pair for the object that have changed like (id:3, pet: fav=true .... )
viewModelScope.launch {
petRepository.getPetsList()
.cachedIn(viewModelScope)
.collect {
_petItems.value = it
}
}
Now the code for fragment where mapping and all the magic happens
viewModel.petItems.observe(viewLifecycleOwner) { pagingData ->
val updatedItemsHashMap = viewModel.updatedPetsMap
val updatedPagingData = pagingData.map { pet ->
if (updatedItemsHashMap.containsKey(pet.id))
updatedItemsHashMap.getValue(pet.id)
else
pet
}
viewLifecycleOwner.lifecycleScope.launch {
petAdapter.submitData(updatedPagingData)
}
}
So that is how you can achieve this, the crux is to do mapping of pagingData which is emitted from repository.
Things which won't work:
_petItems.value = PagingData.from(yourList)
This won't work because as per docs this is used for static list, and you would loose the pagination power that comes with pagination 3.0. So mapping pagingData seems the only way.
I need to map domain objects to UI objects and display using a live paged list.
I have tried to map LiveData<PagedList<X>> to LiveData<PagedList<Y>>, and map PositionalDataSource<X> to PositionalDataSource<Y>, but due to package private and private restrictions these both appear to be impossible without placing my code in android.arch.paging package and using reflection, or using a modified version of the paging lib.
Does anyone know of a way to do this without resorting to such undesirable methods?
(Note that this would not be an issue if the paging library API used interfaces instead of abstract base classes - that would allow wrapping any paged list/data source and add appropriate mappings.)
DataSource and DataSource.Factory have mapBy() and mapPageBy(), which could help you in this case. Just be careful cause these two will restrict the size of the "Y"-result-list.
If the size of the result differs from the original list's size then DataSource will throw an Exception.
For me following worked :
val dataSourceFactory =
cache.getDataSourceFactory(params)
.map {
convertXToY(it)
}
Paging 3 library PagingData mapping (RxPagingSource+RxJava2)
val pagingData: PagingData<X> = //TODO
pagingData.map { pagingData ->
pagingData.mapAsync { x ->
Single.just(Y(x))
}
}