Develop Reference

How to call different api resource from paging source or remote mediator Kotlin

Hey I want to call two different api for my Paging Library 3. I want to ask what is best suit for me to use Paging Source or Remote Mediator?. What is the use case of both? Can someone please explain me.
For 1st api call only for single time
#GET("/movie?min=20")
Above api call returns this response
data class movie(
var id: Int?,
var name: String?,
var items : List<Genre>?
}
Now for 2nd api call its loop to call again and again
#GET("/movie?count=20&&before={time}")
Above api call retrun this
data class movie(
var items : List<Genre>?
}
Genre
data class Genre(
var type: String?,
var date: String?,
var cast: String?
}
Genre have data in both api call. I tried to google this and found this Example. But inside this both api return same data. But in my case both returns little bit different. Also id, name is only used in UI component else list will go to adapter. But I didn't understand how to achieved this. I am new in Flow, it too difficult to understand, to be honest I am trying to learning CodeLab. Another important thing when 1st time api call, in which the last item contains date will send to 2nd api call in time parameter and then 2nd api last item date call again 2nd api, this will go in loop. So how can I track this again in loop condition. Third I want to update data at top of list, can we store data in memory than we can update value on that list? Thanks for advance. Sorry for my wrong english.
UPDATE
After #dlam suggestion, I tried to practice some code
MainActivity
class MainActivity : AppCompatActivity() {
private lateinit var binding: ActivityMainBinding
private val viewModel by viewModels<ActivityViewModel>()
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = ActivityMainBinding.inflate(layoutInflater)
setContentView(binding.root)
lifecycleScope.launchWhenCreated {
viewModel.getMovie().collectLatest {
// setupAdapter()
}
}
}
}
ActivityViewModel
class ActivityViewModel(app: Application) : AndroidViewModel(app) {
fun getMovie(): Flow<PagingData<Genre>> {
return Pager(
config = PagingConfig(
pageSize = 20
),
pagingSourceFactory = {
MultiRequestPagingSource(DataSource())
}
).flow
}
}
MultiRequestPagingSource
class MultiRequestPagingSource(private val dataSource: DataSource) : PagingSource<String, Genre>() {
override fun getRefreshKey(state: PagingState<String, Genre>): String? {
return state.anchorPosition?.let { anchorPosition ->
state.closestPageToPosition(anchorPosition)?.nextKey
}
}
override suspend fun load(params: LoadParams<String>): LoadResult<String, Genre> {
val key = params.key ?: ""
return try {
val data = when (params) {
is LoadParams.Refresh -> {
dataSource.fetchInitialMovie()
}
is LoadParams.Append -> {
dataSource.fetchMovieBefore(key)
}
is LoadParams.Prepend -> null
}
LoadResult.Page(
data = data.result,
prevKey = null,
nextKey = data?.nextKey,
)
} catch (exception: IOException) {
LoadResult.Error(exception)
}
}
}
I am getting error on data = data.result
Type mismatch.
Required:
List<TypeVariable(Value)>
Found:
ArrayDeque<Genre>?
DataSource
package com.example.multirequestpaging
class DataSource {
data class MovieResult(
val result: ArrayDeque<Genre>?,
val nextKey: String?
)
fun fetchInitialMovie(): MovieResult {
val response = ApiInterface.create().getMovieResponse(20)
return MovieResult(
addInArrayDeque(response),
response.items?.last()?.date
)
}
fun fetchMovieBefore(key: String): MovieResult {
val response = ApiInterface.create().getMovieResponseBefore(20, key)
return MovieResult(
addInArrayDeque(response),
response.items?.last()?.date
)
}
private fun addInArrayDeque(response: MovieResponse): ArrayDeque<Genre> {
val result: ArrayDeque<Genre> = ArrayDeque()
response.items?.forEach {
result.add(it)
}
return result
}
}
For Full code Project Link
1. I want to add an item to the top of the list. How can I use invalidate function? Sorry I didn't understand where I can use.
2. I want to use id,name in other place so how can i get those variable value in my activity class.
3. Is my code structure is good?. Do I need to improved, please give an example. It will also help beginner, who is learning Paging Library.
Thanks

PagingSource is the main driver for Paging, it's responsible for loading items that get displayed and represents the single source of truth of data.
RemoteMediator is for layered sources, it is essentially a callback which triggers when PagingSource runs out of data, so you can fetch from a secondary source. This is primarily useful in cases where you fetching from both DB + Network, where you want locally cached data to power Paging, and then use RemoteMediator as a callback to fetch more items into the cache from network.
In this scenario you have two APIs, but they both fetch from the same Network source, so you only need PagingSource here. If I'm understanding correctly, you essentially want to call the first API on initial load and the second API on subsequent prepend / append page loads, which you can check / switch on by the type of LoadParams you get. See the subtypes here: https://developer.android.com/reference/kotlin/androidx/paging/PagingSource.LoadParams

Android Paging 3: How to change parameters of RemoteMediator

I am struggling with the Paging 3 Library of Jetpack.
I setup
Retrofit for the network API calls
Room to store the retrieved data
A repository that exposes the Pager.flow (see code below)
A RemoteMediator to cache the network results in the room database
The PagingSource is created by Room.
I understand that the RemoteMediators responsibility is to fetch items from the network and persist them into the Room database. By doing so, we can use the Room database as single point of truth. Room can easily create the PagingSource for me as long as I am using Integers as nextPageKeys.
So far so good. Here is my ViewModel to retrieve a list of Sources:
private lateinit var _sources: Flow<PagingData<Source>>
val sources: Flow<PagingData<Source>>
get() = _sources
private fun fetchSources() = viewModelScope.launch {
_sources = sourcesRepository.getSources(
selectedRepositoryUuid,
selectedRef,
selectedPath
)
}
val sources is collected in the Fragment.
fetchSources() is called whenever one of the three parameters change (selectedRepositoryUuid, selectedRef or selectedPath)
Here is the Repository for the Paging call
fun getSources(repositoryUuid: String, refHash: String, path: String): Flow<PagingData<Source>> {
return Pager(
config = PagingConfig(50),
remoteMediator = SourcesRemoteMediator(repositoryUuid, refHash, path),
pagingSourceFactory = { sourcesDao.get(repositoryUuid, refHash, path) }
).flow
}
Now what I experience is that Repository.getSources is first called with correct parameters, the RemoteMediator and the PagingSource are created and all is good. But as soon as one of the 3 parameters change (let's say path), neither the RemoteMediator is recreated nor the PagingSource. All requests still try to fetch the original entries.
My question: How can I use the Paging 3 library here in cases where the paging content is dependent on dynamic variables?
If it helps to grasp my use-case: The RecyclerView is displaying a paged list of files and folders. As soon as the user clicks on a folder, the content of the RecyclerView should change to display the files of the clicked folder.
Update:
Thanks to the answer of dlam, the code now looks like this. The code is a simplification of the real code. I basically encapsulate all needed information in the SourceDescription class.:
ViewModel:
private val sourceDescription = MutableStateFlow(SourceDescription())
fun getSources() = sourceDescription.flatMapConcat { sourceDescription ->
// This is called only once. I expected this to be called whenever `sourceDescription` emits a new value...?
val project = sourceDescription.project
val path = sourceDescription.path
Pager(
config = PagingConfig(30),
remoteMediator = SourcesRemoteMediator(project, path),
pagingSourceFactory = { sourcesDao.get(project, path) }
).flow.cachedIn(viewModelScope)
}
fun setProject(project: String) {
viewModelScope.launch {
val defaultPath = Database.getDefaultPath(project)
val newSourceDescription = SourceDescription(project, defaultPath)
sourceDescription.emit(newSourceDescription)
}
}
In the UI, the User first selects a project, which is coming from the ProjectViewModel via LiveData. As soon as we have the project information, we set it in the SourcesViewModel using the setProject method from above.
Fragment:
override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
super.onViewCreated(view, savedInstanceState)
// load the list of sources
viewLifecycleOwner.lifecycleScope.launchWhenStarted {
sourcesViewModel.getSources().collectLatest { list ->
sourcesAdapter.submitData(list) // this is called only once in the beginning
}
}
projectsViewModel.projects.observe(viewLifecycleOwner, Observer { project ->
sourcesViewModel.setProject(project)
})
}

The overall output of Paging is a Flow<PagingData>, so typically mixing your signal (file path) into the flow via some flow-operation will work best. If you're able to model the path the user clicks on as a Flow<String>, something like this might work:
ViewModel.kt
class MyViewModel extends .. {
val pathFlow = MutableStateFlow<String>("/")
val pagingDataFlow = pathFlow.flatMapLatest { path ->
Pager(
remoteMediator = MyRemoteMediator(path)
...
).flow.cachedIn(..)
}
}
RemoteMediator.kt
class MyRemoteMediator extends RemoteMediator<..> {
override suspend fun load(..): .. {
// If path changed or simply on whenever loadType == REFRESH, clear db.
}
}
The other strategy if you have everything loaded is to pass the path directly into PagingSource, but it sounds like your data is coming from network so RemoteMediator approach is probably best here.

Caching is not working in Android Paging 3

I have implemented application using codelabs tutorial for new Paging 3 library, which was release week ago.
The problem is application is not working in offline mode. It does not retrieve data from Room database.
Tutorial Repo link :- https://github.com/googlecodelabs/android-paging
Code:-
RepoDao.kt
#Dao
interface RepoDao {
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insertAll(repos: List<Repo>)
#Query("SELECT * FROM repos WHERE " +
"name LIKE :queryString OR description LIKE :queryString " +
"ORDER BY stars DESC, name ASC")
fun reposByName(queryString: String): PagingSource<Int, Repo>
#Query("DELETE FROM repos")
suspend fun clearRepos()
}
GithubRepository.kt
class GithubRepository(
private val service: GithubService,
private val database: RepoDatabase
) {
fun getSearchResultStream(query: String): Flow<PagingData<Repo>> {
val dbQuery = "%${query.replace(' ', '%')}%"
val pagingSourceFactory = { database.reposDao().reposByName(dbQuery) }
return Pager(
config = PagingConfig(pageSize = NETWORK_PAGE_SIZE),
remoteMediator = GithubRemoteMediator(
query,
service,
database
),
pagingSourceFactory = pagingSourceFactory
).flow
}
companion object {
private const val NETWORK_PAGE_SIZE = 50
}
}
SearchRepositoriesViewModel.kt
#ExperimentalCoroutinesApi
class SearchRepositoriesViewModel(private val repository: GithubRepository) : ViewModel() {
private var currentQueryValue: String? = null
private var currentSearchResult: Flow<PagingData<Repo>>? = null
fun searchRepo(queryString: String): Flow<PagingData<Repo>> {
val lastResult = currentSearchResult
if (queryString == currentQueryValue && lastResult != null) {
return lastResult
}
currentQueryValue = queryString
val newResult: Flow<PagingData<Repo>> = repository.getSearchResultStream(queryString).cachedIn(viewModelScope)
currentSearchResult = newResult
return newResult
}
}
SearchRepositoriesActivity.kt
#ExperimentalCoroutinesApi
class SearchRepositoriesActivity : AppCompatActivity() {
.....
private lateinit var viewModel: SearchRepositoriesViewModel
private val adapter = ReposAdapter()
private var searchJob: Job? = null
// this is where adapter get flow data from viewModel
// initially this is called with **Android** as a query
private fun search(query: String) {
searchJob?.cancel()
searchJob = lifecycleScope.launch {
viewModel.searchRepo(query).collectLatest {
adapter.submitData(it)
}
}
}
.....
}
Output:- It is just showing the empty recyclerview when application is open in offline mode.

If you're able to share your code or how you reached that conclusion I could probably help pinpoint the problem a bit better, but the codelab does load data from Room on the branch: step13-19_network_and_database
There are two components here:
PagingSource: Provided by Room by declaring a #Query with a PagingSource return type, will create a PagingSource that loads from Room. This function is called in the pagingSourceFactory lambda in Pager which expects a new instance each call.
RemoteMediator: load() called on boundary conditions where the local cache is out of data, this will fetch from network and store in the Room db, which automatically propagates updates to PagingSource implementation generated by Room.
One other issue you might be seeing could be related to loadStateListener/Flow, essentially the codelab shows an error state by checking for CombinedLoadStates.refresh, but this always defers to the RemoteMediator's load state when available and if you want to show the locally cached data, even when RemoteMediator errors out, you'll need to disable hiding of the list in that case.
Note that you can access individual LoadState with CombinedLoadStates.source or CombinedLoadStates.mediator.
Hopefully this is enough to help you, but it's hard to guess your issue without some more concrete example / information about what you're seeing.
Edit: While the above are still good things to check for, it looks like there's an underlying issue with the library that I'm chasing down here: https://android-review.googlesource.com/c/platform/frameworks/support/+/1341068
Edit2: This is fixed now and will be released with alpha02.

Inserting a pagedlist into room db

I'm implementing paging library with room db and coroutines , the part of loading data and handling all paging library methods is done , now i'm facing an issue which is to save the data into room db , i'm actually getting a pagedlist response from api and updating adapter with it ,and since im getting pagedlist ,i need to insert it into room db so that i can show data later in offline mode but i tried so and it didn't work , no data is showing , not sure if it is the right way to do it .
This is my response from api
mainViewModel.getAll(query,Utils().API_KEY,1).observe(viewLifecycleOwner, Observer {
newsAdapter.submitList(it) //pagedlist
recyclerView.adapter = newsAdapter
})
here is my dao
#Dao
interface NewsDao {
#Query("SELECT * FROM news_table")
fun restoreNews() : androidx.paging.DataSource.Factory<Int,AllNewsModel>
#Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun storeNews(pagedList: PagedList<AllNewsModel>)
}
what i want to achieve is basically inserting the data of the pagedlist into my db so that i can use it to update my adapter in offline mode , thank you

I believe the best approach is to do what you want to do in your repository/boundarycallback class.
You can check this repo from google.
https://github.com/googlecodelabs/android-paging
It is a repo from a google codelab, that you can find it here
https://codelabs.developers.google.com/codelabs/android-paging/#1
You can see that class inside boundarycallback that does the job at the same time it gets the answer from the Api:
private fun requestAndSaveData(query: String) {
if (isRequestInProgress) return
isRequestInProgress = true
searchRepos(service, query, lastRequestedPage, NETWORK_PAGE_SIZE, { repos ->
cache.insert(repos) {
lastRequestedPage++
isRequestInProgress = false
}
}, { error ->
_networkErrors.postValue(error)
isRequestInProgress = false
})
}

Paging Library: How to reload portion of data on demand?

I use Paging Library to paginate my data set. What I'm trying to do is to refresh the RecyclerView after data in my database has been changed.
I have this LiveData:
val listItems: LiveData<PagedList<Snapshot>> = object : LivePagedListProvider<Long, Snapshot>() {
override fun createDataSource() = SnapshotsDataSource()
}.create(null, PagedList.Config.Builder()
.setPageSize(PAGE_SIZE)
.setInitialLoadSizeHint(PAGE_SIZE)
.setEnablePlaceholders(false)
.build()
)
And the DataSource:
class SnapshotsDataSource : KeyedDataSource<Long, Snapshot>(), KodeinGlobalAware {
val db: FirebaseDb = instance()
override fun getKey(item: Snapshot): Long = item.timestamp
override fun loadInitial(pageSize: Int): List<Snapshot> {
val result = db.getSnapshotsTail(pageSize)
return result
}
override fun loadAfter(key: Long, pageSize: Int): List<Snapshot> {
val result = db.getSnapshotsTail(key, pageSize)
return result.subList(1, result.size)
}
override fun loadBefore(key: Long, pageSize: Int): List<Snapshot> {
return emptyList()
}
}
The Adapter is straight forward, so i omit it here.
I've tried to do this when database is modified:
fun reload(position) {
listItems.value!!.loadAround(position)
}
but it didn't help.

try to call listItems.value!!.datasource.invalidate()
not directly DataSource#invalidate()

I am having the same issue with a custom Firestore based DataSource. The only way to load a portion of the data without invalidating all of the data and having the UI flash / reload seems to be via integrating with Google's Room ORM library. Unfortunately, this will cache my data twice, once with Firestore, and again with Room which is unnecessary.
See the documentation under Consider How Content Updates Work. The only way to have realtime updates is via implementing Room with the PagedList: If you're loading data directly from a Room database updates get pushed to your app's UI automatically.

It's not possible. You can invalidate the whole list only: datasource.invalidate().