Develop Reference

Paging 3 - Use Network data as primary source and local data as addition

I want to load network data first and combine it with data from my local room db.
So in that way, I would always present the newest state of the backend to my users (since its always requesting the network first) and update my ui when there are some local db updates.
A use-case example:
1. A UserActivity shows all created posts of a given user (through paging3).
Data has been fetched from network.
2. User starts a PostActivity where he/she likes the post (post was also visible in step 1 (UserActivity)).
This action will trigger a rest call where the result (updated post) will be stored to the local db.
3. User comes back to the UserActivity and sees that the post is marked liked now.
The UI update was done automatically, since the local Db changed in step 2.
Here are some of those things I've tried
Using Rooms PagingSource implementation (LimitOffsetPagingSource)
Initially I thought the Rooms PagingSource + RemoteMediator is exactly what I needed (see this codelab example for implemantation).
The problem with that was though, that after the local database was initially filled with entities (through network), it will be used as the start source for further inital fetches in pagings.That means that changes in my backend would only be noticed by the UI when the user scrolls to the end of the list. Because only then, the LimitOffsetPagingSource will request the RemoteMediator again for further entities, since it is out of items.
A workaround (like used in the codelab) was to delete all entities before paging again.
// clear all tables in the database
if (loadType == LoadType.REFRESH) {
repoDatabase.remoteKeysDao().clearRemoteKeys()
repoDatabase.reposDao().clearRepos()
}
But this would break all my other views which also depended on these entity models.
Example:
1. TopPostsActivity shows top posts via paging
2. Clicking on a user which starts the UserActivity (from the first example)
3. UserActivity starts paging. This will initially delete all entities, so that we can fetch all new elements first.
User doesn't scroll much, so that not all entities are stored again.
4. User goes back to TopPostsActivity, where it will see an empty list. Reason for that is that the UserActivity deleted all depending posts before.
Only an additional refresh call can show the desired posts again.
Merging local and remote PagingData manually
In this try, I used a custom PagingSource for fetching from network in my Pager (just the basics like here Defining a PagingSource).
Since Room can also provide flows which notifies when there are some changes in the dataset, I transformed the data to a PagingData and submitted it to my PagingAdapter.
//Room DAO
#Query("SELECT * FROM post WHERE author_id = :authorId ORDER BY created_at DESC")
fun findAllByAuthor(authorId: String): Flow<PostWithAuthor>
The merge code
...
val localDbFlow = viewModel.repo.dao.findAllByAuthor(viewModel.author.id)
.map { PagingData.from(it) }
val networkFlow = viewModel.pager.flow
val mergedFlow = merge(localDbFlow, networkFlow)
flowAndCollectLatest(mergedFlow, Lifecycle.State.CREATED) {
adapter.submitData(it)
}
This also kinda worked, but the problem was that I am not getting any LoadState changes anymore due to the function PagingData.from(list).
/**
* Create a [PagingData] that immediately displays a static list of items when submitted to
* [AsyncPagingDataAdapter][androidx.paging.AsyncPagingDataAdapter].
*
* #param data Static list of [T] to display.
*/
#JvmStatic // Convenience for Java developers.
public fun <T : Any> from(data: List<T>): PagingData<T> = PagingData(
flow = flowOf(
PageEvent.Insert.Refresh(
pages = listOf(TransformablePage(originalPageOffset = 0, data = data)),
placeholdersBefore = 0,
placeholdersAfter = 0,
sourceLoadStates = LoadStates(
refresh = LoadState.NotLoading.Incomplete,
prepend = LoadState.NotLoading.Complete,
append = LoadState.NotLoading.Complete
)
)
),
receiver = NOOP_RECEIVER
)
So I can't finish any refresh ui actions like with SwipeRefreshLayout, since I'm not getting anything back through the LoadStateListener.
Can anyone please help me with this problem? Is there anything I'm overseeing? This issue really starts depressing me and steals the fun I have to code :'(
I can't create a replacement for the automatically generated LimitOffsetPagingSource (since It really seems critical for one without the required qualification).

Should i save my User Details in Viewmodel as well as SharedPrefernces?

Should i update my User Details in Viewmodel as well as SharedPrefernces?
For example, i have fields like
name
age
currentSalary
organisation
+8 more
I have few doubts now:
Am i supposed to create LiveData of each of these fields?
Also, i have to save them in Sharedprefernces too. So doesn't this feel redundant? First, saving it in Viewmodel and then saving it in Sharedprefernces.
When i move from screen 1 to screen 2, should i fetch user's name from ViewModel or the api?
MainViewModel.kt Sample Code
private val _experience = MutableLiveData<String>()
val experience : LiveData<String>
get() = _experience
private val _name = MutableLiveData<String>()
val name : LiveData<String>
get() = _name
private val _isLoggedIn = MutableLiveData<Boolean>()
val isLoggedIn : LiveData<Boolean>
get() = _isLoggedIn
fun setName(name: String){
_name.value = name
}
fun setExperience(exp: String){
_experience.value = exp
}
fun logIn(){
_isLoggedIn.value = true
}
MyFragment
binding.btnSubmit.setOnClickListener {
val name = binding.etName.text.toString()
val email = binding.etEmail.text.toString()
val age = binding.etAge.text.toString()
val org = binding.etOrg.text.toString()
//saving in viewmodel
mainViewModel.setName(name)
mainViewModel.setCurrentOrganisation(org)
mainViewModel.setEmail(email)
mainViewModel.setAge(Integer.valueOf(age))
//saving in sharedpreference
editor.putInt("age", Integer.valueOf(age))
editor.putString("name", name)
editor.putString("email", email)
editor.putString("org", org)
}
To me, My fragment looks a lot of lines of codes. I don't know if i am using the right approach.

The ViewModel is meant to sit between your View layer (the UI) and the Data layer (the core app functionality, stored data etc). A ViewModel acts as a go-between, passing data to the UI for display, and translating UI events to function calls in the app.
The ViewModel's state is transient - in Android it sticks around long enough to survive things like Activity recreation. If you use the SavedStateHandle component, you can store its running state so it can be rebuilt if the app is destroyed in the background - but this explicitly won't survive the app being closed and restarted. The ViewModel isn't about persisting data, just about the current, temporary state of things.
So actually storing your data is part of the data layer. That's where the SharedPreferences comes in - but you could be storing (and reading) data using all kinds of storage, even across a network. The ViewModel's role is to access that data, and expose it to the UI somehow - possibly even transforming that data into a more suitable form for the UI to consume.
So while you might have duplication going on, there's a reason for it. SharedPreferences is there to actually store and persist the data. LiveData is just there for the UI to see what it should be displaying. They may or may not be the same thing!
Have a look at this Android guide on designing your app architecture - it goes into the theory behind how you organise things, and how the data flow works. You don't necessarily need to follow all of it, but the broad strokes are good to know - that way if you want to deviate from that for simplicity in a particular situation, you'll know why you're doing it and what compromises you're making (and whether they matter in this case).
For your data update... there's two ways you can go about it. One is to make the ViewModel (gonna say VM for brevity) update the data layer, and then have the data layer push new data to the VM, which the VM displays in its LiveData. This is the kind of thing you do when you're using observable queries with a database, where updates to a table push new data to the observer.
All the VM has to worry about is pushing data to the data layer (e.g. calling a delete item function). When the data changes, it's pushed to the observing VM, which just displays the data as usual (e.g. setting it on a LiveData), which causes the UI to get an update and display the new state... So instead of the VM getting the delete event, and having to worry about updating its own internal state, it just passes the delete request to the data layer. Then the new state arrives later, and the VM just uses that - a new list, whatever. The data layer tells the view model what to display, in the same way the VM tells the UI what to display.
(It might not be worth the effort writing a thing that updates the SharedPreferences and then tells the VM to display that data, so you could just do all this in the VM as a kind of combination of VM and data layer - but it helps to know why you're doing it, what it's a shortcut for, y'know?)
As for the "lots of LiveData objects" bit, have a look at this section on UI State. Basically, the officially recommended approach is that the the UI's state as a whole is pushed by the VM. So instead of separate LiveData objects for each property (name, age etc) you'd have a single object that contains that data, and a LiveData that pushes that. Whenever something about that data changes, you push a new instance of that data object. (Data classes can help here, with their copy functions that let you change specific values and keep the rest)
So your UI just observes that one state LiveData, and wires it up to the UI components - your TextViews, CheckBoxes etc. The approach they're talking about isn't just data - it's also UI state, which may not be what you're storing in your actual data layer (e.g. if a particular section is expanded, or if some list items are checked for a potential delete operation). Exactly how much you want to encapsulate in one object is up to you - but pushing your actual data in a single data structure isn't a bad idea!
Sorry that was a bit long, but hopefully it helps a bit

Android architecture LiveData and Repositories

I am converting my application to room database and try to follow the google architecture best practices based on "Room with a View".
I am having trouble to understand the repository in terms of clean architecture.
The Words database example contains only one table and one view using it, making it a simple HelloWorld example. But lets start with that.
There is a view which displays a list of words. Thus all words need to be read from the database and displayed.
So we have a MainActivity and a Database to connect.
Entity Word
WordDao to access DB
WordViewModel: To separate the activity lifecycle from the data lifecycle a ViewModel is used.
WordRepository: Since the data maybe kept in a database or the cloud or whatever the repository is introduced to handle decision, where data comes from.
Activity with the View
It would be nice if the view is updated when the data changes, so LiveData is used.
This in turn means, the repository is providing the LiveData for the full table:
// LiveData gives us updated words when they change.
val allWords: LiveData<List<Word>>
This is all fine for a single view.
Now to my questions on expanding this concept.
Let us assume, the word table has two columns "word" and "last_updated" as time string.
For easier comparison the time string needs to be converted to milliseconds, so I have a function.
Question: Where to put the fun queryMaxServerDateMS() to get the max(last_updated)?
/**
* #return Highest server date in table in milliseconds or 1 on empty/error.
*/
fun queryMaxServerDateMS(): Long {
val maxDateTime = wordDao.queryMaxServerDate()
var timeMS: Long = 0
if (maxDateTime != null) {
timeMS = parseDateToMillisOrZero_UTC(maxDateTime)
}
return if (timeMS <= 0) 1 else timeMS
}
For me it would be natural to put this into the WordRepository.
Second requirement: Background job to update the word list in the database.
Suppose I now want a Background Job scheduled on a regular basis which checks the server, if new entries were made and downloads them to the database. The app may not be open.
This question just relays to the question of the above queryMaxServerDateMS.
The job will basically check first, if a new entry was made by asking the server if an entry exists which is newer then the max known entry.
So I would need to get a new class WordRepository, do my query, get max last_update and ask the server.
BUT: I do not need the LiveData in the background job and when val repositoy = WordRepository the full table is read, which is needless and time-, memory and batteryconsuming.
I also can think of a number of different fragments that would require some data of the word table, but never the full data, think of a product detail screen which lists one product.
So I can move it out to another Repository or DbHelper however you want to call it.
But in the end I wonder, if I use LiveData, which requires the View, ViewModel and Repository to be closely coupled together:
Question: Do I need a repository for every activity/fragment instead of having a repository for every table which would be much more logical?

Yes, with your current architecture you should put it in the Repository.
No, you don't need a repository for every activity/fragment. Preferably, 1 repository should be created for 1 entity. You can have a UseCase for every ViewModel.
In Clean architecture there's a concept of UseCase / Interactor, that can contain business logic, and in Android it can act as an additional layer between ViewModel and Repository, you can create some UseCase class for your function queryMaxServerDateMS(), put it there and call it from any ViewModel you need.
Also you can get your LiveData value synchronously, by calling getValue().

You do not need repository for each activity or fragment. To answer your question about getting max server time - when you load words from db you pretty much have access to entire table. That means you can either do that computation yourself to decide which is the latest word that's added or you can delegate that work to room by adding another query in dao and access it in your repo. I'd prefer latter just for the simplicity of it.
To answer your question about using repo across different activities or fragment - room caches your computations so that they are available for use across different users of your repo (and eventually dao). This means if you have already computed the max server time in one activity and used it there, other lifecycle owners can use that computed result as far as the table has not been altered (there might be other conditions as well)
To summarize you're right about having repository for tables as opposed to activities or fragments

Paging Library with custom DataSource not updating row on Room update

I have been implementing the new Paging Library with a RecyclerView with an app built on top of the Architecture Components.
The data to fill the list is obtained from the Room database. In fact, it is fetched from the network, stored on the local database and provided to the list.
In order to provide the necessary data to build the list, I have implemented my own custom PageKeyedDataSource. Everything works as expected except for one little detail. Once the list is displayed, if any change occurs to the data of a list's row element, it is not automatically updated. So, if for example my list is showing a list of items which have a field name, and suddenly, this field is updated in the local Room database for a certain row item, the list does not update the row UI automatically.
This behaviour only happens when using a custom DataSource unlike when the DataSource is obtained automatically from the DAO, by returning a DataSource Factory directly. However, I need to implement a custom DataSource.
I know it could be updated by calling the invalidate() method on the DataSource to rebuild the updated list. However, if the app is showing 2 lists at a time (half screen each for example), and this item appears in both lists, it would be needed to call invalidate() for both lists separately.
I have thought with a solution in which, instead of using an instance of the item's class to fill each ViewHolder, it uses a LiveData wrapped version of it, to make each row observe for changes on its own item and update that row UI when necessary. Nevertheless, I see some downsides on this approach:
A LifeCycleOwner (such as the Fragment containing the RecyclerView for example) must be passed to the PagedListAdapter and then forward it to the ViewHolder in order to observe the LiveData wrapped item.
A new observer will be registered for each list's new row, so I do not know at all if it has an excessive computational and memory cost, considering it would be done for every list in the app, which has a lot of lists in it.
As the LifeCycleOwner observing the LiveData wrapped item would be, for example, the Fragment containing the RecyclerView, instead of the ViewHolder itself, the observer will be notified every time a change on that item occurs, even if the row containing that item is not even visible at that moment because the list has been scrolled, which seems to me like a waste of resources that could increase the computational cost unnecessarily.
I do not know at all if, even considering those downsides, it could seem like a decent approach or, maybe, if any of you know any other cleaner and better way to manage it.
Thank you in advance.

Quite some time since last checked this question, but for anyone interested, here is the cause of my issue + a library I made to observe LiveData properly from a ViewHolder (to avoid having to use the workaround explained in the question).
My specific issue was due to a bad use of Kotlin's Data Classes. When using them, it is important to note that (as explained in the docs), the toString(), equals(), hashCode() and copy() will only take into account all those properties declared in the class' constructor, ignoring those declared in the class' body. A simple example:
data class MyClass1(val prop: Int, val name: String) {}
data class MyClass2(val prop: Int) {
var name: String = ""
}
fun main() {
val a = MyClass1(1, "a")
val b = MyClass1(1, "b")
println(a == b) //False :) -> a.name != b.name
val c = MyClass2(2)
c.name = "c"
val d = MyClass2(2)
d.name = "d"
println(c == d) //True!! :O -> But c.name != d.name
}
This is specially important when implementing the PagedListAdapter's DiffCallback, as if we are in a example's MyClass2 like scenario, no matter how many times we update the name field in our Room database, as the DiffCallback's areContentsTheSame() method is probably always going to return true, making the list never update on that change.
If the reason explained above is not the reason of your issue, or you just want to be able to observe LiveData instances properly from a ViewHolder, I developed a small library which provides a Lifecycle to any ViewHolder, making it able to observe LiveData instances the proper way (instead of having to use the workaround explained in the question).
https://github.com/Sarquella/LifecycleCells

Create single onComplete for dynamic list of Completables

I'm using this library for wrapping Firebase transactions with RxJava. I'm new to RxJava, so this is mainly a question regarding how to use it.
Scenario: There is a many-to-many relationship between Persons and Labels. A Person can have multiple Labels, and a Label can be given to many Persons. When a Person is created, I must:
add them to the list of Persons
update each Label given to them to allow for querying all Persons that belong to a specific label
I have a list of Labels I want to write to my Firebase database.
List<Label> labels; // Let's assume it's been instantiated and added to
I want to write each of these to the DB:
FirebaseDatabase database = FirebaseDatabase.getInstance();
DatabaseReference peopleRef = database.getReference().child("people");
DatabaseReference labelsRef = database.getReference().child("labels");
int newPersonId = peopleRef.push().getKey();
I can do this easily if I don't care about whether the calls are successful.
// Let's assume I already saved the Person to the DB
for (Label label : labels){
// For each label, index the Person saved (Looks like 'personId: true')
labelsRef.child(label).child(newPersonId).setValue(true);
}
But what if I do care about the result? If I want to react to all Labels being updated (like navigate away from the current Activity), I need to know if they've all been updated successfully.
RxFirebase is implemented such that setting a value in the DB returns a Completable. I essentially want to zip together n number of Completables and do something only when they succeed or fail.
So far, I can do this if I only want to update one Label, but I want to update n Labels.
The following code snippet chains 2 Completables together, but only saves 1 Label
RxFirebaseDatabase.setValue(peopleRef.child(newPersonId), person) // Save the Person
.andThen(RxFirebaseDatabase.setValue(labelsRef.child(label).child(newPersonId), true)) // I can index 1 Label, and this returns a Completable
How would I do this? If you know Firebase well enough, is this even the right way to be saving a List of items?

If I understood your main question correctly, you have a collection of Completable and you need to subscribe to them as one.
The way to solve this is using the Completable.concat or Completable.merge operators.
Completable.concat: Returns a Completable which completes only when all sources complete, one after another.
Completable.merge: Returns a Completable instance that subscribes to all sources at once and completes only when all source Completables complete or one of them emits an error.
Example:
List<Completable> tasks; // initialized elsewhere
Completable
.concat(tasks)
.subscribe(
() -> Log.d(TAG, "All successful"),
throwable -> Log.w(TAG, "One or more failed"))
About your second question, I don't know Firebase well enough.
Update: to obtain the List<Completable> you can do something similar to this:
List<Completable> tasks = new ArrayList<>();
for ( ... ) {
tasks.add(RxFirebaseDatabase.setValue(peopleRef.child(newPersonId), person));
}
Completable.concat(tasks).etc