Lets say i have a list of Objects (Accounts) in my RoomDatabase. And when the application starts i request the Accounts from Server in order to Update the Accounts Table.
So i query the database, getting a
current: List<Account> which is the existing accounts in the database
From the Server we have the news: List<Account> list which is the updated Accounts
I want to find
The elements from the news that exist in the current, in order to update them
The elements from the news that don't exist in the current, in order to insert them
The elements from the current that don't exist in the news, in order to delete them
Is there any extension function that can do that Job with an elegant and fast way, except the obvious that is to do one step at a time with the already known technique? (Loop in each list every time)
You could use partition I suppose
val current = listOf("hey", "hi", "wow")
val news = listOf("hi", "hey", "sup")
fun main() {
val (retain, discard) = current.partition { it in news }
val insert = news.filterNot { it in current }
println("Retain: $retain\nDiscard: $discard\nInsert: $insert")
}
>> Retain: [hey, hi]
>> Discard: [wow]
>> Insert: [sup]
"Elegant and fast" is subjective - is this less elegant?
val retain = current.filter { it in news }
val discard = current.filter { it !in news }
val insert = news.filter { it !in current }
it's arguably neater and easier to read - three conditions, described the exact same way.
Faster? Well that depends on the size of your lists, and you'd have to benchmark the solutions to see how much difference it makes (and if it's worth prioritising it over readability). Converting the lists to sets would give you a performance boost for the in checks, but the conversion itself might be slower than the gains in lookup speed
According on your answer - the most efficient way in your case is just drop your previous accounts from db and write new.
Your backend data always a priority.
Instead of many data manipulations(get from db, compare lists, insert and delete to new list), do just
delete old accounts with delete method in your room dao.
insert new accounts from your backend using insert method in dao.
Extension for this with coroutines:
suspend fun List<Account>.insertActual() = withContext(IO) {
yourDao.deleteAccounts()
yourDao.insertAccounts(this)
}
Related
My goal is to effectively get a list of children (ordered* and indexed**) with the lowest number of data transfer.
* ordered: ordered by points for each user / database child
** indexed: 2 or less ranks behind/after the current user [A specific child] (further elaborated below)
My database structure is as follows:-
I basically want to get the first 3 users ordered by points (simple):-
val usersRef = FirebaseDatabase.getInstance(DB_LINK).getReference("users").orderByChild("points")
usersRef.limitToFirst(3).addValueEventListener(
object : ValueEventListener {
override fun onDataChange(snapshot: DataSnapshot) {
for (ds in snapshot.children) {
val points: String = snapshot.child("points").getValue(String::class.java)!!
val firstName: String = snapshot.child("firstName").getValue(String::class.java) ?: ""
val uid: String = snapshot.key!!
// Update View
}
}
override fun onCancelled(error: DatabaseError) {}
}
)
Then, provided that the currently logged in user isn't one of the first three, I want to get his rank (order according to points in the whole db), 2 users' before him, and 2 users' after him without querying the whole database (it's a user database that can get up to 50K unique users) because querying the whole database is a really expensive client-side task.
I checked firebase data filtering page but found nothing useful about limiting results according to a certain child.
This answer doesn't satisfy my needs, as it loops over the whole database (in my case, 50K records). I need an effective method as I need to really save these firebase bills.
Moreover, I check this answer but it didn't meet my needs because it still queries the whole database, meaning it is not effective and will be billed for each node before the current user. (Maybe he is number 40,000 in the db, so I shouldn't query the whole db each time to get his rank and get billed for 39,999 reads)
I searched for a way to somehow use booleans to filter queries but again found nothing useful. Here is my not-effective code:-
// Gets all children.
usersRef.addValueEventListener(
object : ValueEventListener {
override fun onDataChange(snapshot: DataSnapshot) {
for (ds in snapshot.children) {
val points: String = snapshot.child("points").getValue(String::class.java)!!
val firstName: String = snapshot.child("firstName").getValue(String::class.java) ?: ""
val uid: String = snapshot.key!!
// Update View only if user is `2 <= usersRank - theirRank <= -2`
}
}
override fun onCancelled(error: DatabaseError) {}
}
)
I want to achieve something like this:- (Styling already done, logic remaining)
Is there a way to achieve this? Any alternatives?
EDIT: I found out that firestore offers aggregation queries that may help in this situation. Doing more research to further narrow down the costs.
This operation is not available on a Firebase Realtime Database. A better option would be Firestore.
Why?
Well, A fire-store database can give you the count of objects in a certain query. This is a new feature added by firebase. You basically type the query you want, then add .count() before .get(); that way it'll return the count of objects only. This is called aggregation queries. Learn more about them here.
Cloud Functions - Why isn't it appropriate here?
Using a Cloud Function for aggregations avoids some of the issues with client-side transactions, but comes with a different set of limitations:
Cost - Each rating added will cause a Cloud Function invocation, which may increase your costs. For more information, see the Cloud Functions pricing page.
Latency - By offloading the aggregation work to a Cloud Function, your app will not see updated data until the Cloud Function has finished executing and the client has been notified of the new data. Depending on the speed of your Cloud Function, this could take longer than executing the transaction locally.
Write rates - this solution may not work for frequently updated aggregations because Cloud Firestore documents can only be updated at most once per second. Additionally, If a transaction reads a document that was modified outside of the transaction, it retries a finite number of times and then fails.
Combining this with other methods
Now that you're using COUNT() for this system, there is one more method to help further narrow down the costs. That is Periodic Updates.
Periodic Updates
Who would care about a live ranking of all users? You can make the leaderboard update each minute, hour, or day. For example, stack overflow's leaderboard is updated once a day!
This approach would really work for any number of players and any write rate. However, you might need to adjust the frequency though as you grow depending on your willingness to pay.
Costs Side
For each normal read, you are charged for one read. Very simple. However, for one count, you're charged for 0.001 reads (meaning 1000 counts = 1 read). For more information about costs, check this article by firebase.
Final Thoughts
To connect everything up, we shall now apply this on our problem. Firstly, we'll need to keep the first portion of the code as it is. (The portion that grabs the first 3 users), though with some changes to port it to firebase.
NOTICE: Don't forget to setup a composite index because we're ordering by multiple fields at the same time.
val top3 = HashMap<Int, HashMap<String, String>>()
Firebase.firestore.collection("users").orderBy("points", Query.Direction.DESCENDING)
.orderBy("firstName", Query.Direction.ASCENDING)
.get().addOnSuccessListener {
for ((index, doc) in it.documents.withIndex()) {
val firstName = doc.getString("firstName")!!
val points = doc.getString("points")!!
top3[index+1] = hashMapOf("first" to firstName, "points" to points, "uid" to doc.id)
}
}
More about ordering and limiting here.
Then, we'll need to implement the COUNT() feature.
Firebase.firestore.collection("users")
.whereGreaterThan("points", UserProfile.getInstance()!!.getProfile().points)
.count().get(AggregateSource.SERVER).addOnSuccessListener {
println("Your rank is: ${it.count+1}")
}
Basically what I did here was:-
Selecting the Collection
Ordered Ascending by first name so no duplicate ranks.
Count them, and pass onto the function.
The final step is just updating the hash map top3 and rank of user each hour/day/minute/...
I need to track only Realm INSERT operations. Is there a listener for something like this?
I use a Realm-java for Android and work with a server that sends data in independent parts. For example, a Person or a Pet can be received via websocket in any order. For example, the server can first send me a Pet and then after a few minutes a Person. Or vice versa. I can't control it. I want to save data from the server to the database without any logic. At the same time, the listener, who reacts ONLY to the insertion, puts a link to the Pet in the Person after the recording. For example, it connects a newly received Person with a Pet already in the database, whose ownerId is equal to the inserted Person's id.
I found a solution: to track only the insert, you need to use DynamicRealm and OrderedRealmCollectionChangeListener on the search result.
val dRealm = DynamicRealm.getInstance(conf)
dRealm
.where('Person')
.findAllAsync()
.apply {
addChangeListener { results: RealmResults<DynamicRealmObject>, changeSet: OrderedCollectionChangeSet ->
if (changeSet.insertions.isNotEmpty()) {
// do what you need
}
}
}
.asFlowable()
.subscribe()
I have a situation where I need some information about my results, in my code I have this data class/Room's entity.
#Entity(primaryKeys = ["searchId","page","pr_id"])
data class ProductResponse(
var searchId: Int,
val page: Int,
#Embedded(prefix = "pr_")
val products: ProductSearchFormatted
)
Where you can see in the Android Studios App Inspector:
The data is saved as spected in ROOM, when I try to load it from ROOM:
#Query(
"SELECT * FROM PagedSearchResponse WHERE searchId ==:input"
)
fun loadPagedSearchResponse(input: Int): PagingSource<Int, ProductResponse>
I just need the data in the same order that was previously saved, and got the data in different order (ordered by pr_id):
I found out that if I change the primaryKeys order, like
#Entity(primaryKeys = ["pr_id","searchId","page"])
data class ProductResponse(
var searchId: Int,
val page: Int,
#Embedded(prefix = "pr_")
val products: ProductSearchFormatted
)
Now the data's order from ROOM is correct.
Why does this happen? Does the primaryKeys order matter?
Changing primaryKeys order changes the order of the data saved in ROOM
NO it does not, the data is ALWAYS saved in the order in which it is inserted. What is changing in your case, is the ORDER in which the data is extracted. That is because you aren't saying in what ORDER you want the data to be extracted and are leaving that choice to the query planner.
Why is this happened?, does the primaryKeys order matter?.
Yes it can do, especially in the absence of other indexes. Certainly pr_id before search_id will make a difference as the order within the index will be different and that as the WHERE clause is on the search_id then it is likely that the primary key index will be used (as in both cases search_id is an initial column (see the links below))
The query planner is an AI that tries to pick the fastest and most efficient algorithm for each SQL statement.
see :-
https://www.sqlite.org/queryplanner.html
https://www.sqlite.org/optoverview.html
https://www.sqlite.org/queryplanner-ng.html
If you want data to be in ORDER then you should specify an ORDER clause (ORDER BY ....). That is the only way to guarantee an ORDER. Assuming an ORDER without an ORDER clause will very likely result in issues.
Saying that using pr_id prior to search_id makes the composite (multiplte column) index likely to be more beneficial as the pr_id (according to the data shown) is less repeated than the search_id.
The way I fix it, it was simpler than I expected.
In the entity, I added a field called "order", and in my Mediator (I'm using Paging 3) basically did this:
list.mapIndex { s,t ->
ProductResponse(
...
order = s
...
)
}
That way, I'm following the EXACT order from Backend without the need to modify any primaryKeys.
whats up?
I have an app that displays a list of items on Firestore using Kotlin and RecyclerView (from FirebaseUI api).
The DB is structured like this:
Users/userID/document_month_year
I need to query the data from the current user.
Each user has his own document_month_year document.
I read a lot of posts here, but each one tell one thing.. there´s no consense and nothing seems to work.
This query just sends me all documents from all users, how can I fix this?
private val queryFilteredByPaidStatus = db.collectionGroup(collectionName).whereEqualTo("users", userId)
Like this is an important question, here is the awnser that I
private val queryTest = db.document("users/"+userId).collection(collectionName)
fun getData() : FirestoreRecyclerOptions<Expense> {
return FirestoreRecyclerOptions
.Builder<Expense>()
.setQuery(queryTest, Expense::class.java)
.build()
}
Create a separate collection for documents to be read i.e month_year and for each document, add a field inside it which tells you the uid of the authenticated user, to which the document belongs to. Now you can query the collection like:
firestoreDB.collections("month_year").whereEqualTo("uid",auth.currentUser.uid)
Please help, i'm kinda newbie when it comes to android app with a database.
In Greendao documentation there's a point of having this code:
List users = usersDao.queryBuilder().orderDesc(Properties.Id).list();
But somehow, it is not documented well on how to exactly get the values of rows from that query builder? or is there even a point where I could only get the first row, or last row from the database?
Thanks in advance for someone who will help.
Entity class are similar to all other classes. So greendao generates properties for all rows and you can access them as you access all others properties form "normal" classes.
To access Id property of UserEntity you can use getter user.getId().
From this query you are getting list of users. You can access it as you are accessing any other list in Java.
To get first user from database you can use code similar to:
List<User> users = usersDao.queryBuilder().orderDesc(...).limit(1).list();
if (users.size() < 1) return null;
return users.get(0);
To get last user I suggest to use similar query with reversed order:
List<User> users = usersDao.queryBuilder().orderAsc(...).limit(1).list();
if (users.size() < 1) return null;
return users.get(0);
If your query returns only one unique entity you can use unique method:
User user = usersDao.queryBuilder().where(...).unique();
You don't need to build these queries all the time. You can build it once, and then reuse it:
private Query<User> mSomeUserQuery;
...
// initialization
mSomeUserQuery = usersDao.queryBuilder().where(...).build();
// usage
User user = mSomeUserQuery.forCurrentThread().unique();
Greendao lists are not different to other lists in Java. You just have to google how to iterate through lists. Result: Ways to iterate over a List in java?
But here is the answer with the example to get the name value from an user:
for(User user : users){
String name = user.getName();
}