I need to track only Realm INSERT operations. Is there a listener for something like this?
I use a Realm-java for Android and work with a server that sends data in independent parts. For example, a Person or a Pet can be received via websocket in any order. For example, the server can first send me a Pet and then after a few minutes a Person. Or vice versa. I can't control it. I want to save data from the server to the database without any logic. At the same time, the listener, who reacts ONLY to the insertion, puts a link to the Pet in the Person after the recording. For example, it connects a newly received Person with a Pet already in the database, whose ownerId is equal to the inserted Person's id.
I found a solution: to track only the insert, you need to use DynamicRealm and OrderedRealmCollectionChangeListener on the search result.
val dRealm = DynamicRealm.getInstance(conf)
dRealm
.where('Person')
.findAllAsync()
.apply {
addChangeListener { results: RealmResults<DynamicRealmObject>, changeSet: OrderedCollectionChangeSet ->
if (changeSet.insertions.isNotEmpty()) {
// do what you need
}
}
}
.asFlowable()
.subscribe()
Related
My goal is to effectively get a list of children (ordered* and indexed**) with the lowest number of data transfer.
* ordered: ordered by points for each user / database child
** indexed: 2 or less ranks behind/after the current user [A specific child] (further elaborated below)
My database structure is as follows:-
I basically want to get the first 3 users ordered by points (simple):-
val usersRef = FirebaseDatabase.getInstance(DB_LINK).getReference("users").orderByChild("points")
usersRef.limitToFirst(3).addValueEventListener(
object : ValueEventListener {
override fun onDataChange(snapshot: DataSnapshot) {
for (ds in snapshot.children) {
val points: String = snapshot.child("points").getValue(String::class.java)!!
val firstName: String = snapshot.child("firstName").getValue(String::class.java) ?: ""
val uid: String = snapshot.key!!
// Update View
}
}
override fun onCancelled(error: DatabaseError) {}
}
)
Then, provided that the currently logged in user isn't one of the first three, I want to get his rank (order according to points in the whole db), 2 users' before him, and 2 users' after him without querying the whole database (it's a user database that can get up to 50K unique users) because querying the whole database is a really expensive client-side task.
I checked firebase data filtering page but found nothing useful about limiting results according to a certain child.
This answer doesn't satisfy my needs, as it loops over the whole database (in my case, 50K records). I need an effective method as I need to really save these firebase bills.
Moreover, I check this answer but it didn't meet my needs because it still queries the whole database, meaning it is not effective and will be billed for each node before the current user. (Maybe he is number 40,000 in the db, so I shouldn't query the whole db each time to get his rank and get billed for 39,999 reads)
I searched for a way to somehow use booleans to filter queries but again found nothing useful. Here is my not-effective code:-
// Gets all children.
usersRef.addValueEventListener(
object : ValueEventListener {
override fun onDataChange(snapshot: DataSnapshot) {
for (ds in snapshot.children) {
val points: String = snapshot.child("points").getValue(String::class.java)!!
val firstName: String = snapshot.child("firstName").getValue(String::class.java) ?: ""
val uid: String = snapshot.key!!
// Update View only if user is `2 <= usersRank - theirRank <= -2`
}
}
override fun onCancelled(error: DatabaseError) {}
}
)
I want to achieve something like this:- (Styling already done, logic remaining)
Is there a way to achieve this? Any alternatives?
EDIT: I found out that firestore offers aggregation queries that may help in this situation. Doing more research to further narrow down the costs.
This operation is not available on a Firebase Realtime Database. A better option would be Firestore.
Why?
Well, A fire-store database can give you the count of objects in a certain query. This is a new feature added by firebase. You basically type the query you want, then add .count() before .get(); that way it'll return the count of objects only. This is called aggregation queries. Learn more about them here.
Cloud Functions - Why isn't it appropriate here?
Using a Cloud Function for aggregations avoids some of the issues with client-side transactions, but comes with a different set of limitations:
Cost - Each rating added will cause a Cloud Function invocation, which may increase your costs. For more information, see the Cloud Functions pricing page.
Latency - By offloading the aggregation work to a Cloud Function, your app will not see updated data until the Cloud Function has finished executing and the client has been notified of the new data. Depending on the speed of your Cloud Function, this could take longer than executing the transaction locally.
Write rates - this solution may not work for frequently updated aggregations because Cloud Firestore documents can only be updated at most once per second. Additionally, If a transaction reads a document that was modified outside of the transaction, it retries a finite number of times and then fails.
Combining this with other methods
Now that you're using COUNT() for this system, there is one more method to help further narrow down the costs. That is Periodic Updates.
Periodic Updates
Who would care about a live ranking of all users? You can make the leaderboard update each minute, hour, or day. For example, stack overflow's leaderboard is updated once a day!
This approach would really work for any number of players and any write rate. However, you might need to adjust the frequency though as you grow depending on your willingness to pay.
Costs Side
For each normal read, you are charged for one read. Very simple. However, for one count, you're charged for 0.001 reads (meaning 1000 counts = 1 read). For more information about costs, check this article by firebase.
Final Thoughts
To connect everything up, we shall now apply this on our problem. Firstly, we'll need to keep the first portion of the code as it is. (The portion that grabs the first 3 users), though with some changes to port it to firebase.
NOTICE: Don't forget to setup a composite index because we're ordering by multiple fields at the same time.
val top3 = HashMap<Int, HashMap<String, String>>()
Firebase.firestore.collection("users").orderBy("points", Query.Direction.DESCENDING)
.orderBy("firstName", Query.Direction.ASCENDING)
.get().addOnSuccessListener {
for ((index, doc) in it.documents.withIndex()) {
val firstName = doc.getString("firstName")!!
val points = doc.getString("points")!!
top3[index+1] = hashMapOf("first" to firstName, "points" to points, "uid" to doc.id)
}
}
More about ordering and limiting here.
Then, we'll need to implement the COUNT() feature.
Firebase.firestore.collection("users")
.whereGreaterThan("points", UserProfile.getInstance()!!.getProfile().points)
.count().get(AggregateSource.SERVER).addOnSuccessListener {
println("Your rank is: ${it.count+1}")
}
Basically what I did here was:-
Selecting the Collection
Ordered Ascending by first name so no duplicate ranks.
Count them, and pass onto the function.
The final step is just updating the hash map top3 and rank of user each hour/day/minute/...
Lets say i have a list of Objects (Accounts) in my RoomDatabase. And when the application starts i request the Accounts from Server in order to Update the Accounts Table.
So i query the database, getting a
current: List<Account> which is the existing accounts in the database
From the Server we have the news: List<Account> list which is the updated Accounts
I want to find
The elements from the news that exist in the current, in order to update them
The elements from the news that don't exist in the current, in order to insert them
The elements from the current that don't exist in the news, in order to delete them
Is there any extension function that can do that Job with an elegant and fast way, except the obvious that is to do one step at a time with the already known technique? (Loop in each list every time)
You could use partition I suppose
val current = listOf("hey", "hi", "wow")
val news = listOf("hi", "hey", "sup")
fun main() {
val (retain, discard) = current.partition { it in news }
val insert = news.filterNot { it in current }
println("Retain: $retain\nDiscard: $discard\nInsert: $insert")
}
>> Retain: [hey, hi]
>> Discard: [wow]
>> Insert: [sup]
"Elegant and fast" is subjective - is this less elegant?
val retain = current.filter { it in news }
val discard = current.filter { it !in news }
val insert = news.filter { it !in current }
it's arguably neater and easier to read - three conditions, described the exact same way.
Faster? Well that depends on the size of your lists, and you'd have to benchmark the solutions to see how much difference it makes (and if it's worth prioritising it over readability). Converting the lists to sets would give you a performance boost for the in checks, but the conversion itself might be slower than the gains in lookup speed
According on your answer - the most efficient way in your case is just drop your previous accounts from db and write new.
Your backend data always a priority.
Instead of many data manipulations(get from db, compare lists, insert and delete to new list), do just
delete old accounts with delete method in your room dao.
insert new accounts from your backend using insert method in dao.
Extension for this with coroutines:
suspend fun List<Account>.insertActual() = withContext(IO) {
yourDao.deleteAccounts()
yourDao.insertAccounts(this)
}
I have a program where data is stored in a local database using Room, and I receive data through RxJava2. In Provider, the database created a method that takes one parameter - an identifier, by which it sends a request and receives certain data that corresponds to the identifier. But I want to pass as a parameter not one identifier, but an array of identifiers and get an array as well, but I don't know how. I don’t want to implement it through for, because I think there is a better solution, but I couldn’t find it. My code is shown below.
...
userDatabase.userDao().getById(id)
.subscribeOn(Schedulers.io())
.observeOn(SchedulerProvider.ui())
.map {
InfoStruct(
it.Name.toString(),
it.Id.toString()
)
}
.subscribe(
{println("${it.userName} || ${it.userId}")},
{println("Error")}
)
...
I pass one value to the getById method parameter - an identifier, and send a request like this: select * where id =: id. I want to pass an array of identifiers as a parameter to get data for several users at once, but I don't want to change the structure of the request. How can this be done?
You could use a dao, e.g. getByManyIds(), that uses a WHERE clause that returns a list based upon a provided list.
The SELECT clause could be along the lines of
#Query("SELECT * FROM the_table WHERE id IN (:idlist)"
List<your_type> getByManyIds(appropriate_type idlist);
appropriate_type should be an array of the id's int[], long[], String[] etc as is suitable (Room will generate the appropriate comma separated list).
Assuming that your Dao method looks like this:
#Query("SELECT * FROM user WHERE id = :id"
Single<User> getById(String id);
You need to do is change it to this:
#Query("SELECT * FROM user WHERE id IN (:ids)"
Single<List<User>> getByIds(List<String> ids);
And then use it like that:
userDatabase.userDao().getByIds(listOfIds)
.subscribeOn(Schedulers.io())
.observeOn(SchedulerProvider.ui())
.map { users ->
users.map {
InfoStruct(
it.Name.toString(),
it.Id.toString()
)
}
}
....
In my Android Kotlin App. User profile (A form) needs to be updated using data from RestAPI eg. education and location data from API looks like this
{"educations": [{"edu_uid": 123, "edu_sector": "xxxx", "edu_degree": "xxx"}, .....}
{"locations": [{"loc_uid": 123, "loc_place": "xxxx", "loc_zip": "xxx"}, .....}
After consuming this data. We show it in an adapter. User choose his choice. We update XML views like #id/textViewEduSector and #id/textViewEduDegree. After filling up the form, When the User submit data I send only "edu_uid" "loc_uid" to the Server. Here is representation of my implementation:
profile_fragment -> education_fragment -> education_adapter onEducationClick ->
navigateUp() -> profile_fragment update #id/textViewEduSector and #id/textViewEduDegree.
Then location and so. At the end when User submit the form,how can I pick "edu_uid" and "loc_uid" to sendback data to API endponit eg. "/profile/update".
How can I collect these uids temporarily. I have a Room database with all User fields. Here I donot want to retain hidden fields, "edu_uid" or "loc_uid". Is it necessary to update Room via Dao and then picking data from there and sent to API or any other workaround.? Fast and efficient?
Since it is hard to get the point of your problem, i try to give my general thoughts about your use-case.
Try to separate next issues:
What is single source of truth in your app? Is it a local database? If not - why?
How to represent you data sources in UI?
How to synchronise your data structures with server?
Data structures. Try to use a database (ROOM) as a single source of truth in your app. If you do, you get additional benefits of working without access to Rest API. In this case your ROOM-entities set should include - Education (edu_uid as Primary Key), Location (loc_uid as Primary Key) and User/UserProfile (with edu_uid and loc_uid as Foreign keys). In this case you should't add fields "edu_sector" or "edu_degree" to UserProfile data class. And with a action "submit" you have to save changes to your local database (with edu_id and loc_uid) and to solve your "REST API-question" at next step.
Representation in UI. If you use MVVM, you should't have a problems with "edu_id" or "loc_id". You declare references to Education and Location in your ViewModel and in your XML you can easy consume any fields of these objects.
Sync with server. It should be easy if you have efficient local database with all needed fields as foreign keys (again in your DAO you can get all the rest fields you want from other tables with table joins)
Again, maybe I didn't get your real problem. It is really unclear
Firebase documentation on structuring the database mentions the following:
...when you fetch data at a location in your database, you also retrieve all of its child nodes.
I have the following basic data structure:
databaseRoot: {
orders: {
order1: {
firebaseId: firebaseUid,
...
}
}
orderItems: {
order1: {
firebaseId: firebaseUid,
...
}
}
}
In my android start-up code I intend to sync the orders and orderItems that belong to the current user to handle the offline scenario.
FirebaseDatabase.getReference("orders").orderByChild("firebaseId").equalTo(firebaseUser.uid).keepSynced(true)
FirebaseDatabase.getReference("orderItems").orderByChild("firebaseId").equalTo(firebaseUser.uid).keepSynced(true)
My Question:
Do the sync references consider the data location to be the entire Orders and OrderItems nodes, respectively? Or does it resolve down to the specific child nodes that satisfy the firebase user id equal to the current user?
My hope is for the latter to be the result, as the client does not need any records beyond those belonging to the current user.
Calling keepSynced(true) on a location is essentially the same as keeping a listener with an empty callback on that location.
So these two commands will ensure you that the orders and orderItems for the specific user are in cache.
But if that is the type of behavior you want, you might want to consider keep the data in a simpler structure: /orders/firebaseUid/order1, /orders/firebaseUid/order2, etc. With this structure you won't need a query to get the orders for a user and can just read from (and call keepSynced() on) /orders/firebaseUid.