Getting big data unnecessarily while using orderbychild() from Firebase? - android

I'm developing an app that has a video and an article feed. I implemented swipe refresh layout. Whenever user swipes it, it loads last 5 video datas from Firebase.
My database is like this (it's not much data) =>
Swipe refresh layout listener =>
mSwipeRefreshLayout.setOnRefreshListener(() -> {
manager.scrollToPosition(0);
refreshVideoFeed();
});
And refreshVideoFeed method triggers this =>
Query query = myRef.orderByChild("videoDate").limitToLast(5);
query.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
..................
It works fine. However, I realized that app is using lots of data when i looked at Realtime Database dashboard (even app is not in product)
Then I opened Profiler in Android Studio to see what is going on. I ordered by child 'videoDate' and got last 5 videos. It costs 76 KB for refreshing video feed.
Then again I ordered by child 'videoDate' and got last 20 videos. Again it costs 77KB !
Also, I implemented the same mechanism for article feed. I have 236 articles in my realtime database. Whenever user swipes, it loads 10 articles and it costs 3,6 MB!
I wonder why this is happening and how to avoid that. It seems Firebase SDK fetchs all Videos to client and then filters them. But I want to fetch only specific range of data that I specified.

Ordering and filtering data is only done on the Firebase servers if there's an index defined on the property/value you filter on. If there's no index, the server sends all data to the client, which then orders and filters it. There should a quite explicit message in the log output when this happens.
To define an index, you go to the security rules panel in your Firebase console and on the node that myRef points to add an .indexOn property. Say myRef refers to /articles, it'd look something likeL
{
"rules": {
"articles": {
".indexOn": "videoDate"
}
}
}
If you order/filter on different properties, you can add multiple indexes:
".indexOn": [ "videoDate", "category" ]

Related

How do I retrieve firebase data based on the key in a nested child?

So suppose I am building an app that lets users manage trips. When a trip is created , any number of users can be added in it. I want this data to be stored in a single place and then for each user be able to retrieve the trips that that person is included in. My data looks like this:
-trips
-<trip-id>
-title
-budget
-etc
-people
-<person-id>
-name
-uid
-<person-id>
-name
-uid
-<trip-id>
-
-
-
This trip will contain all the trips created by all the users. To show any person their trips, I want to retrieve only the lists that person exists in.
This is what I've tried to do including other similar approaches.
rootReference.child("trips").orderByChild("uid").equalTo(FirebaseAuth.instance.currentUser.uid).addValueEventListener(object:ValueEventListener){
override fun onDataChange(snapshot: DataSnapshot) {
//this should only return the trips that current user exists in.
}
}
I have checked the documentation for searching and filtering on firebase but there is nothing that show filtering based nested keys. One particular example is this. I understand it perfectly. If for example I try to filter my trips based on the main attributes like title, budget, it works, but not when I use an attribute of a nested child.
What other approach can I use to filter based to nested keys or should I structure the data differently? Any help is greatly appreciated.
Firebase Realtime Database queries operate on a flat list of child nodes directly under the path that you query.
So the value you order/filter on has to be at a fixex path under each immediate child node. Since that isn't the case for your uid, you can't query across all trips for the UID of all users of those trips.
You can query across one trip for a UID of a user (and then get back that user), or you can query across all trips for properties of the trip itself, such as its title or budget.
If you want to query across all users on all trips, consider keeping an additional list where you have the UID of the user as the key, and then all their trips under there:
"user_trips": {
"uid1": {
"tripid1": true,
"tripid2": true
},
"uid2": {
"tripid2": true,
"tripid3": true
}
}
Also see:
Firebase Query Double Nested
Firebase query if child of child contains a value
Many to Many relationship in Firebase

Firestore One time fetch vs Realtime listener in Recyclerview with filter

I have feed like the image in my app. I am getting list of some documents from firetsore collection and I am populating them in List and then use it in recycler view to show details of user.
When feed is clicked, I want to show ditrict3 and district4 data. Like below:
collectionRef = FirebaseFirestore.getInstance().collection("male");
collectionRef.whereIn("district", Arrays.asList("district3", "district4"))
.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
#Override
public void onComplete(#NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
List<FilterDetails> filterDetailsList = new ArrayList<>();
for (DocumentSnapshot document : task.getResult()) {
FilterDetails filterDetails = document.toObject(FilterDetails.class);
filterDetailsList.add(filterDetails);
getParentActivity().hideLoading();
}
feedItemAdapter.updateData(filterDetailsList);
}
}
});
When district1 will be clicked I will change whereEqualTo("district", "district1") and for different selection I will change.
User can change this more frequently and if I will do one time fetch then its going to cost me very much. But If I will use realtime snapshot then data will be read again and again if other users will change their info. Solutions I thought,
1) To get all the data before where clause and do district filtering on client side(app). But in this if there are 2000 users in each state. I would have to load that much data.
2) To do onetime fetch for whatever user clicks and then show them same list. I will make separate list for all fetches.
I am really confused upon this. Any insight would be helpful.
Keep in mind that it caches recently read documents locally, and won't charge you again if future queries are read form that cache. I'd recommend running some tests with addSnapshotListener and monitoring the usage that it generates. It may be a lot less tan you imagined, and at the very least you'll have more concrete data to validate/dispel your concerns.
If you're worried still about the pricing of Cloud Firestore, I recommend doing some back-of-a-napkin calculations of realistic scenarios. I highly recommend comparing Cloud Firestore with an alternative like the Realtime Database in that case. Most cloud-based databases come with a fairly unique pricing model, and only you can decide what combination of features and price work for your app.

How to create index on a child node nested child key?

Let's say my current structure firebase realtime database:
{
chatRooms:{
chat1:{
participants:{
user1-id:true,
user2-id:true
}
participantsCount:2,
lastTimeUpdated: {TIMESTAMP},
chatRoomName:"A chatroom",
maxParticipantsCount:10
}
}
users:{
user1-id:{
name:user1-name,
email:user1-email
}
user2-id:{
name:user2-name,
email:user2-email
}
}
}
I need to display all chatrooms that contains a specific user along with all its information, and my solution now is:
#userId is currently authenticated, firebase user.
ref.child("chats").orderByChild("/participants/"+userId).equalTo(true)
which returns the list of chats containing $userId which equals to true, along with additional information to display to the user. Is there any possible way to change this query AND/OR database structure to support optimal indexing? If I wanted to add indices, it would be something like:
chatrooms:{
participants:{
indexOn:"user1-id","user2-id"
# number of ids grows as participants increase
# Not only that, I will have to add one by one manually.
}
}
How can I add indices and also at the same time do a query in Android that will display all chatrooms that a user belongs to along with their information?
AFAIK, I cannot "loop" over the keys of the child in Android
To loop over the children of a snapshot in Android do:
for (DataSnapshot childSnapshot: dataSnapshot.getChildren()) {
System.out.println(childSnapshot.getKey());
}
Relevant section in the docs: https://firebase.google.com/docs/database/android/lists-of-data#listen_for_value_events
If you're asking how to display the information about each chat room that a user is in, that will take a client-side join like this:
firebase.database().ref("userChatrooms").child(firebase.auth().currentUser.uid).once(function(roomsSnapshot) {
roomsSnapshot.forEach(function(roomKey) {
firebase.database().ref("chatrooms").child(roomKey.key).once(function(roomSnapshot) {
console.log(roomSnapshot.child("name").val());
})
})
})
This type of client-side join is not nearly as slow as most developers expect, since Firebase pipelines the nested requests.

Can't sort data from firebase realtime database by timeStamp

I am making a challenges between users in my app .I am trying to get the last 15 users who enters in challenges. I store the time each time the users enter a new challenge. The query was working well in the begging but after that it stops showing new users and only old users appears in it.
this is the query code :
usersReference.orderByChild("lastChallengeDate")
.limitToLast(15)
.addListenerForSingleValueEvent(new ValueEventListener()
and this is the database structre of the user child :
When I opened the log I found this warn although I am using index on in my rules
W/PersistentConnection: pc_0 - Using an unspecified index. Consider adding '".indexOn": "lastChallengeDate"' at users to your security and Firebase Database rules for better performance
If you perform a query on a location, Firebase sorts the children under that location on the property you specify. There is no value in lastChallengeDate directly under each child of users. Instead the property is under lastChallengeDate/time, so you should order on that
usersReference.orderByChild("lastChallengeDate/time")
.limitToLast(15)
.addListenerForSingleValueEvent(new ValueEventListener()
You also need to define an index on users (or whatever the name is of the node you query):
{
"rules": {
"users": {
".indexOn": "lastChallengeDate/time"
}
}
}
Be sure to also study:
the documentation on queries, which includes an example of querying such a nested property
the documentation on defining indexes
some of the many questions with the same error message

Search child nodes in firebase database using android

Hello guys here is my firebase database:
I want to get list of all medicines with particular symptoms.
Here is my code i.e what i have done
public void initializeMedicineListener(String node,String type,String value){
mDatabase=mFirebaseInstance.getReference();
Query query = mDatabase.child("Medicine").child("symptoms").orderByChild("name").equalTo("Neck pain");
query.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
medicineList=new ArrayList<Medicine>();
if (dataSnapshot.exists()) {
for (DataSnapshot medicine : dataSnapshot.getChildren()) {
Medicine data = medicine.getValue(Medicine.class);
medicineList .add(data);
}
}
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
});
}
But i am getting null results.
Please guide me guys.Am i doing something wrong??
When you run a query at a location, Firebase check each child node at that location for the property that you order on and the range/condition you filter on.
mDatabase.child("Medicine").child("symptoms").orderByChild("name").equalTo("Neck pain");
So this checks the children of /Medicine/symptoms for their name property and only returns them if they have a value equal to Neck pain.
There are two problems with this:
Your JSON doesn't have a /Medicine/symptoms. Instead you have a Medicine, where each child node has a symptoms node.
Even if it did, your symptoms child doesn't have a value Neck pain. Instead you have an array, where each value may be Neck pain.
The closest you can now get to the query you want is:
mDatabase.child("Medicine").orderByChild("symptoms/0/name").equalTo("Neck pain");
This query returns medicines for which the first symptom name is equal to Neck pain. Firebase cannot perform a query across all array members to see if it contains a specific value in any position.
As usual with NoSQL databases: if you can't perform the use-case you want with your current data structure, you can typically change/expand your data structure to allow the use-case. And usually this is done by very directly mapping what you want to show on your screen to the structure in the database.
Your current data structures allows you to efficiently look up the symptoms (and other data) for a given medicine. That's great. It does however not allow you to efficiently look up the medicines for a given symptom. To allow that you can for example add a structure that maps each specific symptom back to its medicines:
symptoms
"Neck pain"
-L6hb2...bRb0: true
-L6rW...Fuxkf: true
With this additional structure (known as an inverted index or reverse index) you can now look up the medicines for Neck pain by simple loading /symptoms/Neck pain.
For more on this approach to categorization, see my answer here: Firebase query if child of child contains a value
You can't work with firebase database as relational databases .. instead of what you looking for .. you can create new node like this
medicineSymp
---neckpain
-------medicine1ID
-------medicine1ID
-------medicine1ID
this way you could get all medicines much faster and easier

Categories

Resources