While firebase recommends use of indexes to keep the data structure flat, I am running into an issue here.
Consider this users entity (from Firebase docs sample)
{
"users": {
"mchen": {
"name": "Mary Chen",
// index Mary's groups in her profile
"groups": {
// the value here doesn't matter, just that the key exists
"alpha": true,
"charlie": true
"delta" : true
...
"10000th entry":true
}
},
...
}
Now I can read a user from the database like this.
List<User> users = new Select().from(User.class).limit(5).execute();
But, my question is if the "groups" key has say 10,000 entries what would happen? Will it fetch the whole list? Wouldn't it crash the app, due to lack of memory.
In general, if I am trying to fetch a high level node (which has nested nodes), what part of it will be fetched?
Related
I have a list of users, assume 10k.
Now if I run a query like below:
query = FirebaseDatabase.getInstance().getReference().child("users")
.orderByChild(SELECTED_QUERY)
.limitToFirst(150);
Will it consume bandwidth of 150 users' data or 10k users' data?
This question may be silly, but I haven't found expected answer.
Whether the SDK retrieves all users or just the first 150 depends on whether you've defined an index on SELECTED_QUERY. Without an index the SDK will retrieve all of users and filter locally on the client; with the index the filtering will happen on the server.
Given your comment, you'll need to add an index to your rules like this:
{
"rules": {
...
"users": {
...
".indexOn": "name"
}
}
}
I have a data set on firebase real time database like this.
Users
---Match
------User1
mId:"12345"
xId:""
------User2
mId:"54321"
xId:""
Basically developing a multi player matching functionality here using android application.Assuming user 1 matched with user 2. User1 will update his own xId value with user2 mId value and user2 will update his own xId value with mId value of user1.
This works fine if there are two users. Both connects to each other and works well. Issue comes when 3 users come at the same time. User1 writes to user2 and user2 writes to user3 and creates inconsistent data set.
I want to lock a value once it is changed or we can say a simple rule like.
update user1 only : if(user1.xid=="")
update user2 only : if(user2.xid=="")
Is it possible using firebase security rule?
To only allow writing new data (so never overwrite or delete existing data), you'll want this rule:
{
"rules": {
"Users": {
"Match": {
"uid": {
".validate": "!data.child('xid').exists() && newData.child('xid').exists()"
}
}
}
}
}
For more elaborate information check out the Firebase documentation for security rules, specifically the section on New vs. Existing Data.
I have a realtime database structure like below.
Users have some friends. For example, I am Joe and I'd like to fetch users who have friends as Joe.
In this example, Dave and Harry have a friend called Joe, so my query should fetch Dave and Harry but shouldn't Micheal.
Is there a way to do this without fetching data in a loop?
{
"Users": {
"Joe": {
"Friends": ["Dave","Harry","Micheal"]
},
"Dave": {
"Friends": ["Joe","Jack","Brent"]
},
"Harry": {
"Friends": ["Jack","Joe"]
},
"Micheal": {
"Friends": ["Ken","Jack","Brent"]
}
}
}
Firebase Realtime database cannot perform a query across array members, to see if it contains a specific value in any position or to add/remove a record. Firebase documentation recommends against using arrays. One of the many reasons Firebase recommends against using arrays is that it makes the security rules impossible to write.
As usual with NoSQL databases: if you can't perform the use-case you want with your current data structure, you can typically change/expand your data structure to allow the use-case. The best practice is to use maps instead of arrays. A possible database structure could look like this:
"Users": {
"Joe": {
"Friends"
"Dave": true,
"Harry": true,
"Micheal": true
},
"Dave": {
"Friends"
"Joe": true,
"Jack":true,
"Brent": true
},
"Harry": {
"Friends"
"Jack": true,
"Joe": true
},
"Micheal": {
"Friends"
"Ken":true,
"Jack": true,
"Brent": true
}
}
You can also take a look, here, for more informations.
The structure of the table is:
chats
--> randomId
-->--> participants
-->-->--> 0: 'name1'
-->-->--> 1: 'name2'
-->--> chatItems
etc
What I am trying to do is query the chats table to find all the chats that hold a participant by a passed in username string.
Here is what I have so far:
subscribeChats(username: string) {
return this.af.database.list('chats', {
query: {
orderByChild: 'participants',
equalTo: username, // How to check if participants contain username
}
});
}
Your current data structure is great to look up the participants of a specific chat. It is however not a very good structure for looking up the inverse: the chats that a user participates in.
A few problems here:
you're storing a set as an array
you can only index on fixed paths
Set vs array
A chat can have multiple participants, so you modelled this as an array. But this actually is not the ideal data structure. Likely each participant can only be in the chat once. But by using an array, I could have:
participants: ["puf", "puf"]
That is clearly not what you have in mind, but the data structure allows it. You can try to secure this in code and security rules, but it would be easier if you start with a data structure that implicitly matches your model better.
My rule of thumb: if you find yourself writing array.contains(), you should be using a set.
A set is a structure where each child can be present at most once, so it naturally protects against duplicates. In Firebase you'd model a set as:
participants: {
"puf": true
}
The true here is really just a dummy value: the important thing is that we've moved the name to the key. Now if I'd try to join this chat again, it would be a noop:
participants: {
"puf": true
}
And when you'd join:
participants: {
"john": true,
"puf": true
}
This is the most direct representation of your requirement: a collection that can only contain each participant once.
You can only index known properties
With the above structure, you could query for chats that you are in with:
ref.child("chats").orderByChild("participants/john").equalTo(true)
The problem is that this requires you to define an index on `participants/john":
{
"rules": {
"chats": {
"$chatid": {
"participants": {
".indexOn": ["john", "puf"]
}
}
}
}
}
This will work and perform great. But now each time someone new joins the chat app, you'll need to add another index. That's clearly not a scaleable model. We'll need to change our data structure to allow the query you want.
Invert the index - pull categories up, flattening the tree
Second rule of thumb: model your data to reflect what you show in your app.
Since you are looking to show a list of chat rooms for a user, store the chat rooms for each user:
userChatrooms: {
john: {
chatRoom1: true,
chatRoom2: true
},
puf: {
chatRoom1: true,
chatRoom3: true
}
}
Now you can simply determine your list of chat rooms with:
ref.child("userChatrooms").child("john")
And then loop over the keys to get each room.
You'll like have two relevant lists in your app:
the list of chat rooms for a specific user
the list of participants in a specific chat room
In that case you'll also have both lists in the database.
chatroomUsers
chatroom1
user1: true
user2: true
chatroom2
user1: true
user3: true
userChatrooms
user1:
chatroom1: true
chatroom2: true
user2:
chatroom1: true
user2:
chatroom2: true
I've pulled both lists to the top-level of the tree, since Firebase recommends against nesting data.
Having both lists is completely normal in NoSQL solutions. In the example above we'd refer to userChatrooms as the inverted index of chatroomsUsers.
Cloud Firestore
This is one of the cases where Cloud Firestore has better support for this type of query. Its array-contains operator allows filter documents that have a certain value in an array, while arrayRemove allows you to treat an array as a set. For more on this, see Better Arrays in Cloud Firestore.
The structure of the table is:
chats
--> randomId
-->--> participants
-->-->--> 0: 'name1'
-->-->--> 1: 'name2'
-->--> chatItems
etc
What I am trying to do is query the chats table to find all the chats that hold a participant by a passed in username string.
Here is what I have so far:
subscribeChats(username: string) {
return this.af.database.list('chats', {
query: {
orderByChild: 'participants',
equalTo: username, // How to check if participants contain username
}
});
}
Your current data structure is great to look up the participants of a specific chat. It is however not a very good structure for looking up the inverse: the chats that a user participates in.
A few problems here:
you're storing a set as an array
you can only index on fixed paths
Set vs array
A chat can have multiple participants, so you modelled this as an array. But this actually is not the ideal data structure. Likely each participant can only be in the chat once. But by using an array, I could have:
participants: ["puf", "puf"]
That is clearly not what you have in mind, but the data structure allows it. You can try to secure this in code and security rules, but it would be easier if you start with a data structure that implicitly matches your model better.
My rule of thumb: if you find yourself writing array.contains(), you should be using a set.
A set is a structure where each child can be present at most once, so it naturally protects against duplicates. In Firebase you'd model a set as:
participants: {
"puf": true
}
The true here is really just a dummy value: the important thing is that we've moved the name to the key. Now if I'd try to join this chat again, it would be a noop:
participants: {
"puf": true
}
And when you'd join:
participants: {
"john": true,
"puf": true
}
This is the most direct representation of your requirement: a collection that can only contain each participant once.
You can only index known properties
With the above structure, you could query for chats that you are in with:
ref.child("chats").orderByChild("participants/john").equalTo(true)
The problem is that this requires you to define an index on `participants/john":
{
"rules": {
"chats": {
"$chatid": {
"participants": {
".indexOn": ["john", "puf"]
}
}
}
}
}
This will work and perform great. But now each time someone new joins the chat app, you'll need to add another index. That's clearly not a scaleable model. We'll need to change our data structure to allow the query you want.
Invert the index - pull categories up, flattening the tree
Second rule of thumb: model your data to reflect what you show in your app.
Since you are looking to show a list of chat rooms for a user, store the chat rooms for each user:
userChatrooms: {
john: {
chatRoom1: true,
chatRoom2: true
},
puf: {
chatRoom1: true,
chatRoom3: true
}
}
Now you can simply determine your list of chat rooms with:
ref.child("userChatrooms").child("john")
And then loop over the keys to get each room.
You'll like have two relevant lists in your app:
the list of chat rooms for a specific user
the list of participants in a specific chat room
In that case you'll also have both lists in the database.
chatroomUsers
chatroom1
user1: true
user2: true
chatroom2
user1: true
user3: true
userChatrooms
user1:
chatroom1: true
chatroom2: true
user2:
chatroom1: true
user2:
chatroom2: true
I've pulled both lists to the top-level of the tree, since Firebase recommends against nesting data.
Having both lists is completely normal in NoSQL solutions. In the example above we'd refer to userChatrooms as the inverted index of chatroomsUsers.
Cloud Firestore
This is one of the cases where Cloud Firestore has better support for this type of query. Its array-contains operator allows filter documents that have a certain value in an array, while arrayRemove allows you to treat an array as a set. For more on this, see Better Arrays in Cloud Firestore.