I thought I read that you can query subcollections with the new Firebase Firestore, but I don't see any examples. For example I have my Firestore setup in the following way:
Dances [collection]
danceName
Songs [collection]
songName
How would I be able to query "Find all dances where songName == 'X'"
Update 2019-05-07
Today we released collection group queries, and these allow you to query across subcollections.
So, for example in the web SDK:
db.collectionGroup('Songs')
.where('songName', '==', 'X')
.get()
This would match documents in any collection where the last part of the collection path is 'Songs'.
Your original question was about finding dances where songName == 'X', and this still isn't possible directly, however, for each Song that matched you can load its parent.
Original answer
This is a feature which does not yet exist. It's called a "collection group query" and would allow you query all songs regardless of which dance contained them. This is something we intend to support but don't have a concrete timeline on when it's coming.
The alternative structure at this point is to make songs a top-level collection and make which dance the song is a part of a property of the song.
UPDATE
Now Firestore supports array-contains
Having these documents
{danceName: 'Danca name 1', songName: ['Title1','Title2']}
{danceName: 'Danca name 2', songName: ['Title3']}
do it this way
collection("Dances")
.where("songName", "array-contains", "Title1")
.get()...
#Nelson.b.austin Since firestore does not have that yet, I suggest you to have a flat structure, meaning:
Dances = {
danceName: 'Dance name 1',
songName_Title1: true,
songName_Title2: true,
songName_Title3: false
}
Having it in that way, you can get it done:
var songTitle = 'Title1';
var dances = db.collection("Dances");
var query = dances.where("songName_"+songTitle, "==", true);
I hope this helps.
UPDATE 2019
Firestore have released Collection Group Queries. See Gil's answer above or the official Collection Group Query Documentation
Previous Answer
As stated by Gil Gilbert, it seems as if collection group queries is currently in the works. In the mean time it is probably better to use root level collections and just link between these collection using the document UID's.
For those who don't already know, Jeff Delaney has some incredible guides and resources for anyone working with Firebase (and Angular) on AngularFirebase.
Firestore NoSQL Relational Data Modeling - Here he breaks down the basics of NoSQL and Firestore DB structuring
Advanced Data Modeling With Firestore by Example - These are more advanced techniques to keep in the back of your mind. A great read for those wanting to take their Firestore skills to the next level
What if you store songs as an object instead of as a collection? Each dance as, with songs as a field: type Object (not a collection)
{
danceName: "My Dance",
songs: {
"aNameOfASong": true,
"aNameOfAnotherSong": true,
}
}
then you could query for all dances with aNameOfASong:
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
NEW UPDATE July 8, 2019:
db.collectionGroup('Songs')
.where('songName', isEqualTo:'X')
.get()
I have found a solution.
Please check this.
var museums = Firestore.instance.collectionGroup('Songs').where('songName', isEqualTo: "X");
museums.getDocuments().then((querySnapshot) {
setState(() {
songCounts= querySnapshot.documents.length.toString();
});
});
And then you can see Data, Rules, Indexes, Usage tabs in your cloud firestore from console.firebase.google.com.
Finally, you should set indexes in the indexes tab.
Fill in collection ID and some field value here.
Then Select the collection group option.
Enjoy it. Thanks
You can always search like this:-
this.key$ = new BehaviorSubject(null);
return this.key$.switchMap(key =>
this.angFirestore
.collection("dances").doc("danceName").collections("songs", ref =>
ref
.where("songName", "==", X)
)
.snapshotChanges()
.map(actions => {
if (actions.toString()) {
return actions.map(a => {
const data = a.payload.doc.data() as Dance;
const id = a.payload.doc.id;
return { id, ...data };
});
} else {
return false;
}
})
);
Query limitations
Cloud Firestore does not support the following types of queries:
Queries with range filters on different fields.
Single queries across multiple collections or subcollections. Each query runs against a single collection of documents. For more
information about how your data structure affects your queries, see
Choose a Data Structure.
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
Queries with a != clause. In this case, you should split the query into a greater-than query and a less-than query. For example, although
the query clause where("age", "!=", "30") is not supported, you can
get the same result set by combining two queries, one with the clause
where("age", "<", "30") and one with the clause where("age", ">", 30).
I'm working with Observables here and the AngularFire wrapper but here's how I managed to do that.
It's kind of crazy, I'm still learning about observables and I possibly overdid it. But it was a nice exercise.
Some explanation (not an RxJS expert):
songId$ is an observable that will emit ids
dance$ is an observable that reads that id and then gets only the first value.
it then queries the collectionGroup of all songs to find all instances of it.
Based on the instances it traverses to the parent Dances and get their ids.
Now that we have all the Dance ids we need to query them to get their data. But I wanted it to perform well so instead of querying one by one I batch them in buckets of 10 (the maximum angular will take for an in query.
We end up with N buckets and need to do N queries on firestore to get their values.
once we do the queries on firestore we still need to actually parse the data from that.
and finally we can merge all the query results to get a single array with all the Dances in it.
type Song = {id: string, name: string};
type Dance = {id: string, name: string, songs: Song[]};
const songId$: Observable<Song> = new Observable();
const dance$ = songId$.pipe(
take(1), // Only take 1 song name
switchMap( v =>
// Query across collectionGroup to get all instances.
this.db.collectionGroup('songs', ref =>
ref.where('id', '==', v.id)).get()
),
switchMap( v => {
// map the Song to the parent Dance, return the Dance ids
const obs: string[] = [];
v.docs.forEach(docRef => {
// We invoke parent twice to go from doc->collection->doc
obs.push(docRef.ref.parent.parent.id);
});
// Because we return an array here this one emit becomes N
return obs;
}),
// Firebase IN support up to 10 values so we partition the data to query the Dances
bufferCount(10),
mergeMap( v => { // query every partition in parallel
return this.db.collection('dances', ref => {
return ref.where( firebase.firestore.FieldPath.documentId(), 'in', v);
}).get();
}),
switchMap( v => {
// Almost there now just need to extract the data from the QuerySnapshots
const obs: Dance[] = [];
v.docs.forEach(docRef => {
obs.push({
...docRef.data(),
id: docRef.id
} as Dance);
});
return of(obs);
}),
// And finally we reduce the docs fetched into a single array.
reduce((acc, value) => acc.concat(value), []),
);
const parentDances = await dance$.toPromise();
I copy pasted my code and changed the variable names to yours, not sure if there are any errors, but it worked fine for me. Let me know if you find any errors or can suggest a better way to test it with maybe some mock firestore.
var songs = []
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
var songLength = querySnapshot.size
var i=0;
querySnapshot.forEach(function(doc) {
songs.push(doc.data())
i ++;
if(songLength===i){
console.log(songs
}
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
It could be better to use a flat data structure.
The docs specify the pros and cons of different data structures on this page.
Specifically about the limitations of structures with sub-collections:
You can't easily delete subcollections, or perform compound queries across subcollections.
Contrasted with the purported advantages of a flat data structure:
Root-level collections offer the most flexibility and scalability, along with powerful querying within each collection.
Related
I am trying to show to user items, he participated in (for example liked posts). Now I store in user document (in users collection) IDs of documents from another collection (posts) and when I want to show them in recycler view firstly I get IDs from user document. Then I get all posts by IDs. Is there any workaround, where I would be able to store user ID in subcollection of post document and then get query of all liked/commented/whatever posts by user? So user document will not have reference to post's IDs and in posts collection I am able to do something like:
Query ref = from db.collection("posts") get all posts where post.likedBy == user;
I do no like idea of putting all users who liked the post into post document - user downloads all ids.
posts (collection)
-postID (post document)
-authorID, ... (fields)
users (collections)
-userID (user document)
-string[] idsOfPosts (fields)
You should use Subcollections as your data model.
Documents in subcollections can contain subcollections as well,
allowing you to further nest data. You can nest data up to 100 levels
deep.
Also you can use a collection group query to retrieve documents from a collection group instead of from a single collection. The link provides you with sample code snippets in different languages.
EDIT:
Based on the use case you have provided in the comments:
I would say the way you are describing your data model to get all posts liked by a user, it would need a query inside a query. Not sure if it's even feasible or efficient.
Here is my suggestion:
Build your data model similar to the following
This way running the following query (I'm using NodeJs) would give you all posts liked by user1.
let postsRef = db.collection('posts');
const user1 = postsRef.where('Liked', 'array-contains',
'user1');
let query1 = user1.get()
.then(snapshot => {
if (snapshot.empty) {
console.log('No matching documents.');
return;
}
snapshot.forEach(doc => {
console.log(doc.id, '=>', doc.data());
});
})
.catch(err => {
console.log('Error getting documents', err);
});
Output:
EDIT: (11/12/2019)
Based on what you have described in the comments, here is an idea that might solve your issue:
Instead of having a list of the Users who liked the post, you can have a reference to a Document that contains the list of users. You can reference to as many Documents as you wish.
Example:
The Documents can be even in a different Collection.
I am using Firebase function and Firebase Firestore to develope an API which will store users data.
I wanted to locate the documents using the properties stored in their field. This is the Firebase document which states how to achieve the same.
// Create a reference to the cities collection
var citiesRef = db.collection('cities');
// Create a query against the collection
var queryRef = citiesRef.where('state', '==', 'CA');
I wanted to handle two situations
Where there is no document with the present conditions
Where there are more than two documents with the present conditions
How could the above two situation be handled?
Following our "discussion" in the comments above, in a Cloud Function you could do as follows, using the QuerySnapshot returned by the get() method:
admin.firestore().collection("cities")
.where('state', '==', 'CA')
.get()
.then(querySnapshot => {
if (querySnapshot.size == 0) {
console.log("0 documents");
} else if (querySnapshot.size > 2) {
console.log("More than 2 documents");
}
});
As said, above, just be aware that this will cost a read for each document in the collection. In case you have a very large collection, you could write a Cloud Function that update a counter each time a doc is added/removed to/from the collection.
The accepted answer does not show how to extract the data from each document and imo is only half the answer. the following will get you iterating through every document and extracting the data.
db.collection("cities").get().then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
// doc.data() is never undefined for query doc snapshots
console.log(doc.id, " => ", doc.data());
});
});
Update on this question
Firestore launched another feature similar to the in query, the array-contains-any query. This feature allows you to perform array-contains queries against multiple values at the same time.
Use the array-contains-any operator to combine up to 10 array-contains clauses on the same field with a logical OR. An array-contains-any query returns documents where the given field is an array that contains one or more of the comparison values:
ref = ref.where('node_sku' , 'array-contains-any' , list);
Firestore - Array contains any
Question
I am trying to filter data with multiple where() methods using the array-contains operator, however I am having the following error:
An error occured: Error: 3 INVALID_ARGUMENT:
A maximum of 1 'ARRAY_CONTAINS' filter is allowed.
Basically, I have a collection containing the following docs:
product1
- node_sku ['sku1', 'sku2', 'sku3' ]
product2
- node_sku ['sku4', 'sku5', 'sku6' ]
Then I have my array with items I wish to find on the database:
const list = ['sku4', 'sku2']
I want to search for sku4 and sku2, if finds any returns the object.
My problem is that array-contains is not allowed to run more than once.
Here's the code I am using to filter:
const list = ['sku4', 'sku2'];
let database = firestore_db.collection('glasses');
let ref = database;
list.forEach( (val) => {
ref = ref.where('node_sku' , 'array-contains' , val);
});
ref.get()
.then( (snapshot) => glassesRequestComplete(snapshot, req, response))
.catch( (error) => glassesRequestError(error, response) );
As you notice, my problem is that I am looping through my list array, however my ref is throwing an error since I am firing 'array-contains' more than once.
I have also tried to compare against the array:
ref.where('node_sku' , 'array-contains' , list);
which does not return any , I believe array-contains does not compare against arrays, only strings/numbers/booleans maybe?
Does anybody know a solution for that without having to query them individually?
Thank you
One solution would be to store the 'tags' in an Object as its property names.
node_sku = {
"sku1": true,
"sku2": true,
...
"skun": true
}
Then you can create the AND WHERE clauses like this:
list.forEach( (val) => {
ref = ref.where(`node_sku.${val}` , '==' , true);
});
This approach is an answer to the question "without having to query them individually". However, as soon as you want ordering and pagination, you will run into another invisible brick wall of compound indices.
As you've found you can only have a single array-contains operation in a query. The idea of allowing multiple is currently not supported, but could be valid. So I recommend you file a feature request.
The only solution I can think of for your use-case right now, is to also include the combinations. So if you want to allow testing for the existing of two SKUs, you explode the array to also contain each combination (merging keys in lexicographical order):
product1
- node_sku ['sku1', 'sku2', 'sku3', 'sku1_sku2', 'sku1_sku3', 'sku2_sku3' ]
For two that might still be reasonable, but the combinatory explosion will make it infeasible beyond that.
Firestore launched another feature similar to the in query, the array-contains-any query. This feature allows you to perform array-contains queries against multiple values at the same time.
ref = ref.where('node_sku' , 'array-contains-any' , list);
Firestore - Array contains any
Use something like this :
ref.where("node_sku", "in", list).get()
.then((snapshot) => glassesRequestComplete(snapshot, req, response)).catch( (error) => glassesRequestError(error, response) );
My solution is to pull back all the items using 'array-contains-any', then use lodash _.intersection() to find the items that match all the terms. It works for single term searching as well.
const searchTerms = searchKey.split(" ");
const searchTermCount = searchTerms.length;
const productsRef = collection(this.db, "products");
const termsQuery = query(productsRef,
where("search", "array-contains-any", searchTerms),
orderBy("popularity", "desc"),
orderBy("name_lower"),
limit(resultLimit));
const snaps = await getDocs(termsQuery);
let products = [];
snaps.forEach((doc) => {
const p = doc.data();
const allTerms = _.intersection(p.search, searchTerms).length === searchTermCount;
if (allTerms)
products.push(p);
});
The structure of the table is:
chats
--> randomId
-->--> participants
-->-->--> 0: 'name1'
-->-->--> 1: 'name2'
-->--> chatItems
etc
What I am trying to do is query the chats table to find all the chats that hold a participant by a passed in username string.
Here is what I have so far:
subscribeChats(username: string) {
return this.af.database.list('chats', {
query: {
orderByChild: 'participants',
equalTo: username, // How to check if participants contain username
}
});
}
Your current data structure is great to look up the participants of a specific chat. It is however not a very good structure for looking up the inverse: the chats that a user participates in.
A few problems here:
you're storing a set as an array
you can only index on fixed paths
Set vs array
A chat can have multiple participants, so you modelled this as an array. But this actually is not the ideal data structure. Likely each participant can only be in the chat once. But by using an array, I could have:
participants: ["puf", "puf"]
That is clearly not what you have in mind, but the data structure allows it. You can try to secure this in code and security rules, but it would be easier if you start with a data structure that implicitly matches your model better.
My rule of thumb: if you find yourself writing array.contains(), you should be using a set.
A set is a structure where each child can be present at most once, so it naturally protects against duplicates. In Firebase you'd model a set as:
participants: {
"puf": true
}
The true here is really just a dummy value: the important thing is that we've moved the name to the key. Now if I'd try to join this chat again, it would be a noop:
participants: {
"puf": true
}
And when you'd join:
participants: {
"john": true,
"puf": true
}
This is the most direct representation of your requirement: a collection that can only contain each participant once.
You can only index known properties
With the above structure, you could query for chats that you are in with:
ref.child("chats").orderByChild("participants/john").equalTo(true)
The problem is that this requires you to define an index on `participants/john":
{
"rules": {
"chats": {
"$chatid": {
"participants": {
".indexOn": ["john", "puf"]
}
}
}
}
}
This will work and perform great. But now each time someone new joins the chat app, you'll need to add another index. That's clearly not a scaleable model. We'll need to change our data structure to allow the query you want.
Invert the index - pull categories up, flattening the tree
Second rule of thumb: model your data to reflect what you show in your app.
Since you are looking to show a list of chat rooms for a user, store the chat rooms for each user:
userChatrooms: {
john: {
chatRoom1: true,
chatRoom2: true
},
puf: {
chatRoom1: true,
chatRoom3: true
}
}
Now you can simply determine your list of chat rooms with:
ref.child("userChatrooms").child("john")
And then loop over the keys to get each room.
You'll like have two relevant lists in your app:
the list of chat rooms for a specific user
the list of participants in a specific chat room
In that case you'll also have both lists in the database.
chatroomUsers
chatroom1
user1: true
user2: true
chatroom2
user1: true
user3: true
userChatrooms
user1:
chatroom1: true
chatroom2: true
user2:
chatroom1: true
user2:
chatroom2: true
I've pulled both lists to the top-level of the tree, since Firebase recommends against nesting data.
Having both lists is completely normal in NoSQL solutions. In the example above we'd refer to userChatrooms as the inverted index of chatroomsUsers.
Cloud Firestore
This is one of the cases where Cloud Firestore has better support for this type of query. Its array-contains operator allows filter documents that have a certain value in an array, while arrayRemove allows you to treat an array as a set. For more on this, see Better Arrays in Cloud Firestore.
I am trying to understand how I can take a list of Ids, and then ask for Firebase to take each path and sort them by date and bring me back the top 20 all in one Firebase call. At the moment I am doing this by looping through the ids and grabbing each path - adding the items to a list then sorting them.
However this is inefficient especially as the list gets bigger.
My data looks something like this:
follows: {
UserId :{
projId1: true,
projId2: true
}
}
projects: {
projId1 :{
title: SomeText,
date: TimeStamp
}
}
A Firebase query runs at one specific location and it can only order on data that exists at fixed paths under the children of that location. There is no way to sort a Firebase query on data that exists at a different path.
So in your case, if you are loading data from /follows/$uid, there is not way to order the results on data from the /projects nodes.
That leaves you with two options:
sort the data on the client
at the sort properties to the follows children
Sorting data on the client is a valid option as long as your data set is not too large and you're not filtering too much.
But to allow sorting on the server, you'll have to duplicate the relevant properties under the node where you query:
follows: {
UserId :{
projId1: TimeStamp,
projId2: TimeStamp
}
}
projects: {
projId1 :{
title: SomeText,
date: TimeStamp
}
}
With this structure you can order then projects on timestamp with
Query query = database.child(user.getUid()).orderByValue();
query.addChildEventListener(...