For developing an Android app in Kotlin, I am trying to find a way to retrieve a document from a Firestore collection based on a certain index and I am not sure how to go about it. For example, if my collection has 50 documents, what would be the best way to retrieve the 10th document so that I can populate text views on my app based on the field values of that document?
Thank you for taking the time to answer my question.
When you add documents to Firestore using the CollectionReference#add() method, you'll see a random document ID that is assigned automatically for each document. If you check the Firebase Console or you try to query a collection programmatically, you will not find any order for the containing documents. This means that there is no index involved. So it's not correct to say, hey Firestore, give me the 10th document, because when you query a collection of 50 documents, the 10th document might be the one you are interested in, but if you add/delete documents, the 10th document won't be the same.
You might think, ok, I can provide my own document IDs, starting from 0 - 49, and when I query the collection I can order them by their document IDs. But, this is not an option, since the document IDs are always strings, and when you order strings, the order is lexicographical.
The best solution I can think of, if you need to get a particular document from a collection, is to add a specific field in that document, so it can be identified very easily. For instance, you can add a particular name or ID for each document, and then you can perform a query that looks like this:
rootRef.collection("users").whereEqualTo("name", "Bhavey")
In this way, you can simply identify each one of the documents in your collection.
To accomplish this, will require some setup.
let's say you have a collection of documents and you want to manage them and find the 10th document, Firestore generates unique hash id's that are based on the timestamp making them have an inherent order based on the time they were generated.
All you need to do is manage a dedicated document in the parent of this collection that contains an array of all document id's in the subcollection.
reading this document and it's array would get you an ordered list of documents without incurring the additional reads of the documents as other methods, even listing the ID's requires 1 read each.
on the successful operation of it being added, you can use a Cloud Function onCreate trigger to push the document id to the master array and the same with onDelete. But if cloud Functions are not available or you want to brute force it from the client, you can do the following:
db.collection("users")
.document("uid")
.collection("posts")
.add(city)
.addOnSuccessListener { documentReference ->
db.collection("users")
.document("uid")
.update("index", FieldValue.arrayUnion(documentReference.id))
.addOnSuccessListener { Log.d(TAG, "Index Added!") }
.addOnFailureListener { e -> Log.w(TAG, "Index Error", e) }
}
.addOnFailureListener { e -> Log.w(TAG, "Error source document", e) }
Cloud Functions sources:
trigger_a_function_when_a_new_document_is_created
trigger_a_function_when_a_document_is_deleted
*note: to scale this past 1mb firestore document limits - you will need multiple index documents with a counter and to manage this with Cloud Functions. The end result should double your writes without crippling your reads
I am trying to show to user items, he participated in (for example liked posts). Now I store in user document (in users collection) IDs of documents from another collection (posts) and when I want to show them in recycler view firstly I get IDs from user document. Then I get all posts by IDs. Is there any workaround, where I would be able to store user ID in subcollection of post document and then get query of all liked/commented/whatever posts by user? So user document will not have reference to post's IDs and in posts collection I am able to do something like:
Query ref = from db.collection("posts") get all posts where post.likedBy == user;
I do no like idea of putting all users who liked the post into post document - user downloads all ids.
posts (collection)
-postID (post document)
-authorID, ... (fields)
users (collections)
-userID (user document)
-string[] idsOfPosts (fields)
You should use Subcollections as your data model.
Documents in subcollections can contain subcollections as well,
allowing you to further nest data. You can nest data up to 100 levels
deep.
Also you can use a collection group query to retrieve documents from a collection group instead of from a single collection. The link provides you with sample code snippets in different languages.
EDIT:
Based on the use case you have provided in the comments:
I would say the way you are describing your data model to get all posts liked by a user, it would need a query inside a query. Not sure if it's even feasible or efficient.
Here is my suggestion:
Build your data model similar to the following
This way running the following query (I'm using NodeJs) would give you all posts liked by user1.
let postsRef = db.collection('posts');
const user1 = postsRef.where('Liked', 'array-contains',
'user1');
let query1 = user1.get()
.then(snapshot => {
if (snapshot.empty) {
console.log('No matching documents.');
return;
}
snapshot.forEach(doc => {
console.log(doc.id, '=>', doc.data());
});
})
.catch(err => {
console.log('Error getting documents', err);
});
Output:
EDIT: (11/12/2019)
Based on what you have described in the comments, here is an idea that might solve your issue:
Instead of having a list of the Users who liked the post, you can have a reference to a Document that contains the list of users. You can reference to as many Documents as you wish.
Example:
The Documents can be even in a different Collection.
I have a firestore collection of the following documents:
[
{
start: { geohash: 'u3qchtmpuy2d' },
destination: { geohash: 'u3qcjvxfh9cs' },
timestamp: '28 June 2019 at 20:24:00 UTC+2',
...
}
]
and I tried to query it like this (its Firestore Web SDK)
// 5-characters geohash is 4.89km × 4.89km
const start = 'u3qch';
const destination = 'u3qcj';
const timestamps = {
min: firestore.Timestamp.fromDate(
moment(someDateTime).subtract(10, 'minutes').toDate()
),
max: firestore.Timestamp.fromDate(
moment(someDateTime).add(10, 'minutes').toDate(),
),
};
firestore
.collection('deliveries')
.where('start.geohash', '>=', start)
.where('start.geohash', '<', geohashEnd(start))
.where('destination.geohash', '>=', destination)
.where('destination.geohash', '<', geohashEnd(destination))
.where('timestamp', '>=', timestamps.min)
.where('timestamp', '<', timestamps.max)
.get()
the combination of >= and < is from Firestore: query documents by startsWith a string, hence geoHashEnd() function (out of the scope of this question).
It resulted in the following error:
FirebaseError: Invalid query. All where filters with an inequality (<, <=, >, or >=) must be on the same field. But you have inequality filters on 'start.geohash' and 'destination.geohash'
My question: what is the best approach to query my firestore collection by two geohash strings and an additional field, at once?
According to the official documentation regarding query limitations:
Query limitations
Cloud Firestore does not support the following types of queries:
Queries with range filters on different fields, as described in the previous section.
So as you can see, Cloud Firestore can only do a range filter on a single field and not on multiple fields as you intended to do. The most reasonable explanation is that Firestore cannot guarantee its performance in this case. Firestore must be able to return all results of a query in a single stream.
To solve this, you'll have to query the database twice and combine the results of those queries client side. It's not perfect, since you need to query twice but I think it will do the trick.
Please also note, that querying using range filters on geohashes will not return very accurate results. For that, I recommend you see Frank van Puffelen's remarkable video regarding this topic:
https://www.youtube.com/watch?v=mx1mMdHBi5Q
I am using Firebase function and Firebase Firestore to develope an API which will store users data.
I wanted to locate the documents using the properties stored in their field. This is the Firebase document which states how to achieve the same.
// Create a reference to the cities collection
var citiesRef = db.collection('cities');
// Create a query against the collection
var queryRef = citiesRef.where('state', '==', 'CA');
I wanted to handle two situations
Where there is no document with the present conditions
Where there are more than two documents with the present conditions
How could the above two situation be handled?
Following our "discussion" in the comments above, in a Cloud Function you could do as follows, using the QuerySnapshot returned by the get() method:
admin.firestore().collection("cities")
.where('state', '==', 'CA')
.get()
.then(querySnapshot => {
if (querySnapshot.size == 0) {
console.log("0 documents");
} else if (querySnapshot.size > 2) {
console.log("More than 2 documents");
}
});
As said, above, just be aware that this will cost a read for each document in the collection. In case you have a very large collection, you could write a Cloud Function that update a counter each time a doc is added/removed to/from the collection.
The accepted answer does not show how to extract the data from each document and imo is only half the answer. the following will get you iterating through every document and extracting the data.
db.collection("cities").get().then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
// doc.data() is never undefined for query doc snapshots
console.log(doc.id, " => ", doc.data());
});
});
I thought I read that you can query subcollections with the new Firebase Firestore, but I don't see any examples. For example I have my Firestore setup in the following way:
Dances [collection]
danceName
Songs [collection]
songName
How would I be able to query "Find all dances where songName == 'X'"
Update 2019-05-07
Today we released collection group queries, and these allow you to query across subcollections.
So, for example in the web SDK:
db.collectionGroup('Songs')
.where('songName', '==', 'X')
.get()
This would match documents in any collection where the last part of the collection path is 'Songs'.
Your original question was about finding dances where songName == 'X', and this still isn't possible directly, however, for each Song that matched you can load its parent.
Original answer
This is a feature which does not yet exist. It's called a "collection group query" and would allow you query all songs regardless of which dance contained them. This is something we intend to support but don't have a concrete timeline on when it's coming.
The alternative structure at this point is to make songs a top-level collection and make which dance the song is a part of a property of the song.
UPDATE
Now Firestore supports array-contains
Having these documents
{danceName: 'Danca name 1', songName: ['Title1','Title2']}
{danceName: 'Danca name 2', songName: ['Title3']}
do it this way
collection("Dances")
.where("songName", "array-contains", "Title1")
.get()...
#Nelson.b.austin Since firestore does not have that yet, I suggest you to have a flat structure, meaning:
Dances = {
danceName: 'Dance name 1',
songName_Title1: true,
songName_Title2: true,
songName_Title3: false
}
Having it in that way, you can get it done:
var songTitle = 'Title1';
var dances = db.collection("Dances");
var query = dances.where("songName_"+songTitle, "==", true);
I hope this helps.
UPDATE 2019
Firestore have released Collection Group Queries. See Gil's answer above or the official Collection Group Query Documentation
Previous Answer
As stated by Gil Gilbert, it seems as if collection group queries is currently in the works. In the mean time it is probably better to use root level collections and just link between these collection using the document UID's.
For those who don't already know, Jeff Delaney has some incredible guides and resources for anyone working with Firebase (and Angular) on AngularFirebase.
Firestore NoSQL Relational Data Modeling - Here he breaks down the basics of NoSQL and Firestore DB structuring
Advanced Data Modeling With Firestore by Example - These are more advanced techniques to keep in the back of your mind. A great read for those wanting to take their Firestore skills to the next level
What if you store songs as an object instead of as a collection? Each dance as, with songs as a field: type Object (not a collection)
{
danceName: "My Dance",
songs: {
"aNameOfASong": true,
"aNameOfAnotherSong": true,
}
}
then you could query for all dances with aNameOfASong:
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
querySnapshot.forEach(function(doc) {
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
NEW UPDATE July 8, 2019:
db.collectionGroup('Songs')
.where('songName', isEqualTo:'X')
.get()
I have found a solution.
Please check this.
var museums = Firestore.instance.collectionGroup('Songs').where('songName', isEqualTo: "X");
museums.getDocuments().then((querySnapshot) {
setState(() {
songCounts= querySnapshot.documents.length.toString();
});
});
And then you can see Data, Rules, Indexes, Usage tabs in your cloud firestore from console.firebase.google.com.
Finally, you should set indexes in the indexes tab.
Fill in collection ID and some field value here.
Then Select the collection group option.
Enjoy it. Thanks
You can always search like this:-
this.key$ = new BehaviorSubject(null);
return this.key$.switchMap(key =>
this.angFirestore
.collection("dances").doc("danceName").collections("songs", ref =>
ref
.where("songName", "==", X)
)
.snapshotChanges()
.map(actions => {
if (actions.toString()) {
return actions.map(a => {
const data = a.payload.doc.data() as Dance;
const id = a.payload.doc.id;
return { id, ...data };
});
} else {
return false;
}
})
);
Query limitations
Cloud Firestore does not support the following types of queries:
Queries with range filters on different fields.
Single queries across multiple collections or subcollections. Each query runs against a single collection of documents. For more
information about how your data structure affects your queries, see
Choose a Data Structure.
Logical OR queries. In this case, you should create a separate query for each OR condition and merge the query results in your app.
Queries with a != clause. In this case, you should split the query into a greater-than query and a less-than query. For example, although
the query clause where("age", "!=", "30") is not supported, you can
get the same result set by combining two queries, one with the clause
where("age", "<", "30") and one with the clause where("age", ">", 30).
I'm working with Observables here and the AngularFire wrapper but here's how I managed to do that.
It's kind of crazy, I'm still learning about observables and I possibly overdid it. But it was a nice exercise.
Some explanation (not an RxJS expert):
songId$ is an observable that will emit ids
dance$ is an observable that reads that id and then gets only the first value.
it then queries the collectionGroup of all songs to find all instances of it.
Based on the instances it traverses to the parent Dances and get their ids.
Now that we have all the Dance ids we need to query them to get their data. But I wanted it to perform well so instead of querying one by one I batch them in buckets of 10 (the maximum angular will take for an in query.
We end up with N buckets and need to do N queries on firestore to get their values.
once we do the queries on firestore we still need to actually parse the data from that.
and finally we can merge all the query results to get a single array with all the Dances in it.
type Song = {id: string, name: string};
type Dance = {id: string, name: string, songs: Song[]};
const songId$: Observable<Song> = new Observable();
const dance$ = songId$.pipe(
take(1), // Only take 1 song name
switchMap( v =>
// Query across collectionGroup to get all instances.
this.db.collectionGroup('songs', ref =>
ref.where('id', '==', v.id)).get()
),
switchMap( v => {
// map the Song to the parent Dance, return the Dance ids
const obs: string[] = [];
v.docs.forEach(docRef => {
// We invoke parent twice to go from doc->collection->doc
obs.push(docRef.ref.parent.parent.id);
});
// Because we return an array here this one emit becomes N
return obs;
}),
// Firebase IN support up to 10 values so we partition the data to query the Dances
bufferCount(10),
mergeMap( v => { // query every partition in parallel
return this.db.collection('dances', ref => {
return ref.where( firebase.firestore.FieldPath.documentId(), 'in', v);
}).get();
}),
switchMap( v => {
// Almost there now just need to extract the data from the QuerySnapshots
const obs: Dance[] = [];
v.docs.forEach(docRef => {
obs.push({
...docRef.data(),
id: docRef.id
} as Dance);
});
return of(obs);
}),
// And finally we reduce the docs fetched into a single array.
reduce((acc, value) => acc.concat(value), []),
);
const parentDances = await dance$.toPromise();
I copy pasted my code and changed the variable names to yours, not sure if there are any errors, but it worked fine for me. Let me know if you find any errors or can suggest a better way to test it with maybe some mock firestore.
var songs = []
db.collection('Dances')
.where('songs.aNameOfASong', '==', true)
.get()
.then(function(querySnapshot) {
var songLength = querySnapshot.size
var i=0;
querySnapshot.forEach(function(doc) {
songs.push(doc.data())
i ++;
if(songLength===i){
console.log(songs
}
console.log(doc.id, " => ", doc.data());
});
})
.catch(function(error) {
console.log("Error getting documents: ", error);
});
It could be better to use a flat data structure.
The docs specify the pros and cons of different data structures on this page.
Specifically about the limitations of structures with sub-collections:
You can't easily delete subcollections, or perform compound queries across subcollections.
Contrasted with the purported advantages of a flat data structure:
Root-level collections offer the most flexibility and scalability, along with powerful querying within each collection.