How to batch read set of documents in Firestore? [duplicate] - android

I am wondering if it's possible to get multiple documents by a list of ids in one round trip (network call) to the Firestore database.

if you're within Node:
https://github.com/googleapis/nodejs-firestore/blob/master/dev/src/index.ts#L978
/**
* Retrieves multiple documents from Firestore.
*
* #param {...DocumentReference} documents - The document references
* to receive.
* #returns {Promise<Array.<DocumentSnapshot>>} A Promise that
* contains an array with the resulting document snapshots.
*
* #example
* let documentRef1 = firestore.doc('col/doc1');
* let documentRef2 = firestore.doc('col/doc2');
*
* firestore.getAll(documentRef1, documentRef2).then(docs => {
* console.log(`First document: ${JSON.stringify(docs[0])}`);
* console.log(`Second document: ${JSON.stringify(docs[1])}`);
* });
*/
This is specifically for the server SDK
UPDATE: Cloud Firestore Now Supports IN Queries!
myCollection.where(firestore.FieldPath.documentId(), 'in', ["123","456","789"])

In practise you would use firestore.getAll like this
async getUsers({userIds}) {
const refs = userIds.map(id => this.firestore.doc(`users/${id}`))
const users = await this.firestore.getAll(...refs)
console.log(users.map(doc => doc.data()))
}
or with promise syntax
getUsers({userIds}) {
const refs = userIds.map(id => this.firestore.doc(`users/${id}`))
this.firestore.getAll(...refs).then(users => console.log(users.map(doc => doc.data())))
}

They have just announced this functionality, https://firebase.googleblog.com/2019/11/cloud-firestore-now-supports-in-queries.html .
Now you can use queries like, but mind that the input size can't be greater than 10.
userCollection.where('uid', 'in', ["1231","222","2131"])

With Firebase Version 9 (Dec, 2021 Update):
You can get multiple documents by multiple ids in one round-trip using "documentId()" and "in" with "where" clause:
import {
query,
collection,
where,
documentId,
getDocs
} from "firebase/firestore";
const q = query(
collection(db, "products"),
where(documentId(), "in",
[
"8AVJvG81kDtb9l6BwfCa",
"XOHS5e3KY9XOSV7YYMw2",
"Y2gkHe86tmR4nC5PTzAx"
]
),
);
const productsDocsSnap = await getDocs(q);
productsDocsSnap.forEach((doc) => {
console.log(doc.data()); // "doc1", "doc2" and "doc3"
});

You could use a function like this:
function getById (path, ids) {
return firestore.getAll(
[].concat(ids).map(id => firestore.doc(`${path}/${id}`))
)
}
It can be called with a single ID:
getById('collection', 'some_id')
or an array of IDs:
getById('collection', ['some_id', 'some_other_id'])

No, right now there is no way to batch multiple read requests using the Cloud Firestore SDK and therefore no way to guarantee that you can read all of the data at once.
However as Frank van Puffelen has said in the comments above this does not mean that fetching 3 documents will be 3x as slow as fetching one document. It is best to perform your own measurements before reaching a conclusion here.

If you are using flutter, you can do the following:
Firestore.instance.collection('your_collection_name')
.where(FieldPath.documentId, whereIn:["list", "of", "document", "ids"])
.getDocuments();
This will return a Future containing List<DocumentSnapshot> which you can iterate as you feel fit.

Surely the best way to do this is by implementing the actual query of Firestore in a Cloud Function? There would then only be a single round trip call from the client to Firebase, which seems to be what you're asking for.
You really want to be keeping all of your data access logic like this server side anyway.
Internally there will likely be the same number of calls to Firebase itself, but they would all be across Google's super-fast interconnects, rather than the external network, and combined with the pipelining which Frank van Puffelen has explained, you should get excellent performance from this approach.

You can perform an IN query with the document IDs (up to ten):
import {
query,
collection,
where,
getDocs,
documentId,
} from 'firebase/firestore';
export async function fetchAccounts(
ids: string[]
) {
// use lodash _.chunk, for example
const result = await Promise.all(
chunk(ids, 10).map(async (chunkIds) => {
const accounts = await getDocs(
query(
collection(firestore, 'accounts'),
where(documentId(), 'in', chunkIds)
));
return accounts.docs.filter(doc => doc.exists()).map(doc => doc.data());
})
);
return result.flat(1);
}

Here's how you would do something like this in Kotlin with the Android SDK.
May not necessarily be in one round trip, but it does effectively group the result and avoid many nested callbacks.
val userIds = listOf("123", "456")
val userTasks = userIds.map { firestore.document("users/${it!!}").get() }
Tasks.whenAllSuccess<DocumentSnapshot>(userTasks).addOnSuccessListener { documentList ->
//Do what you need to with the document list
}
Note that fetching specific documents is much better than fetching all documents and filtering the result. This is because Firestore charges you for the query result set.

For some who are stucked in same problem
here is a sample code:
List<String> documentsIds = {your document ids};
FirebaseFirestore.getInstance().collection("collection_name")
.whereIn(FieldPath.documentId(), documentsIds).get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
#Override
public void onComplete(#NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
for (DocumentSnapshot document : Objects.requireNonNull(task.getResult())) {
YourClass object = document.toObject(YourClass.class);
// add to your custom list
}
}
}
}).addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
e.printStackTrace();
}
});

For the ones who want to do it using Angular, here is an example:
First some library imports are needed: (must be preinstalled)
import * as firebase from 'firebase/app'
import { AngularFirestore, AngularFirestoreCollection } from '#angular/fire/firestore'
Some configuration for the collection:
yourCollection: AngularFirestoreCollection;
constructor(
private _db : AngularFirestore,
) {
// this is your firestore collection
this.yourCollection = this._db.collection('collectionName');
}
Here is the method to do the query: ('products_id' is an Array of ids)
getProducts(products_ids) {
var queryId = firebase.firestore.FieldPath.documentId();
this.yourCollection.ref.where(queryId, 'in', products_ids).get()
.then(({ docs }) => {
console.log(docs.map(doc => doc.data()))
})
}

I hope this helps you, it works for me.
getCartGoodsData(id) {
const goodsIDs: string[] = [];
return new Promise((resolve) => {
this.fs.firestore.collection(`users/${id}/cart`).get()
.then(querySnapshot => {
querySnapshot.forEach(doc => {
goodsIDs.push(doc.id);
});
const getDocs = goodsIDs.map((id: string) => {
return this.fs.firestore.collection('goods').doc(id).get()
.then((docData) => {
return docData.data();
});
});
Promise.all(getDocs).then((goods: Goods[]) => {
resolve(goods);
});
});
});
}

Yes, it is possible. Sample in .NET SDK for Firestore:
/*List of document references, for example:
FirestoreDb.Collection(ROOT_LEVEL_COLLECTION).Document(DOCUMENT_ID);*/
List<DocumentReference> docRefList = YOUR_DOCUMENT_REFERENCE_LIST;
// Required fields of documents, not necessary while fetching entire documents
FieldMask fieldMask = new FieldMask(FIELD-1, FIELD-2, ...);
// With field mask
List<DocumentSnapshot> documentSnapshotsMasked = await FirestoreDb.GetAllSnapshotsAsync(docRefList, fieldMask);
// Without field mask
List<DocumentSnapshot>documentSnapshots = await FirestoreDb.GetAllSnapshotsAsync(docRefList);
Documentation in .NET:
Get all snapshots
Field mask

This doesn't seem to be possible in Firestore at the moment. I don't understand why Alexander's answer is accepted, the solution he proposes just returns all the documents in the "users" collection.
Depending on what you need to do, you should look into duplicating the relevant data you need to display and only request a full document when needed.

if you are using the python firebase admin sdk this is how you query for multiple documents using their uids
from firebase_admin import firestore
import firebase_admin
from google.cloud.firestore_v1.field_path import FieldPath
app = firebase_admin.initialize_app(cred)
client = firestore.client(app)
collection_ref = client.collection('collection_name')
query = collection_ref.where(FieldPath.document_id(), 'in', listOfIds)
docs = query.get()
for doc in docs:
print(doc.id, doc.to_dict())
Instead of importing FieldPath you can also simply use the string __name__. Now your query will be collection_ref.where('__name__', 'in', listOfIds)

The best you can do is not use Promise.all as your client then must wait for .all the reads before proceeding.
Iterate the reads and let them resolve independently. On the client side, this probably boils down to the UI having several progress loader images resolve to values independently. However, this is better than freezing the whole client until .all the reads resolve.
Therefore, dump all the synchronous results to the view immediately, then let the asynchronous results come in as they resolve, individually. This may seem like petty distinction, but if your client has poor Internet connectivity (like I currently have at this coffee shop), freezing the whole client experience for several seconds will likely result in a 'this app sucks' experience.

Related

Flutter : check if there's specific data in the subcollection

I'm creating a post that uploads data to Firestore's sub-collection and brings it up to MyBookmark page. It's good to create a sub-collection and upload data simply. And now I'd like to add a 'data duplication prevention' function here.
If the post is already saved in the bookmark, should not upload it.
For this purpose, I would like to check if the post is already in the collection when I press the bookmark button.
IconButton(
onPressed: () async {
//get userModel
UserModelState _userModelstate =
Provider.of<UserModelState>(context, listen: false);
//=========================================
//duplication data test
DocumentReference bookmarkRef = Firestore.instance
.collection(COLLECTION_USERS)
.document(_userModelstate.userModel.userKey)
.collection(COLLECTION_BOOKMARk)
// .where(KEY_BOOKMARK_PRODUCTKEY, isEqualTo: productKey)
.document();
DocumentSnapshot bookmarkSnapshot = await bookmarkRef.get();
//test (return "No exist")
if(bookmarkSnapshot.exists) {
print("Yes exist");
} else {
print("No exist");
}
I tried writing a code to check if there was data in the collection, but it is always printed as "No exist".
How can I confirm the existence of a specific document in collection?
Thank you.
If the productKey is supposed to be unique in the Bookmark collection of the user, consider using the productKey as the document ID. Since document IDs are by definition unique within their collection, using them guarantees unique product keys without you having to write any code for it.
That said, you current code can't work because you call document(). Whenever you call document() without any parameters, it generates a reference to a new unique document. And since you immediately call get() on that reference, the document will (also: by definition) not exist yet.
To check if a document with a specific product ID exist, you will need to run a query:
CollectionReference bookmarksRef = Firestore.instance
.collection(COLLECTION_USERS)
.document(_userModelstate.userModel.userKey)
.collection(COLLECTION_BOOKMARk);
Query bookmarkQuery = bookmarksRef.where(KEY_BOOKMARK_PRODUCTKEY, isEqualTo: productKey);
QuerySnapshot bookmarkSnapshot = await bookmarkQuery.get();
if (bookmarkSnapshot.size > 0) {
print("product key already in use");
}

How to retrieve data from firebase realtime database based on child value

Hi I am pretty new to Firebase real time database and this is my first project. Sorry if this is a stupid question.
I am saving my data as follows.
firebase database structure:
Now I want to retrieve all parent chat ids on which the student is participating, using the student_id variable.
I tried as per this SO question and this structure database and retrieve data documentation, but its not retrieving values. Anybody have an idea?
I would suggest saving the chatroom IDs your students are in in a separate location. For example:
Path:
“/users/$uid/chatrooms”
Data:
{
0: 350,
1: 423
}
Thus you could retrieve the chat room ids first before getting the chatroom data.
import { initializeApp } from “firebase”;
import { getDatabase, get, set, ref } from “firebase/database”;
const userChatroomIdsRef = ref(db, ‘/users/${uid}/chatrooms‘);
get(userChatroomIdsRef).then(result => {
const chatroomIds = result.val();
if (!(chatroomIds && chatroomIds instanceof Array)) return; // firebase will return null if its an empty array.
const getChatroomInfoPromises = chatroomIds.map(id => get(ref(db, ‘/chat/${id}/${uid}’)).then(result => result.val());
Promise.all(getChatroomInfoPromises).then(chatroomInfoArray => { YOUR LOGIC HERE });
});
Removing/adding students from/to chatrooms would now be simple as you could just change the array of chatroomIds.
const userChatroomIdsRef = ref(db, ‘/users/${uid}/chatrooms‘);
get(userChatroomIdsRef).then(result => {
const oldIds = result.val();
const newChatroomIds = oldIds.filter(id => id !== ID TO DELETE);
return set(userChatroomIdsRef, newChatroomIds)
});
This is of course assuming that you know the uid of your student_id. If you do not know what uid each student_id has, you must must store a reference. I would suggest saving all student info in the “/users/$uid/” directory. Here you could save the studentId so you can programmatically use it.
In all other firebase logic I would try to use the native firebase uid for querying. This will make your life easier.
It’s always good the keep information organized on the database so your logic is simple.
Please check my code for syntax errors; I wrote this on an iPhone.

Firebase - query by grandchild key [duplicate]

Given the data structure below in firebase, i want to run a query to retrieve the blog 'efg'. I don't know the user id at this point.
{Users :
"1234567": {
name: 'Bob',
blogs: {
'abc':{..},
'zyx':{..}
}
},
"7654321": {
name: 'Frank',
blogs: {
'efg':{..},
'hij':{..}
}
}
}
The Firebase API only allows you to filter children one level deep (or with a known path) with its orderByChild and equalTo methods.
So without modifying/expanding your current data structure that just leaves the option to retrieve all data and filter it client-side:
var ref = firebase.database().ref('Users');
ref.once('value', function(snapshot) {
snapshot.forEach(function(userSnapshot) {
var blogs = userSnapshot.val().blogs;
var daBlog = blogs['efg'];
});
});
This is of course highly inefficient and won't scale when you have a non-trivial number of users/blogs.
So the common solution to that is to a so-called index to your tree that maps the key that you are looking for to the path where it resides:
{Blogs:
"abc": "1234567",
"zyx": "1234567",
"efg": "7654321",
"hij": "7654321"
}
Then you can quickly access the blog using:
var ref = firebase.database().ref();
ref.child('Blogs/efg').once('value', function(snapshot) {
var user = snapshot.val();
ref.child('Blogs/'+user+'/blogs').once('value', function(blogSnapshot) {
var daBlog = blogSnapshot.val();
});
});
You might also want to reconsider if you can restructure your data to better fit your use-case and Firebase's limitations. They have some good documentation on structuring your data, but the most important one for people new to NoSQL/hierarchical databases seems to be "avoid building nests".
Also see my answer on Firebase query if child of child contains a value for a good example. I'd also recommend reading about many-to-many relationships in Firebase, and this article on general NoSQL data modeling.
Given your current data structure you can retrieve the User that contains the blog post you are looking for.
const db = firebase.database()
const usersRef = db.ref('users')
const query = usersRef.orderByChild('blogs/efg').limitToLast(1)
query.once('value').then((ss) => {
console.log(ss.val()) //=> { '7654321': { blogs: {...}}}
})
You need to use limitToLast since Objects are sorted last when using orderByChild docs.
It's actually super easy - just use foreslash:
db.ref('Users').child("userid/name")
db.ref('Users').child("userid/blogs")
db.ref('Users').child("userid/blogs/abc")
No need of loops or anything more.

Firestore update only one field

I have a database. The sequence is: collections - document - hashmaps. For example:
users - the name of the collection
users.uid - the name of the document
Hashmap the document consists of a lot of hashmaps user data
Hashmap
Hashmap
etc
Hashmap: the name of user is the key and telephone, location etc are the values. I need to update only one field(location) for one username, but can't understand how do this?
I tried the next way (update phone number for alex):
User user = new User();
user.setPhone(131902331);
Map<String,RealmObject> userMap = new HashMap<>();
userMap.put("alex",user);
mFirebaseFirestore
.collection("users")
.document(mFirebaseAuth.getUid())
.set(userMap, SetOptions.merge())
.addOnSuccessListener(new OnSuccessListener<Void>() {
#Override
public void onSuccess(Void aVoid) {
LOG.info("Success");
}
})
.addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
LOG.error("Failure "+e.toString());
}
});
What I am doing wrong?
I know it's almost a year old question, but this might help someone. Use dot notation
db.collection("users")
.document("frank")
.update({
"age": 13,
"favorites.color": "Red"
});
When you call set() on a document, the existing contents of that documents are replaced with the data you pass in.
If you want to only update the values of the field you specify in a map, use update():
mFirebaseFirestore
.collection("users")
.document(mFirebaseAuth.getUid())
.update(userMap)
See the Firestore documentation on updating a document.
Please note that as of 2021, the syntax in MrAleister's answer has changed a bit if using firebase-firestore-ktx (see documentation). I did not edit answer because perhaps it is still valid for people using other versions of the library.
// Assume the document contains:
// {
// name: "Frank",
// favorites: { food: "Pizza", color: "Blue", subject: "recess" }
// age: 12
// }
//
// To update age and favorite color:
db.collection("users").document("frank")
.update(mapOf(
"age" to 13,
"favorites.color" to "Red"
))
I'm finding some odd notation in some of these answers. Maybe firestore parameters were updated? At any rate, I wanted to post an answer of my own.
If you want to update a single element in your document, do so with the below code:
db.collection("users").document("frank")
.update("age", 13);
The update() parameter accepts as many key/value pairs as you need, which are not separated by a colon, and are NOT put in some sort of array notation { } as shown in some of these answers. They are simply added in and separated by commas. See below:
.update( key , value , additionalkey, additionalvalue)
After update(), feel free to add your OnSuccess/OnFailure listeners as needed.
Dynamic nested reference:
def nest_update(new, parent):
return {parent+'.'+new: val for new, val in list(new.items())}
Usage:
old = {'nested': {'a': 123,'b': 456, 'c': 789}}
new = {'b': 0}
print('\nOld: ', old)
print('New: ', nest_update(new, 'nested'))
print('New should be: ', {'nested.b': 0})
# OUTPUT
Old: {'nested': {'a': 123, 'b': 456, 'c': 789}}
New: {'nested.b': 0}
New should be: {'nested.b': 0}

How to add array value in Firebase

I want to add some certain data to a Firebase as arrays. Example:
groups : ['a','b','c']
How can I add and read data in Firebase from Android?
When you have a structure like that, you actually shouldn't be using an array to model it. It seems much more like a set in my eyes.
In the Firebase Database sets are best modeled as keys, since that automatically guarantees that items are unique. So your structure then becomes:
groups: {
"a": true,
"b": true,
"c": true
}
The true values are just markers, since Firebase won't allow you to store keys without a value.
Now to add a group to this, you'd use Firebase's setValue() function:
DatabaseReference root = FirebaseDatabase.getInstance().reference();
DatabaseReference groupsRef = root.child("groups");
groupsRef.child("d").setValue(true);
From the documentation:
setValue() - Record or change exists values
If you want to only append datas, you can to use updateChildren().
In Java, if we know that the data is array-like, it can be cast as a List:
Firebase julieRef = new Firebase("https://SampleChat.firebaseIO-demo.com/users/julie/");
julieRef.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot snapshot) {
GenericTypeIndicator<List<String>> t = new GenericTypeIndicator?<List<String>>() {};
List messages = snapshot.getValue(t);
if( messages === null ) {
System.out.println('No messages');
}
else {
System.out.println("The first message is: " + messages.get(0) );
}
}
// onCancelled...
});
Check this best practices post from the Firebase Blog.

Categories

Resources