Develop Reference

How to get random and unique document from Google Cloud Firestore? [duplicate]

I have 1000 documents in a single collection in Cloud Firestore, is it possible to fetch random documents?
Say for example: Students is a collection in Firestore and I have 1000 students in that collection, my requirement is to pick 10 students randomnly on each call.

As per Alex's answer I got hint to get duplicate records from Firebase Firestore Database (Specially for small amount of data)
I got some problems in his question as follow:
It gives all the records same as randomNumber is not updated.
It may have duplicate records in final list even we update randomNumber everytime.
It may have duplicate records which we are already displaying.
I have updated answer as follow:
FirebaseFirestore database = FirebaseFirestore.getInstance();
CollectionReference collection = database.collection(VIDEO_PATH);
collection.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
#Override
public void onComplete(#NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
List<VideoModel> videoModelList = new ArrayList<>();
for (DocumentSnapshot document : Objects.requireNonNull(task.getResult())) {
VideoModel student = document.toObject(VideoModel.class);
videoModelList.add(student);
}
/* Get Size of Total Items */
int size = videoModelList.size();
/* Random Array List */
ArrayList<VideoModel> randomVideoModels = new ArrayList<>();
/* for-loop: It will loop all the data if you want
* RANDOM + UNIQUE data.
* */
for (int i = 0; i < size; i++) {
// Getting random number (inside loop just because every time we'll generate new number)
int randomNumber = new Random().nextInt(size);
VideoModel model = videoModelList.get(randomNumber);
// Check with current items whether its same or not
// It will helpful when you want to show related items excepting current item
if (!model.getTitle().equals(mTitle)) {
// Check whether current list is contains same item.
// May random number get similar again then its happens
if (!randomVideoModels.contains(model))
randomVideoModels.add(model);
// How many random items you want
// I want 6 items so It will break loop if size will be 6.
if (randomVideoModels.size() == 6) break;
}
}
// Bind adapter
if (randomVideoModels.size() > 0) {
adapter = new RelatedVideoAdapter(VideoPlayerActivity.this, randomVideoModels, VideoPlayerActivity.this);
binding.recyclerView.setAdapter(adapter);
}
} else {
Log.d("TAG", "Error getting documents: ", task.getException());
}
}
});
Hope this logic helps to all who has small amount of data and I don't think It will create any problem for 1000 to 5000 data.
Thank you.

Yes it is and to achieve this, please use the following code:
FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference studentsCollectionReference = rootRef.collection("students");
studentsCollectionReference.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
#Override
public void onComplete(#NonNull Task<QuerySnapshot> task) {
if (task.isSuccessful()) {
List<Student> studentList = new ArrayList<>();
for (DocumentSnapshot document : task.getResult()) {
Student student = document.toObject(Student.class);
studentList.add(student);
}
int studentListSize = studentList.size();
List<Students> randomStudentList = new ArrayList<>();
for(int i = 0; i < studentListSize; i++) {
Student randomStudent = studentList.get(new Random().nextInt(studentListSize));
if(!randomStudentList.contains(randomStudent)) {
randomStudentList.add(randomStudent);
if(randomStudentList.size() == 10) {
break;
}
}
}
} else {
Log.d(TAG, "Error getting documents: ", task.getException());
}
}
});
This is called the classic solution and you can use it for collections that contain only a few records but if you are afraid of getting huge number of reads then, I'll recommend you this second approach. This also involves a little change in your database by adding a new document that can hold an array with all student ids. So to get those random 10 students, you'll need to make only a get() call, which implies only a single read operation. Once you get that array, you can use the same algorithm and get those 10 random ids. Once you have those random ids, you can get the corresponding documents and add them to a list. In this way you perform only 10 more reads to get the actual random students. In total, there are only 11 document reads.
This practice is called denormalization (duplicating data) and is a common practice when it comes to Firebase. If you're new to NoSQL database, so for a better understanding, I recommend you see this video, Denormalization is normal with the Firebase Database. It's for Firebase realtime database but same principles apply to Cloud Firestore.
But rememebr, in the way you are adding the random products in this new created node, in the same way you need to remove them when there are not needed anymore.
To add a student id to an array simply use:
FieldValue.arrayUnion("yourArrayProperty")
And to remove a student id, please use:
FieldValue.arrayRemove("yourArrayProperty")
To get all 10 random students at once, you can use List<Task<DocumentSnapshot>> and then call Tasks.whenAllSuccess(tasks), as explained in my answer from this post:
Android Firestore convert array of document references to List<Pojo>

I faced a similar problem (I only needed to get one random document every 24 hours or when users refresh the page manually but you can apply this solution on your case as well) and what worked for me was the following:
Technique
Read a small list of documents for the first time, let's say from 1 to 10 documents (10 to 30 or 50 in your case).
Select random document(s) based on a randomly generated number(s) within the range of the list of documents.
Save the last id of the document you selected locally on the client device (maybe in shared preferences like I did).
if you want a new random document(s), you will use the saved document id to start the process again (steps 1 to 3) after the saved document id which will exclude all documents that appeared before.
Repeat the process until there are no more documents after the saved document id then start over again from the beginning assuming this is the first time you run this algorithm (by setting the saved document id to null and start the process again (steps 1 to 4).
Technique Pros and Cons
Pros:
You can determine the jump size each time you get a new random document(s).
No need to modify the original model class of your object.
No need to modify the database that you already have or designed.
No need to add a document in the collection and handle adding random id for each document when adding a new document to the collection like solution mentioned here.
No need to load a big list of documents to just get one document or small-sized list of documents,
Works well if you are using the auto-generated id by firestore (because the documents inside the collection are already slightly randomized)
Works well if you want one random document or a small-sized random list of documents.
Works on all platforms (including iOS, Android, Web).
Cons
Handle saving the id of the document to use in the next request of getting random document(s) (which is better than handling a new field in each document or handling adding ids for each document in the collection to a new document in the main collection)
May get some documents more than one time if the list is not large enough (in my case it wasn't a problem) and I didn't find any solution that is avoiding this case completely.
Implementation (kotlin on android):
var documentId = //get document id from shared preference (will be null if not set before)
getRandomDocument(documentId)
fun getRandomDocument(documentId: String?) {
if (documentId == null) {
val query = FirebaseFirestore.getInstance()
.collection(COLLECTION_NAME)
.limit(getLimitSize())
loadDataWithQuery(query)
} else {
val docRef = FirebaseFirestore.getInstance()
.collection(COLLECTION_NAME).document(documentId)
docRef.get().addOnSuccessListener { documentSnapshot ->
val query = FirebaseFirestore.getInstance()
.collection(COLLECTION_NAME)
.startAfter(documentSnapshot)
.limit(getLimitSize())
loadDataWithQuery(query)
}.addOnFailureListener { e ->
// handle on failure
}
}
}
fun loadDataWithQuery(query: Query) {
query.get().addOnSuccessListener { queryDocumentSnapshots ->
val documents = queryDocumentSnapshots.documents
if (documents.isNotEmpty() && documents[documents.size - 1].exists()) {
//select one document from the loaded list (I selected the last document in the list)
val snapshot = documents[documents.size - 1]
var documentId = snapshot.id
//SAVE the document id in shared preferences here
//handle the random document here
} else {
//handle in case you reach to the end of the list of documents
//so we start over again as this is the first time we get a random document
//by calling getRandomDocument() with a null as a documentId
getRandomDocument(null)
}
}
}
fun getLimitSize(): Long {
val random = Random()
val listLimit = 10
return (random.nextInt(listLimit) + 1).toLong()
}

Based on #ajzbc answer I wrote this for Unity3D and its working for me.
FirebaseFirestore db;
void Start()
{
db = FirebaseFirestore.DefaultInstance;
}
public void GetRandomDocument()
{
Query query1 = db.Collection("Sports").WhereGreaterThanOrEqualTo(FieldPath.DocumentId, db.Collection("Sports").Document().Id).Limit(1);
Query query2 = db.Collection("Sports").WhereLessThan(FieldPath.DocumentId, db.Collection("Sports").Document().Id).Limit(1);
query1.GetSnapshotAsync().ContinueWithOnMainThread((querySnapshotTask1) =>
{
if(querySnapshotTask1.Result.Count > 0)
{
foreach (DocumentSnapshot documentSnapshot in querySnapshotTask1.Result.Documents)
{
Debug.Log("Random ID: "+documentSnapshot.Id);
}
} else
{
query2.GetSnapshotAsync().ContinueWithOnMainThread((querySnapshotTask2) =>
{
foreach (DocumentSnapshot documentSnapshot in querySnapshotTask2.Result.Documents)
{
Debug.Log("Random ID: " + documentSnapshot.Id);
}
});
}
});
}

A second approach as described by Alex Mamo would look similar to this:
Get the array list with the stored document ids
Get a number of strings (I stored the doc ids as string) from that list
In the code below you get 3 random and unique strings from the array and store it in a list, from where you can access the strings and make a query. I am using this code in a fragment:
#Nullable
#Override
public View onCreateView(#NonNull LayoutInflater inflater, #Nullable ViewGroup container, #Nullable Bundle savedInstanceState) {
View view = inflater.inflate(R.layout.fragment_category_selection, container, false);
btnNavFragCat1 = view.findViewById(R.id.btn_category_1);
btnNavFragCat1.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
questionKeyRef.document(tvCat1).get().addOnCompleteListener(new OnCompleteListener<DocumentSnapshot>() {
#Override
public void onComplete(#NonNull Task<DocumentSnapshot> task) {
if (task.isSuccessful()) {
DocumentSnapshot document = task.getResult();
List<String> questions = (List<String>) document.get("questions"); // This gets the array list from Firestore
List<String> randomList = getRandomElement(questions, 0);
removeDuplicates(randomList);
...
}
}
});
}
});
...
return view;
}
private List<String> getRandomElement(List<String> list, int totalItems) {
int PICK_RANDOM_STRING = 3;
Random rand = new Random();
List<String> newList = new ArrayList<>();
int count = 0;
while (count < PICK_RANDOM_STRING) {
int randomIndex = rand.nextInt(list.size());
String currentValue = list.get(randomIndex);
if (!newList.contains(currentValue)) {
newList.add(currentValue);
count++;
}
}
return newList;
}
private void removeDuplicates(List<String> list) {
try {
Log.e("One", list.get(0));
Log.e("Two", list.get(1));
Log.e("Three", list.get(2));
query1 = list.get(0); // In this vars are the strings stored with them you can then make a normal query in Firestore to get the actual document
query2 = list.get(1);
query3 = list.get(2);
} catch (Exception e) {
e.printStackTrace();
}
}
Here is the array that I get from Firestore:

firebase equalto multiple values

I'm working with firebase, I can get data from firebase with this code
String value1 = "Jack"
DatabaseReference data = FirebaseDatabase.getInstance().getReference().child("users");
Query personsQuery = data.orderByChild("desc").equalTo(value1);
one value works well, how can I get data with array value like this
String value1[] = {"Larry", "Moe", "Curly"};
Edit:
this code lists all names
mPeopleRVAdapter = new FirebaseRecyclerAdapter<Productget, NewsViewHolder>(personsOptions) {
#Override
protected void onBindViewHolder(Product.NewsViewHolder holder, final int position, final Productget model) {
holder.setTitle(model.getTitle());
}
#Override
public Product.NewsViewHolder onCreateViewHolder(ViewGroup parent, int viewType) {
View view = LayoutInflater.from(parent.getContext())
.inflate(R.layout.product_row, parent, false);
return new Product.NewsViewHolder(view);
}
};
mPeopleRV.setAdapter(mPeopleRVAdapter);
Can I filter this code for Larry, Moe and Curly?

You can try to get your data with a snapshot and store it into a map like this
Map<String, String> map = (Map<String, String>) dataSnapshot.getValue();
check this answer

Firebase Realtime Database won't be able to do that out of the box, you may need to structure your data differently in other to get desired results, see https://firebase.google.com/docs/database/android/structure-data#best_practices_for_data_structure
For instance you could do composite keys (if you are trying to filter on different childs/fields at once). Also you could try to loop though those different conditions ("Larry", "Moe", "Curly"), download the entire parent node and filter your self; depends on what you are trying to achieve.
If you are ok to Fetch all data (all users), then you could filter by name using a HashMap for example:
First Create and populate the HashMap:
HashMap<Stirng, User> hashmap = new HashMap<>();
ValueEventListener userListener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
User user = dataSnapshot.getValue(User.class);
hashmap.put(user.getName(), user);
}
};
mPostReference.addValueEventListener(userListener);
Then on your adapter you could filter on your data:
//String value1[] = {"Larry", "Moe", "Curly"};
ArrayList<User> fullNameList = new ArrayList<>(); //filtered users array list
for (int i = 0; i<value1.length; i++){
if(hashmap.containsKey(value1[i])){
fullNameList.add(hashmap.get(value1[i]));
}
}
Firestore on the other hand does support more complex queries:
https://firebase.google.com/docs/firestore/query-data/queries#compound_queries

Objectify: delete entire row by id

I need to check if a field contains specific String, and if it does - delete the entire row by id.
This code doesn't working:
Query<Bet> query = ofy().load().type(Movie.class);
for (Movie m : query) {
List<String> list = m.getActors();
String search = "Brad Pitt";
for (String str : list) {
if (str.trim().contains(search)) {
ofy().delete().type(Movie.class).id(m.getId()).now();
}
}
}

In this case (deleting all movies with Brad Pitt in it as an actor) you could delete the entity like this:
Query<Movie> query = ofy().load().type(User.class);
for (Movie m : query) {
List<String> list = m.getActors();
String search = "Brad Pitt";
for (String str : list) {
if (str.trim().contains(search)) {
ofy().delete().entity(m).now();
}
}
}
Not that i delete with delete().entity(...). Another option would be to delete by key like so delete().key(Key.create(Movie.class, idToRemove). The former does something quite similar but since you have the whole entity you don't need to complicate things. Also if you delete with entity(...) it will work when the entity has a #Parent whereas if you delete by key you'd have to additionally specify the ancestor in Key.create(ancestorKey, Movie.class, idToRemove).
I usually do multiple deletes like this:
Query<Movie> query = ofy().load().type(User.class);
List<Movie> toDelete = new ArrayList<>();
for (Movie m : query) {
List<String> list = m.getActors();
String search = "Brad Pitt";
for (String str : list) {
if (str.trim().contains(search)) {
toDelete.add(m);
}
}
}
ofy().delete().entities(toDelete).now();
Performing database operations in a loop is bad style and should be avoided if possible.
One more thing:
If you must delete an entity by id the line would look like this:
ofy().delete().type(Movie.class).id(idToDelete);
However, as i hinted at before, if your Movie class has a parent this will not work because you must always specify the whole key, thus with ancestor the line would look like this:
Key<MyParentClass> parent = Key.create(MyParentClass.class, myParentId);
ofy().delete().type(Movie.class).parent(parent).id(idToDelete);
which is equivalent to
ofy().delete().key(Key.create(Key.create(MyParentClass.class, myParentId), Movie.class, idToDelete));

Query multiple keys in couchbase lite view

Hi I am new to couchbase/couchbase-lite and i try to query a view with multiple keys without success. Her is how the map function looks:
public void map(Map<String, Object> doc, Emitter emitter) {
if (doc.get("type").equals("my_type") {
List<Object> keys = new ArrayList<Object>();
keys.add(doc.get("key_1"));
keys.add(doc.get("key_2"));
emitter.emit(keys, null);
}
}
My problem is that i need to query the view either only with key_1 or with a combination of key_1 and key_2 like so
List<Object> keys = new ArrayList<Object>();
keys.add(key_1);
if (key_2 != null) keys.add(key_2);
query.setKeys(keys);
results = query.run()
However the results are always empty. Do i overlook anything?

Two emits doesn't work. If you give a ArrayList in setKeys() method, each key in the List match the each key in emit. If you want to match two keys, add keys ArrayList into another List. Then pass it to setKeys() method. Like this,
List<Object> keys = new ArrayList<Object>();
List<Object> allKeys = new ArrayList<Object>();
keys.add(key_1);
if (key_2 != null) keys.add(key_2);
allKeys.add(keys);
query.setKeys(allKeys);
results = query.run();

This is the solution, We need to add keys into another List<Object> https://github.com/couchbase/couchbase-lite-android/issues/740

Querying CouchBaseLite Views?

I am querying CouchBaseLite view but I am getting this exception every time I query it. It is not returning the result.
detailMessage: last sequence < 0(-1)
View:
com.couchbase.lite.View viewItemsByDate = database.getView(String.format("%s/%s", designDocName, byDateViewName));
viewItemsByDate.setMap(new Mapper() {
#Override
public void map(Map<String, Object> document, Emitter emitter) {
Object createdAt = document.get("text");
if (createdAt != null) {
emitter.emit(createdAt.toString(), null);
}
}
}, "1.0");
and My query to this view is:
com.couchbase.lite.View view = database.getView(byDateViewName);
Query query = view.createQuery();//database.createAllDocumentsQuery();
List<Object> keyArray = new ArrayList<Object>();
keyArray.add("A");
keyArray.add("B");
query.setKeys(keyArray);
QueryEnumerator rowEnum = query.run();
for (Iterator<QueryRow> it = rowEnum; it.hasNext();) {
QueryRow row = it.next();
Log.d("Document ID:", row.getDocumentId());
}
And I have documents in the database with key "text": "A" and "text": "B" but still it is showing me exception. last sequence < 0(-1)
Why it could not find the rows that I queried for?

The problem is that you are querying a different view name than the view name you defined.
Change
database.getView(String.format("%s/%s", designDocName, byDateViewName));
to
database.getView(byDateViewName);
and it fixes the problem.
See Couchbase Lite Android issue #66