firestore getDocuments().size() != getDocumentChanges().size() - android

I query my firestore db with a complex query
.whereIn(gender_node, getInterestedIn())
.whereGreaterThanOrEqualTo(birthdate_year_node, getShowAgeMax() + 1)
.whereLessThanOrEqualTo(birthdate_year_node, getShowAgeMin())
but I run into an IndexOutOfBoundException, because getDocuments().size() is not equal to getDocumentChanges().size().
I can't find anything in the documentation, does anybody know when or why getDocuments().size() != getDocumentChanges().size() ?
I assumed there would be a DocumentChange for every Document.
It doesn't happen every time, which makes it hard to debug.
There are 7 test documents and getDocuments().size() is always 7, but getDocumentChanges().size() is sometimes 5. Always seems to be the number 5, as far as I have seen.

getDocument().size() and getDocumentChanges().size() are two different methods.
The getDocumentChange().size() returns the list of documents that
changed since the last snapshot.
The getDocument().size() returns
the list of documents.
The details about these methods can be found out in the documentation section.
Suppose, if you have 7 documents and there are changes on 2 documents since last snapshot, then getDocument().size() would be 7( including the newly added or modified documents) here and getDocumentChanges().size() would always refer to the number of changed documents, which in this case is 2.
And hence is the condition,
getDocument().size() ! = getDocumentChanges().size()
true.

Is it possible that you have an extra paranthese closed in this line? (just after birthdate_year_node)
.whereLessThanOrEqualTo(birthdate_year_node),getShowAgeMin())
Otherwise, can you run your code with each one of the 3 "where" clauses seperately so we know which one of the three is creating the issue.

Related

Multiple and() conditions in where clause for couchbase lite Android client

In my Android app we use couchbase lite database version 2.8.6
I run three database queries.
One item in where clause.
QueryBuilder.select(
SelectResult.property("id"),
SelectResult.property("timestamp"),
SelectResult.property("rating"))
.from(DataSource.database(database))
.where(Expression.property("type").equalTo(Expression.string(DOC_TYPE)))
In the result I see three items from database printed to a console. As expected. Format here ans below is id|timestamp|rating
4e39f79c-9e11-4aba-9fb6-95d910f46cd9|0|-2147483648
e95646ee-ba3a-4978-b2a8-5383f31be2f1|0|-2147483648
e02d0eb3-6c9b-4942-b43c-a752eefc77a8|1630525956184|2147483647
I add and() condition to where() to get all items where type = 'type' AND rating < 1
QueryBuilder.select(
SelectResult.property("id"),
SelectResult.property("timestamp"),
SelectResult.property("rating"))
.from(DataSource.database(database))
.where(Expression.property("type").equalTo(Expression.string(DOC_TYPE))
.and(Expression.property("rating").lessThan(Expression.intValue(1))
)
Result is as expected as we search everything with rating < 1, third item is filtered out.
4e39f79c-9e11-4aba-9fb6-95d910f46cd9|0|-2147483648
e95646ee-ba3a-4978-b2a8-5383f31be2f1|0|-2147483648
Finally, I want to see records where type = 'type' AND rating < 1 AND timestamp <= 1
QueryBuilder.select(
SelectResult.property("id"),
SelectResult.property("timestamp"),
SelectResult.property("rating"))
.from(DataSource.database(database))
.where(Expression.property("type").equalTo(Expression.string(DOC_TYPE))
.and(Expression.property("rating").lessThan(Expression.intValue(1))
.and(Expression.property("timestamp")).lessThanOrEqualTo (Expression.longValue(1))
)
)
And now the result is really strange as I receive three items form the database. And the third one has timestamp much greater than 1 put into the query.
4e39f79c-9e11-4aba-9fb6-95d910f46cd9|0|-2147483648
e95646ee-ba3a-4978-b2a8-5383f31be2f1|0|-2147483648
e02d0eb3-6c9b-4942-b43c-a752eefc77a8|1630525956184|2147483647
Any ideas how to make it work with multiple and()? If I try to remove the second and() and keep the third one everything works as expected.
After deep investigations I found several problems in my code:
.and() should be called at the end of the "child" condition and not at the "parent" level.
For the conditions like this
val condition1 = Expression.property("type").equalTo(Expression.string(DOC_TYPE))
val condition2 = Expression.property("rating").lessThan(Expression.intValue(1))
val condition3 = Expression.property("timestamp")).lessThanOrEqualTo (Expression.longValue(1))
The correct way is
.where(condition1.and(
condition2.and(
condition3.and(
...
)
)
)
but NOT like I've tried
.where(condition1.and(condition2)
.and(condition3)
.and(...)
)
For the creation of the object I used a code that converts an object to Map<String, Any> before saving to couchbase. Everything seemed to be OK till I realized that Long was converted to Double.
The last point, I have much complicated queries and inside one of them I accidentally used .add() instead of .and(). No comments.
Hope this will help to somebody to save some time.
I would expect that the query would be as you did originally:
.where(condition1.and(condition2)
.and(condition3)
.and(...))
I think there is an additional parenthesis here that seems to be wrong:
.and(Expression.property("timestamp"))

How to query "nearby" & "latest" in firestore with limit in Android?

I have one Android project where I need to query nearby items & these items should be sorted by time.
Basically, I need docs that are in 100KM. Sorted by time (Field).
So I have checked Firestore docs for this & I got solution (Add geoHash to docs & then query them by geoHasBounds) But there is an issue what if there are 1k docs in 100km then it will load all which is not good, so how can I limit those different queries & gets only 25-30 docs then next 25-30 docs ??
In short, this is what I need-
How can I query the latest 25 docs in 100KM radius & when the user scroll down the next 25 docs?
this is my code of query-
List<GeoQueryBounds> bounds = GeoFireUtils.getGeoHashQueryBounds(center, radiusInM);
final List<Task<QuerySnapshot>> tasks = new ArrayList<>();
for (GeoQueryBounds b : bounds) {
Query newQuery = itemQuery.orderBy("geoHash").startAt(b.startHash).endAt(b.endHash);
tasks.add(newQuery.get());
}
// Collect all the query results together into a single list
Tasks.whenAllComplete(tasks).........
What you are looking for is called pagination. I have answered a question here on Stackoverflow, where I have explained a recommended way in which you can paginate queries by combining query cursors with the "limit() method". I also recommend you take a look at this video for a better understanding.
If you are willing to try Paging 3 library, please see below an article that will help you achieve that.
How to paginate Firestore using Paging 3 on Android?
Edit:
The Tasks.whenAllComplete() method:
Returns a Task with a list of Tasks that completes successfully when all of the specified Tasks complete.
So you can then simply convert each object to a type of object that you need and paginate that list accordingly. Unfortunately, this implies getting all the objects in the first place. Otherwise, you can divide those queries into separate queries and treat them accordingly, by using separate paginantion.

How to do a simple search in string in Firebase database?

I want to create a simple search in my app, but cannot find anything on interwebs about it, that's more recent than 2014. There must be a better way. There are startAt and endAt functions but they don't work as expected and are case sensitive. How do you guys solve this problem? How can this functionality still not exist in 2016?
In my case I was able to partly achieve a SQL LIKE in the following way:
databaseReference.orderByChild('_searchLastName')
.startAt(queryText)
.endAt(queryText+"\uf8ff")
The character \uf8ff used in the query is a very high code point in the Unicode range (it is a Private Usage Area [PUA] code). Because it is after most regular characters in Unicode, the query matches all values that start with queryText.
In this way, searching by "Fre" I could get the records having "Fred, Freddy, Frey" as value in _searchLastName property from the database.
Create two String variables
searchInputToLower = inputText.getText().toString().toLowerCase();
searchInputTOUpper = inputText.getText().toString().toUpperCase();
Then in the Query set it to:
DatabaseReference reference = FirebaseDatabase.getInstance().getReference().child("Products");//Your firebase node you want to search inside..
FirebaseRecyclerOptions<Products> options =
new FirebaseRecyclerOptions.Builder<Products>()//the Products is a class that get and set Strings from Firebase Database.
.setQuery(reference.orderByChild("name").startAt(searchInputUpper).endAt(searchInputLower + "\uf8ff"),Products.class)
.build();
the "name" it's the node inside the Products Main Node.
the .startAt(searchInputUpper) & .endAt(searchInputLower + "\uf8ff") to make the search as contains all characters that typed in the inputText.getText() that you get.
finally I got it you can use where clause to get you result like SQL
LIKE keyword like% or %like
syntax :
Firestore.collection(collectionName).orderBy(field).where(field, ">=", keyword.toUpperCase()).where(field, "<=", keyword.toUpperCase() + "\uf8ff").get()
I my case used:
var query = 'text'
databaseReference.orderByChild('search_name')
.startAt(`%${query}%`)
.endAt(query+"\uf8ff")
.once("value")
In this way, searching by "test" I could get the records having "Test 1, Contest, One test" as value in 'search' property from the database.
Firebase is noSQL therefore it does not have searches built in like you'll find in SQL. You can either sort by values/key or you can equalto
https://firebase.google.com/docs/database/android/retrieve-data
You can find examples at the link above. That is the latest documentation for firebase.
If you are looking for SQL like searches. Then take a look at elastic search. But that will increase the complexity since you need a platform to put it on. For that i could recommend Heroku or maybe GoogleCloudServers
Here is a blog post about advanced searches with elastic search
https://firebase.googleblog.com/2014/01/queries-part-2-advanced-searches-with.html
This question might be old but there is a documented way of how to achieve this way, It is simple to implement. Quoted:
To enable full text search of your Cloud Firestore data, use a third-party search service like Algolia. Consider a note-taking app where each note is a document:
Algolia will be part of your firebase functions and will do all the searches you want.
// Update the search index every time a blog post is written.
exports.onNoteCreated = functions.firestore.document('notes/{noteId}').onCreate(event => {
// Get the note document
const note = event.data.data();
// Add an 'objectID' field which Algolia requires
note.objectID = event.params.noteId;
// Write to the algolia index
const index = client.initIndex(ALGOLIA_INDEX_NAME);
return index.saveObject(note);
});
To implement the search, the best way is to use instant search - android
Sample Search Image: GIF
The feature you're looking for is called full-text search and this is something most databases (including Firebase) don't provide out-of-the-box because it requires storing the data in a format that's optimized for text search (vs optimized for filtering) - these are two different problem sets with a different set of trade-offs.
So you would have to use a separate full-text search engine in conjunction with Firebase to be able to do this, especially if you need features like faceting, typo tolerance, merchandizing, etc.
You have a few options for a full-text search engine:
There's Algolia which is easy to get up and running but can get expensive quickly
There's ElasticSearch which has a steep learning curve but uber flexible
There's Typesense which aims to be an open source alternative to Algolia.
I don't know about the certainty of this approach but using the firebase version 10.2.6 on android, i get to do something like this:
firebaseDatabase.getReference("parent")
.orderByChild("childNode")
.startAt("[a-zA-Z0-9]*")
.endAt(searchString)
It seems to work well sometimes
Finally joined SO just to answer this.
For anyone coming here from/for the python firestore.client here's a solution that seems to work for me.
It's based on the accepted answer's concept but via the client rather than db.reference() and mixed with the answer from user12750908.
from firebase_admin import firestore
users = db.collection("users")\
.order_by("last_name")\
.where("last_name", ">=", last_name.upper())\
.where("last_name", "<=", last_name.lower() + "\uf8ff")\
.stream()
It works for the simple test I did, but I'll update my answer if I have issues with it later. And just a reminder, this is similar to
LIKE search%
and not
LIKE %search%.
Edit 1
I didn't see any tags for the question, but the title attribute mentions Android so this may not necessarily answer the question directly, but if you have a python API, this should work. I'm unfortunately not sure if there's an equivalent client/db separation in the Android version like there is in the Firebase Admin for Python. I didn't want to delete the answer since I hadn't seen any answers for firestore client during my search for a similar answer and hope it helps anyone else stumbling around.
Edit 09-03-2020 This works a portion of the time it seems. Most of the time I didn't seem to have an issue, but when I applied it to another project I was getting unexpected results. Long story short you may need to replicate how you save the data you're comparing against. For example, you may need to have a field to save the last_name in all caps and another field to save it in all lowercase, then you change the first where clause to compare last_name_upper and the second to compare last_name_lowercase. In my second project so far this seems to yield more accurate results so you may want to give that a try if the previous answer doesn't work well
EDIT 09-07-2020 Previous edit from 09-03-2020 is partially accurate. During my haste of thinking I had it fully resolved I completely forgot firebase doesn't let you use <, >, <=, >= across different fields. You may need to do two queries and merge them, but you'd probably still be reading more docs than you really intend. Doing the comparison against either the upper or lower version with the appropriate search term seems to give the original results expected from the original answer. For example:
.orderBy("last_name_upper")
.where("last_name_upper", ">=", this.searchForm.text.toUpperCase())
.where("last_name_upper", "<=", this.searchForm.text.toUpperCase() + "\uf8ff")
As firebase documentation, firebase doesn't support full text search.
But to do that you can use third-party tools.
Check this link to learn more https://firebase.google.com/docs/firestore/solutions/search

Android - Do I need to sort a collection for min and max?

I came here (SO) a few days ago to research how to get the min and max from a collection in Android and found a solution to the effect of the following (sorry haven't got a link to the actual answer I used):
Max = (TextView)findViewById(R.id.Max);
Collections.sort(list);
Max.setText(String.format("%.2f", Collections.max(list)));
My question is do I actually need to sort the list before pulling the min/max value? I have tried running the code without sorting the list and it seems to work OK. I am just worried because the answer I used definitely sorted the list first so I assume there must be a reason, I just don't know what it is!
In addition #BobbyDigital's answer who corectly points out the th method iterates over the complete list, I would just like to mention that the result of using the max function might depend on the type of the list elements. If you see the doc , it says that
Returns the maximum element of the given collection, according to the natural ordering of its elements.
If you see Why does Collections.max() not return actual max value for a Collection of String? question, the person used a list of Strings. On extracting max using the abve number he did not get the max number as it was returning the value that's the largest lexicographically. So, just to mention his code:
ArrayList<String> dirNo = new ArrayList<String>();
dirNo.add("1");
dirNo.add("2");
dirNo.add("3");
dirNo.add("4");
dirNo.add("5");
dirNo.add("6");
dirNo.add("7");
dirNo.add("8");
dirNo.add("9");
dirNo.add("10");
dirNo.add("11");
System.out.println("max : " + Integer.parseInt(Collections.max(dirNo))
+ "");
The above code gave 9 as the answer. So be careful while using it. You mgiht want to convert everything to Integer etc based on your needs.
P.S: The example is from the question mentioned and the answer is inspired from this answer by NPE on same question.
No it doesn't have to be sorted. The method iterates over the entire collection.
See the Java docs for the method!

Samsung device returns text messages when querying for call-log

Some of my users report that on their Samsung devices (GT-N7000 & SGH-I777) a query I make in my app for the CallLog.Calls displays also text messages.
I've created a dump of their CallLog ContentProvider, and it seems to have extra fields not mentioned in the Android API, and not returned on any of our test devices.
Specifically, looking through the dump, there's a field called logtype, which seems to equal 100 for calls, and 300 for text messages.
Having searching online for this field, I didn't find any official documentation for this field's values, but I came across lots of other possible values for this field mainly via crash stack traces, which reveal underlining queries by the ContentProvider:
logtype=300 OR logtype=200
logtype=100 OR logtype=500 OR logtype=800 OR logtype=900 OR
logtype=1000
So I assume that 300/200 are used for text messages, and 100/500/800/900/1000 are used for calls, but I'm not sure, since I haven't seen 500/800/900/1000 being used on the reporting users' devices.
Can someone shed some light for the possible values of logtype, and their meaning?
Hello If you will check callLog.Calls columns you will find messageid field which says that this is message for samsung phones.
So if you just want to get the list of calls without messages simply do :
int messageIdIndex=cursor.getColumnIndex("messageid");
while (cursor.moveToNext())
{
if(messageIdIndex>=0)
messageID=cursor.getLong(messageIdIndex);
if(messageID<=0)
{
//do whatever you need with calls log data
}
}
cursor.close();
I've managed to tentatively solve it by querying CallLog.Calls for the column logtype, if an exception is thrown, I query normally, otherwise, I query with selection of (logtype=100 OR logtype=500)
This seems to be working for my reporting users, but I'm still not sure if it covers all bases, since there are many possible values for logtype for which I don't know the meaning.
If anyone has a better answer, please add it.
When i was debugging a sgs2 device i have found this. may be useful for someone.
SELECT number, name, type, date, duration FROM logs WHERE (
logs.logtype=100 OR
logs.logtype=110 OR
logs.logtype=900 OR
logs.logtype=500 OR
logs.logtype=800 OR
logs.logtype=120 OR
logs.logtype=510 OR
logs.logtype=1000 OR
(logs.logtype=200 AND number NOT IN (SELECT number FROM logs WHERE number LIKE '%#%')) OR logs.logtype=300)
AND ((type != 4)) AND (logtype=100 OR logtype=500)))
ORDER BY date DESC
So far, we have found Samsung devices using the value 1150, 100 or 1000 for the 'logtype' field in the Calls content provider.
I cannot confirm if any of the other values mentioned here are used.

Categories

Resources