I am new to couchbase and I am trying to implement couchbase lite in one of my Android applications. What i am struggling with in particular is the concept of views and the rules for the map function as stated in the docs.
In the database the app stores documents with various doc types. In one query i need to get the entire document by document type ("payments")
and by value of an attribute of the document (doc["approved"] = true)
Hence I would create a view like so:
com.couchbase.lite.View view = database.getView("payments");
if (view.getMap() == null) {
Mapper map = new Mapper() {
#Override
public void map(Map<String, Object> doc, Emitter emitter) {
if (doc.get("type").equals("payments") && doc.get("approved") == true) {
emitter.emit(doc.get("name"), doc);
}
}
};
view.setMap(map, "1");
}
Note that the doc["approved"] value can be updated over time. In one of the rules about map functions in the docs it says:
It must be a "pure" function: ... That means any time it's called with the
same input, it must produce exactly the same output.
Would the implementation of the map function as show above violate that rule?
In the docs it further says :
In particular, avoid these common mistakes: ... Don't make any assumptions
about when the map function is called. That's an implementation detail
of the indexer. (For example, it's not called every time a document
changes.).
Does that mean when the approved status of one of the documents is updated from false to true that the following query not nessesarily contains the updated document? If so what would I need to do to achieve this? I am quite uncertain about what rule that exacly means? Could anyone try to open my eyes please?
What "pure" means is that you cannot use outside state in your map function. All your determinations must be based solely on the parameters that were passed into it. Your map function does not violate this.
I think the missing piece in your understanding is the difference between storage and indexing. You can store revisions of a document to the database, right? That in and of itself will not cause the view's index to be updated. That's what the documentation means by "not called every time a document changes." The index will be updated by default when the next query is run, so the newest state of the document will be output. It could realistically have been changed many times since the last query was run.
Related
Scenario
I am using onSnapshot() to listen to document changes in Firestore. My document contains an array field. I would like to see which items were added/deleted from the nested array in the document. Is there any way to achieve this?
Possible Solution
I see there is an oldDocuments() field in the QuerySnapshot.snapshot object which could be used to achieve my goal. I could just compare the oldDocument with the returned snapshot to see what changed in the nested array. The problem is oldDocument is a private field that I can't access through the API.
I am using Android Kotlin as my client.
There is nothing built in that can help you see what exactly in a document has changed. However, there are two workarounds.
The first one would be to attach a persistent listener and view changes between snapshots. That means that when you attach the listener you get a list of all documents your query returns. As soon as a document is changed, onEvent will fire. This means that you'll be able to know the document that has changed. Knowing that you already have a list with the initial documents, you can find the old document based on the ID and compare it against the new one.
Another solution would be to save a new document each time something changes. This is some kind of document versioning. In this way, you'll always be able to check the new document against the old one using only two document reads.
I have a small problem with the initial setup of my database.
My structure is this:
I have an array of codes and a map that maps each code to an integer which will be updated.
Everything works as expected. I use the following code to achieve this goal. But there is exactly one problem: The very first writing to this document when it does not exist will not create a map of code: int but will create a field that has the name "comparisons.12345". So instead of a map, I end up with loads of individual fields (One for each code)
My workaround is to create the document manually with the empty map. After that everything works as expected
Of course, I could check on the frontend site if the document exists and add it if not. But this is a ONE-TIME operation. I do not want to pay for the overhead of checking the document EVERY TIME. So is there a way of creating it the correct way or do I have to stick to manual creating (Which is not a problem in production, but very annoying during testing)
val data = mutableMapOf(
"codes" to FieldValue.arrayUnion(comp.getCode()),
"comparisons." + comp.getCode() to FieldValue.increment(1)
)
db.collection("codes").document("info").update(data as Map<String, Any>)
Update
When using set with merge set to true, the behavior is exactly the same when the document does not yet exist. Instead of the map I need, I get loads of documents with a "dot" in their name like this:
There are two ways in which you can update a document. The first one is when using DocumentReference#update(Map<String, Object> data) which:
Updates fields in the document referred to by this DocumentReference. If no document exists yet, the update will fail.
And the second one is when using DocumentReference#set(Object data, SetOptions options) which:
Writes to the document referred to by this DocumentReference. If the document does not yet exist, it will be created. If you pass SetOptions, the provided data can be merged into an existing document.
So there is no difference between using the update() and the set() method with the merge true option for documents that already exist. The difference between these two operations comes only for documents that don't exist. For instance, set() with the merge true option will create the document if the document does not exist, while when using the update(), the operation will fail.
So you should use the second option. In this way, you'll don't need to check the documents for existence.
Edit:
You're getting:
comparisons.12345
Because of the way you're trying to perform the update. To be able to perform an update to an element that exists within an object (map), you have to use another map:
val increment = mutableMapOf(
comp.getCode() to FieldValue.increment(1)
)
val data = mutableMapOf(
"codes" to FieldValue.arrayUnion(comp.getCode()),
"comparisons" to increment
)
db.collection("codes").document("info").set(data, SetOptions.merge())
The above code will work only if comp.getCode() returns 12345.
So i'm a bit new to firebase and i'm developing an android app. What I would like to do is this: I want to create a service that runs in the background that can listen to a whole collection in firestore (For example "messages"), and once a document is added to the collection I would like to check if the document contains a certain value, and if it does push a notification. An outline of the pseudocode:
messages.attachlistener(onNewDocument(doc) -> {
if(doc.getString(username) == my_username)
{
send_notification(doc.getString(message));
cache_message_for_later();
delete_doc_from_table();
}
};
My question is this:
Is attaching a listener to a collection possible? I only found document listeners in the docs. If it is possible, would such a listener even be viable performance wise? (My thinking: this service would be running in the background at all times, but documents would get deleted after being cached so the table size wouldn't be too large and it might balance out?)
You can attach listeners to both a DocumentReference and a Query. As you can see from the API documentation, it turns out that CollectionReference is a subclass of Query, which means you can add a listener to a CollectionReference just fine. A CollectionReference taken as a Query this way will give you all of the documents in that collection, then all of the changes to any of the documents in that collection, just as you see in the documentation (except you don't use any where clauses).
I did a quick check on my code to confirm.
You can do a code like this, without a problem:
val db = FirebaseFirestore.getInstance()
val query = db.collection("Quotes_historyfavorites").document(userId)
.collection(FireBaseConstants.QuoteCollection.FAVORITE)
.orderBy("Added", Query.Direction.DESCENDING)
I am not sure if a top level collection would be possible, but I believe that you would face a moneyissue before having a perfomance issue.
I need to measure the size/length of an element or list in Firebase but I don't need the real content.
Something like firebase.database().ref("users/").once("length" || "size").then(callback)
ps Javascript SDK
Firebase doesn't have a count operator, so the only way is to download all children or keep a separate <children>_count property in sync. The latter is not a trivial task (see my answer here for one approach and this example Cloud Function), so most often developers likely end up going with the downloads-too-much-data-but-is-trivial approach:
ref.child("messages").on("value", function(snapshot) {
console.log("There are "+snapshot.numChildren()+" messages");
})
A more efficient way to count the children would be to fire a REST call with shallow=true parameter, which will give you just the keys. See In Firebase, is there a way to get the number of children of a node without loading all the node data?
Also found a post which does it a different way...
Object.keys(users).length;
Post: Length of a JavaScript object
I'm trying the new Android Mobile Backend. I did all the basic samples and now I want to set one or multiple filters. I tried only with one but the getCloudBackend().list returns a null list.
I have in the entity that I filter the parameter with the value. And if I remove the line where I filter, the service returns a list with data.
What I have to do if I want to filter by one parameter? and if I want to filter by the owner? and if I want to put more than one filter?
This is my code:
getCloudBackend().clearAllSubscription();
CloudQuery cq = new CloudQuery("MyEntity");
cq.setSort(CloudEntity.PROP_UPDATED_AT, Order.DESC);
cq.setLimit(100);
q.setFilter(F.eq("myparameter", "myvalue"));
cq.setScope(Scope.FUTURE_AND_PAST);
getCloudBackend().list(cq, handler);
You should post the error (or response) you get in the LogCat and also in the backend log. Filters are a bit tricky. I suggest you to check CloudQuery documentation to see the filter limitations, and CloudEntity to check the data you can send, retrieve and how.
About queries the documentation says:
Queries in the Cloud Backend API have some restrictions that originate
from the App Engine Datastore. By default, App Engine automatically
creates an index for each user defined property (with the exception of
List/Map properties). This index supports the following forms of
queries: - Kind name specified, no filters, no sorting - Queries with
only equality filters (F.eq) on indexed properties with no sorting -
Queries with only inequality filters, such as F.lt, F.ge, on a single
indexed property - Queries with one sort order on one property with no
filters
For example, for String properties you can only check for equality (eq/ne). You can't mix equaltiy filters with inequality ones. i.e. :
F.and(F.eq(PROPERTY_NAME, "Manuel"), F.gt(PROPERTY_AGE, Integer.valueOf(18)));
Will give you an error DatastoreNeedIndexException.
The documentation also says:
All other query forms will raise a DatastoreNeedIndexException on the
backend. To use a more complex form of queries, you need to add an
Index Configuration on the backend.
To do such you need edit datastore-indexes.xml in your backend. In order to do that you can use the command ./appengine-java-sdk/bin/appcfg.sh update_indexes myapp/war.
Looks promising though I haven't tested it yet...
Filters are case-sensitive, in the Guestbook example:
cq.setFilter(F.eq("MESSAGE","fabulous")); -> This won't work... don't let Google Cloud Console fool you.
cq.setFilter(F.eq("message","fabulous")); -> This will work... don't know why is case sensitive.
Let me know if this was the problem.
What I have to do if I want to filter by one parameter?
I think it is better to use getCloudBackend().listByProperty() method instead of getCloudBackend().list()
Example of this method usage:
getCloudBackend().listByProperty("yourKindName", "yourPropertyName", Op.EQ,
yourPropertyValueObject, null, 1, Scope.PAST, yourHandler);
and if I want to filter by the owner?
You can use the listed above method. Just put your owner property name instead of yourPropertyName
and if I want to put more than one filter?
To use multiple filter for one property I created listByPropertyAnd() method in CloudBackendAsync class:
public void listByPropertyAnd(String kindName, String propertyName,
CloudQuery.Order order, int limit, Scope scope,
CloudCallbackHandler<List<CloudEntity>> handler, F... filters) {
CloudQuery cq = new CloudQuery(kindName);
cq.setFilter(F.and(filters));
cq.setSort(propertyName, order);
cq.setLimit(limit);
cq.setScope(scope);
this.list(cq, handler);
}
Example of this method usage:
getCloudBackend().listByPropertyAnd("yourKindName", "yourKindName", Order.DESC,
1000, Scope.FUTURE_AND_PAST, yourHandler,
F.gt("yourKindName", yourFirstValue),
F.lt("yourKindName", yourSecondValue), F.eq("date", mToday));
In this case you can use any number of filters for one property but it should be compliant with datastore indexing rules.
Also it might be useful to read this Java Datastore Filters and this one Mobile Backend Starter - API Guide