We are using Firebase firestore with our simple grocery delivery app (android) for more than a year. Its super fast and great but it keeps missing data while writing.
Before, without using transactions and batched writes, the problem was much more persistent(about 10/1000 entries). After using the issue has been reduced but not completely gone(about 2/1000 entries).
Scenario:
We have few collections which are set/updated when the product is added namely products,stockand daylogs. When we add/update the product, the transaction writes different data to all these collections. But sometimes(2/1000), the transaction is successful and writes to products and stock but not daylogs.
Here is a simpler version of the code. Although, I don't think it's a code issue because it works fine 99% of the time.
firebasefirestore.runTransaction(new Transaction.Function<String>() {
#Override
public String apply(#NonNull Transaction transaction) throws FirebaseFirestoreException {
DocumentReference thisProduct = rootRef.collection("products").document(docId);
DocumentReference thisStock = rootRef.collection("stock").document(docId);
DocumentReference logs = rootRef.collection("daylogs").document(LogDocumentId);
transaction.update(thisProduct, data);
transaction.update(thisStock, data);
transaction.update(logs, data);
String complete = "1";
return complete;
}
}).addOnSuccessListener(new OnSuccessListener<String>() {
#Override
public void onSuccess(String result) {
if (result.equals("1")) {
Toast.makeText(AddProduct.this, "Data Added", Toast.LENGTH_SHORT).show();
}
}
});
Why is this happening? And How can this be solved?
Sorry for my english.
Please help.
Related
When a new user registers in my app using firebase-authentication custom sign in using email and password, I need to update that data into my firestore.
But Firebase only has FirebaseAuth.createUserWithEmailAndPassword(email, password) to create a new account and hence I cannot update my user's username at the same time.
To update the E-Mail in Firestore, I use Firebase cloud functions. Here's the code:
export const onNewUserJoined = functions.auth.user().onCreate((user) => {
//const newUserDisplayName = user.displayName //CAN'T USE THIS. REASON is BELOW
const newUserUID = user.uid
const newUserEmail = user.email
const timeCreated = Date.now()
console.log(`${newUserUID} has joined.`)
return admin.firestore().collection('Agent').doc(`${newUserUID}`).set({"E-Mail": newUserEmail, "Time": timeCreated})
})
OK, great now I have updated the E-Mail and time created in Firestore successfully.
But next challenge is I need to update the user's username in the same Firestore document. I do it instantly after the createUserWithEmailAndPassword() like this:
DocumentReference dDocRef = FirebaseFirestore.getInstance().document(documentPath);
Map<String, Object> updateUsernameAndPhone = new HashMap<>();
updateUsernameAndPhone.put("username", username);
updateUsernameAndPhone.put("phoneData", phoneModel);
dDocRef.update(updateUsernameAndPhone).addOnSuccessListener(new OnSuccessListener<Void>() {
#Override
public void onSuccess(Void aVoid) {
Toast.makeText(getApplicationContext(), "Data successfully stored in Firestore", Toast.LENGTH_SHORT).show();
}
}).addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
Toast.makeText(getApplicationContext(), e.toString(), Toast.LENGTH_SHORT).show();
}
});
Now, it depends on who acts first, the cloud function or the user's device.
If the cloud functions act first, then there is no issue. The username and phoneModel both get updated into the document successfully. No issues.
But incase, the phone acts first then I get the following error:
As this error has occurred, username isn't in the document and only email and timeCreated are in the document updated by the cloud function which got late to create document so that user's device can update the username with ease.
I CAN'T use .SET instead of .update() in my app because if I use .set() and the cloud functions create the email and timeCreated fields first. Then the device will DELETE them and put username and phoneModel.
So how can I do this?
I can forcefully delay updating the username by putting it in the next activity so that cloud functions get enough time to do their job, but my signUpActivity asks for username along with email and password edit texts. I don't want to create a separate activity for that.
I used to use .update() when my data was stored in realtime database and it used to create the child even if the path didn't exist. But it looks firestore won't update if the field doesn't exist.
Any solution for this?
I tried as per #DougStevenson said and here's my code:
final String newUserUID = Objects.requireNonNull(signUpAuth.getCurrentUser()).getUid();
final String documentPath = "Agent/" + newUserUID;
FirebaseFirestore fFirestoreRef = FirebaseFirestore.getInstance();
final DocumentReference dDocRef = fFirestoreRef.document(documentPath);
fFirestoreRef.runTransaction(new Transaction.Function<Void>() {
#Nullable
#Override
public Void apply(#NonNull Transaction transaction) throws
FirebaseFirestoreException {
DocumentSnapshot documentSnapshot = transaction.get(dDocRef);
transaction.update(dDocRef, "username", username);
transaction.update(dDocRef, "phoneData", phoneModel);
return null;
}
}).addOnSuccessListener(new OnSuccessListener<Void>() {
#Override
public void onSuccess(Void aVoid) {
Toast.makeText(getApplicationContext(), "Data updated in Firestore . . .", Toast.LENGTH_SHORT).show();
}
}).addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
Toast.makeText(getApplicationContext(), e.toString(), Toast.LENGTH_SHORT).show();
}
});
But no luck. It still gives me error: Cannot update a document which does not exist.
Use a transaction to resolve conflicts from multiple clients that are all trying to modify the same document. The transaction handler will be retried on the client if it detects the document was modified before the change could take place.
I have the following transaction using Firestore:
mDb.runTransaction(new Transaction.Function<Void>() {
#Override
public Void apply(final Transaction transaction) throws FirebaseFirestoreException {
DocumentReference documentReference = mDb.collection("collectionOne").document("documentOne");
/*
some code
*/
transaction.update(documentReference, App.getResourses().getString(R.string.field_one), FieldValue.increment(1));
transaction.update(documentReference, App.getResourses().getString(R.string.field_two), FieldValue.increment(1));
return null;
}
}).addOnSuccessListener(new OnSuccessListener<Void>() {
#Override
public void onSuccess(Void aVoid) {
Log.d("Debugging", "Transaction correctly executed.");
}
}).addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
Log.w("Debugging", "Transaction failure.", e);
}
});
My question is: when updating, for example, two fields of the same document within the same transaction, will such a transaction yield to one or two documents reads?
when updating, for example, two fields of the same document within the same transaction, will such a transaction yield to one or two documents reads?
Doesn't matter how many fields you change in a document in one operation, you'll always be charged with one write operation. If you make the writes, one after the other, you'll be charged with two write operations.
I'm creating an Android app that uses Firebase Firestore to store some data. Since collection group queries are not yet supported in Firestore, in order to build some of the result sets that I need, I have to do a Query followed by multiple DocumentReference.get() calls to achieve the desired end result.
My implementation is as follows:
PlaylistFacade.java
public static Task<QuerySnapshot> GetPlaylistSubscriptionsByOwner(FirebaseFirestore db, #NonNull String ownerUserId)
{
Query playlistRef = db.collection("playlistSubscriptions")
.whereEqualTo("ownerId", ownerUserId);
return playlistRef.get();
}
public static Task<DocumentSnapshot> GetPlaylist(FirebaseFirestore db, #NonNull String playlistId)
{
return db.collection("playlists").document(playlistId).get();
}
PlaylistActivity.java
PlaylistFacade.GetPlaylistSubscriptionsByOwner(mFirestore, this.GetCurrentUser().getUid())
.continueWithTask(new Continuation<QuerySnapshot, Task<List<Task<?>>>>() {
#Override
public Task<List<Task<?>>> then(#NonNull Task<QuerySnapshot> task) throws Exception {
List<Task<DocumentSnapshot>> tasks = new ArrayList<>();
for(DocumentSnapshot doc:task.getResult().getDocuments())
{
PlaylistSubscription ps = doc.toObject(PlaylistSubscription.class);
ps.setId(doc.getId());
tasks.add(PlaylistFacade.GetPlaylist(mFirestore, ps.getPlaylistId()));
}
return Tasks.whenAllComplete(tasks);
}
})
.addOnCompleteListener(this, new OnCompleteListener<List<Task<?>>>() {
#Override
public void onComplete(#NonNull Task<List<Task<?>>> task) {
if(task.isSuccessful()){
List<Task<?>> tasks = task.getResult();
List<Playlist> playLists = new ArrayList<>();
int errorCount = 0;
for(Task<?> docTask : tasks)
{
if(docTask.isSuccessful())
{
DocumentSnapshot ds = (DocumentSnapshot)docTask.getResult();
Playlist pl = ds.toObject(Playlist.class);
pl.setId(ds.getId());
playLists.add(pl);
}
else
{
Crashlytics.logException(docTask.getException());
errorCount++;
}
}
if(errorCount > 0)
Toast.makeText(PlaylistActivity.this, "Encountered " + errorCount + " errors.", Toast.LENGTH_LONG).show();
else
Toast.makeText(PlaylistActivity.this, "Success", Toast.LENGTH_LONG).show();
}
else
{
Crashlytics.logException(task.getException());
Toast.makeText(PlaylistActivity.this, task.getException().getMessage(), Toast.LENGTH_LONG).show();
}
}
});
The above works just fine, but I'm curious if there may be a better way to do it.
My questions are as follows:
Is chaining the Task objects using List<> the ideal approach? Am I giving up any potential performance by using this approach?
Will using the Task chain still allow me to take advantage of Firebase's ability to pipeline requests?
Will my use of Tasks.whenAllComplete() allow me to conditionally accept failure of some or all results, or does Firebase's pipelining cause errors to propagate across requests such that I should really just use Tasks.whenAllSuccess() and avoid the need check success of each individual request?
Response times of my implementation seem fine on small result sets. Am I likely to get better performance as my result set grows if I build my result set in a Cloud Function before returning it to my app instead?
At what complexity of Firestore actions should I really be using an Executor as demonstrated in the DocSnippets.java sample on the deleteCollection(...) function?
Would using a Transaction to bundle requests ever net me any performance gains? Performance implications of doing reads inside a transaction aren't discussed in the documentation.
Any news of when collection group queries will be available in Firestore? In time for Google IO, perhaps?
Thank you for your time.
Other helpful resources for those who end up here with similar questions:
Doug Stevenson's Firebase Blog Post on Task Wiring
There's an example provided in NetworkBoundResource, but when I tried it out, if database returns a result, it does not fetch from network anymore. What I want is display data from database and trigger network at the same time and eventually replace data in database after network completes. Example codes will be much appreciated.
I would use room database to save your items in a table. Then you use Paging list to observe to that table. The first time you observe to that table, also do a request to network. When you receive the response delete all items from the table and insert the new ones (all this in a repository class, and in a background thread). This last step will update your paging list automatically as your paging list is observing to that room table.
Guess this article could be helpful:
https://proandroiddev.com/the-missing-google-sample-of-android-architecture-components-guide-c7d6e7306b8f
and the GitHub repo for this article:
https://github.com/PhilippeBoisney/GithubArchitectureComponents
The heart of this solution is UserRepository:
public LiveData<User> getUser(String userLogin) {
refreshUser(userLogin); // try to refresh data if possible from Github Api
return userDao.load(userLogin); // return a LiveData directly from the database.
}
// ---
private void refreshUser(final String userLogin) {
executor.execute(() -> {
// Check if user was fetched recently
boolean userExists = (userDao.hasUser(userLogin, getMaxRefreshTime(new Date())) != null);
// If user have to be updated
if (!userExists) {
webservice.getUser(userLogin).enqueue(new Callback<User>() {
#Override
public void onResponse(Call<User> call, Response<User> response) {
Toast.makeText(App.context, "Data refreshed from network !", Toast.LENGTH_LONG).show();
executor.execute(() -> {
User user = response.body();
user.setLastRefresh(new Date());
userDao.save(user);
});
}
#Override
public void onFailure(Call<User> call, Throwable t) { }
});
}
});
}
My app is using Firebase and there are almost 200 users live at the a given time. Most of the users are complaining that the data doesn't load. I was using ChildEventListener for obtaining the data which keep the connection alive and reflects live changes. There is a limit of 100 connections in the free plan. I guess that is the reason my data is not loading at times. After reading the doc I found another way to read data using ValueEventListener. Below is the code I'm currently using
public void getImages() {
Query imagesQuery = FirebaseDatabase.getInstance().getReference().child("englishDps").child(mChildName).orderByKey().limitToLast(21);
ChildEventListener childEventListener = new ChildEventListener() {
#Override
public void onChildAdded(DataSnapshot dataSnapshot, String s) {
Image image = dataSnapshot.getValue(Image.class);
image.setNodeKey(dataSnapshot.getKey());
mTempImages.add(image);
if (mTempImages.size() == 21) {
mLastKey = mTempImages.get(0).getNodeKey();
Collections.reverse(mTempImages);
mTempImages.remove(mTempImages.size() - 1);
mImages.addAll(mTempImages);
setAdapter();
}
}
#Override
public void onChildChanged(DataSnapshot dataSnapshot, String s) {
}
#Override
public void onChildRemoved(DataSnapshot dataSnapshot) {
}
#Override
public void onChildMoved(DataSnapshot dataSnapshot, String s) {
}
#Override
public void onCancelled(DatabaseError databaseError) {
if (isAdded()) {
Toast.makeText(getActivity(), "Problem loading more images...", Toast.LENGTH_LONG).show();
}
}
};
ValueEventListener valueEventListener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
for(DataSnapshot imageSnapshot : dataSnapshot.getChildren())
{
Image image = imageSnapshot.getValue(Image.class);
image.setNodeKey(imageSnapshot.getKey());
mTempImages.add(image);
if (mTempImages.size() == 21) {
mLastKey = mTempImages.get(0).getNodeKey();
Collections.reverse(mTempImages);
mTempImages.remove(mTempImages.size() - 1);
mImages.addAll(mTempImages);
setAdapter();
}
}
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
};
// imagesQuery.addChildEventListener(childEventListener);
// imagesQuery.addValueEventListener(valueEventListener);
imagesQuery.addListenerForSingleValueEvent(valueEventListener);
}
According to the docs
"While using a ChildEventListener is the recommended way to read lists
of data, there are situations where attaching a ValueEventListener to
a list reference is useful.
Attaching a ValueEventListener to a list of data will return the
entire list of data as a single DataSnapshot, which you can then loop
over to access individual children.
Even when there is only a single match for the query, the snapshot is
still a list; it just contains a single item. To access the item, you
need to loop over the result:.
This pattern can be useful when you want to fetch all children of a list in a single operation, rather than listening for additional
onChildAdded events."
I was thinking this will solve the data loading problem but my previous version of the app will still keep using live connection and I'm still seeing random success and failures for data load call in new version of the app with more than 150+ users live right now on old version of the app. What will happen if the old version of the app opens more than 100 connection and the new version of the app tries to load data ? i.e. if 100 connections in the free plan are used will a query with
imagesQuery.addListenerForSingleValueEvent(valueEventListener);
succeed or fail ?
When an Android app first uses the Firebase Database SDK, it makes a connection to the Firebase servers. If there are at that moment already as many connection as are allowed to your database, the new connection will be rejected. The type of listener has no influence on this.
For a lot of discussions covering this already, see this list. Some good ones:
Limitation of free plan in firebase
How the Connection is calculated in Firebase
When are new connections allowed after limit of 100 concurrent connection is reached in firebase?
How exactly are concurrent users determined for a Firebase app?
How to limit concurrent connections on Firebase Android
Having looked at your code. I recommend inserting a closing connection once the read of images from json are completed. In the free package there is a limit of connections so once they read the images, they're technically still connected.
Looking at your Datasnapshot, they don't do anything but still querying the Firebase. I also recommend look into indexing too.
https://firebase.google.com/docs/database/rest/save-data
https://firebase.google.com/docs/database/security/indexing-data