I'm working on a Flutter app that uses Firebase RTDB. I'm a bit unclear on the mechanism of keepSynced():
Our app creates new collections and adds data to these collections. The app also observes these collections to display the data. In the process, we call keepSynced(true) in order to keep the offline cache alive for these queries. Now we noticed that, when creating a new collection + calling keepSynced(true) while offline, the listener we register on the new collection doesn't receive data until the app goes online again. From then on, the caching and updating the listeners works as expected (online + offline).
So, in code:
// Execute this while offline:
// Enable permanent offline sync for a new node that doesn't exist yet on the Firebase server
await database.ref().child('new_node').keepSynced(true);
// Register listener
database.ref().child('new_node').onValue.listen((event) => print(event));
// Add data
database.ref().child('new_node/child').set({'key': 'value'});
The listener will trigger only after the device goes online.
The documentation on keepSynced() states
By calling keepSynced(true) on a location, the data for that location will automatically be downloaded and kept in sync, even when no listeners are attached for that location. Additionally, while a location is kept synced, it will not be evicted from the persistent disk cache.
So is it actually required to be connected with the Firebase backend ("will automatically be downloaded") while calling keepSynced() in order for the offline caching/querying to work?
You can call keepSynced at any moment, but the client will only be able to download the data from the server when it is connected to the server.
If that is not the behavior you're observing, please edit your question to include a minimal repro.
Update based on the new repro in your question:
The behavior you're seeing is unfortunate, but expected.
The Firebase API guarantees (undocumented these days) that it will never surface a partial node to your app. Since you're only setting /new_node/child, and it has no knowledge of the current state of /new_node, it doesn't raise a value event on new_node since in only has partial information about that node.
If you were to instead listen for child_added on new_node, the SDK would raise an event in this scenario - but it won't raise a value event since until it can ascertain that it has a complete snapshot of new_node.
The two most common paths forward are:
Listen at a lower level in your JSON tree, for example by listening to child_* events (as mentioned above).
Perform an initial write operation on all of new_node, so that the local SDK knows what the expected initial state of the node is against it can then perform the lower-level writes. You could possibly then use security rules to reject this priming write operation on the server if there is also data that can't be wiped.
Related
See here: https://firebase.google.com/docs/database/android/offline-capabilities
I use Firebase Realtime Database to connect a Google Cloud backend with an Android app written in Kotlin. I use offline persistence. I noticed something strange: Say I have 2 endpoints, ep1 and ep2. Then say I issue these commands in the Android app:
Firebase.database.setPersistenceEnabled(true)
FirebaseDatabase.getInstance().reference.child("ep1").keepSynced(true)
Then what I observe is that changes to ep2 are not seen by the Android app.
Am I doing something wrong?
FirebaseDatabase#setPersistenceEnabled(boolean isEnabled) method:
The Firebase Database client will cache synchronized data and keep track of all writes you've initiated while your application is running.
This means that all the writes that you're performing while you're offline, are added to a queue in the local cache. As soon as you regain connectivity, all operations are synchronized with the Firebase servers.
On the other hand, when you're using the following line of code:
FirebaseDatabase.getInstance().reference.child("ep1").keepSynced(true)
Query#keepSynced(boolean keepSynced) method:
By calling keepSynced(true) on a location, the data for that location will automatically be downloaded and kept in sync, even when no listeners are attached to that location.
As the name says, this method keeps your above reference synchronized in the local cache. What it actually does, it literally just attaches an empty listener to the ep1 reference.
Then what I observe is that changes to ep2 are not seen by the Android app.
If there is no listener attached to the ep2 reference, or you don't call .keepSynced(true), you cannot read the data at the location. That's why you cannot see it. So you either attach a listener yourself, in your application code or call .keepSynced(true) as you do in the case of ep1.
But remember, calling .keepSynced(true) has nothing to do with the offline persistence mechanism. When you call setPersistenceEnabled() it means that you enable/disable the option to have a local cache while calling .keepSynced(true), it basically attaches a listener to the reference/query it points to.
I have switched off the mobile network and setPersistenceEnabled to false(disable the cache).
Then add the data to Firestore by calling add().
The add() operation is success and the data is showed on the screen by FirestoreRecyclerAdapter.(The mobile network is still disconnected).
Is the behavior correct ? I think the add() operation should be failed and nothing to show on the screen due to connection lost.
That's expected behavior. Whenever you change a document, the Firestore SDK broadcasts that change for local listeners, regardless of your settings for local persistence. The local change is visible to listeners without any need for a network connection. When the network becomes available again, the local change will be synchronized.
When persistence is disabled, that means local changes are not saved to disk. In that case, when your app process is killed, any changes that you made locally will be completely lost.
This behavior doesn't have anything to do with Firebase UI specfically. It's a a core behavior of the Firestore SDK.
I'm using Firebase's realtime database on Android and the way I understand how it works is that even if the app disconnects from the network, Firebase will simply queue the transactions that the user has initiated and then perform then when connectivity is resumed. This works really well but if the app is closed then this queue seems to be discarded.
The Firebase docs on handling offline capabilities states the following:
Transactions are not persisted across app restarts
Even with persistence enabled, transactions are not persisted across
app restarts. So you cannot rely on transactions done offline being
committed to your Firebase Realtime Database. To provide the best user
experience, your app should show that a transaction has not been saved
into your Firebase Realtime Database yet, or make sure your app
remembers them manually and executes them again after an app restart.
But as far as I know, there is no way of knowing whether or not data has finished being written to the database.
How exactly would you go about making the app manually remembering what still needs to be written to the database? Is there some way of accessing the queue of transactions that is yet to be carried out? Or is there some way of keeping the app running in the background after being closed that could just sync the data when connectivity resumes?
Thanks in advance.
But as far as I know, there is no way of knowing whether or not data has finished being written to the database.
There actually is. The Transaction.Handler interface has a [onComplete method](https://firebase.google.com/docs/reference/android/com/google/firebase/database/Transaction.Handler.html#onComplete(com.google.firebase.database.DatabaseError, boolean, com.google.firebase.database.DataSnapshot)). The boolean that is passed to that argument is a flag to indicate if the transaction was committed:
committed
True if the transaction successfully completed, false if it was aborted or an error occurred
For more information, see the Firebase documentation on transactions.
I think I had the problem you are facing, in my case was a simple confusion. That Firebase warning is not about "transactions" in general, is about the "transaction" method provided by them.
In Android this is reference().runTransaction().
The "transaction" method is used to validate data first, by example, if more than one user can subscribe to an event simultaneously, you can make sure that the last vacant was available.
Since the "transaction" method query the database gives you the data, and the upload data, if there is no network connectivity there is no way to make sure that will work on app restart because there was never a first query to see the data you have to validate.
This seems logical to me, a "transaction" method will create a sort of bridge between the client and the database, this is not random, but because is part of the business logic, then you should warn the user visually that their changes might not be saved since it is offline, or even if it is sensitive not allow the user to do it.
In other cases, the data is indeed stored locally and then uploaded when the app is restarted. So if you do something like
reference.child(key).setValue(myObject);
Thant change will be local until the next time user has an internet connection.
You have to make sure to add the keepSynced to the references you actually need. Setting the syncing to the root, won't solve the problem as a waterfall, make sure to be specific with nodes you need to keep synced, this way the user will see the changed reflected visually in the app.
//Won't work
DatabaseReference root = FirebaseDatabase.getInstance().getReference();
root.keepSynced(true);
//This will work
root.child("event_list").keepSynced(true);
root.child("user_events").child(uid)keepSynced(true);
I thought the whole time when I used the following all data for chat conversation will be available offline at any time. Which somehow isn't and all nodes are loaded from the server.
FirebaseDatabase.getInstance().setPersistenceEnabled(true);
Then, according to DOCS :
Persistence Behavior : By enabling persistence, any data that we sync while online will be persisted to disk and available offline, even when we restart the app. This means our app will work as it would online using the local data stored in the cache.
If data will already be available offline when setPersistenceEnabled(true); , why do I need keepSynced(true) ?
DatabaseReference r = FirebaseDatabase.getInstance().getReference("chat/roomid");
r.keepSynced(true);
The Firebase Database client in your app can keep the data from the database in two places: in memory and/or on disk.
When you attach a listener, it synchronizes data from the database to an in memory representation in your app.
If you've enable persistence, the data is automatically also persisted on disk.
When you detach the last listener from a location, the data for that location is flushed from memory. But it is not deleted from disk.
When you keep a location synchronized, the client essentially attaches an empty listener to that location. So the data in the app will always be up to date with what's in the database on the server (as long as there is a network connection). If you haven't enabled persistence, the data will just be kept up to date in memory. If you've enabled persistence, it will also be kept up to date on disk.
While it's most common to use keepSynced with persistence, there are also use-cases without persistence.
For example, if you have a master-detail app, where you frequently bounce from a list of item names to the details of each item. In that case keeping the list of item names synchronized will save you from having to reload that data when the user comes back from the detail screen.
You could of course also simply keep a listener on the data, which is essentially what keepSynced does behind the scenes.
According to Firebase documentation
By default the Firebase client will keep data in memory while your application is running, but not when it is restarted. By setting this value to true, the data will be persisted to on-device (disk) storage and will thus be available again when the app is restarted (even when there is no network connectivity at that time). Note that this method must be called before creating your first Firebase reference and only needs to be called once per application. If your app uses Firebase Authentication, the client will automatically persist the user's authentication token across restarts, even without persistence enabled. But if the auth token expired while offline and you've enabled persistence, the client will pause write operations until you successfully re-authenticate (or explicitly call unauth) to prevent your writes from being sent unauthenticated and failing due to security rules.
Note that it says the data will persist on disk and be available when apps restart. If you look at the life cycle of an Activity, you'll see that an activity stop when you change to other activity. So, as long as your app is still open and user only navigated to others activities, data will remain persisted.
But don't say anything about killing your app and persist data. That's why you need keepSynced():
By calling keepSynced(true) on a location, the data for that location will automatically be downloaded and kept in sync, even when no listeners are attached for that location. Additionally, while a location is kept synced, it will not be evicted from the persistent disk cache.
Pay attencion when it says "while location is kept synced, it will not be evicted from the persistent disk cache", this means that if you don't use keepSynced(true) your data could be flushed way when the app is killed/closed.
So, to persist and continue with data after your app is killed, you need to use both FirebaseDatabase.getInstance().setPersistenceEnabled(true); and keepSynced(true).
setPersistanceEnabled(true) It will store the data for offline use.
keepsynced(true) by default, Firebase keeps 10mb data in cache,if it grows further it will replace by new data.To avoid the same and keep the whole data keepsynced(true) will help You.
My Firebase database has setPersistenceEnabled=true. When offline all the listeners work fine and fetch data from the local cache.
But, when online, Firebase is making a network call each time I use a addListenerForSingleValueEvent on the same node, even though I have a local cache and nothing has changed on the server.
I have two items at the node I'm attaching the listener to, and I see the following reported after setting log level to debug. This network activity is repeated every time I use the listener while online, even within sub second delays. My understanding is that if Firebase has local data, then server calls won't be made. Any sync checks might happen in the background and with efficient network usage.
conn_18 - received data message: {r=20, b={s=ok, d={}}}
conn_18 - received data message: {r=21, b={s=ok, d=}}
I am trying to aggressively reduce network usage on my app, and any suggestion to stop/reduce Firebase network activity would be very helpful.
Been digging into this myself.
As far as I can tell, the second call you are experiencing is simply the network saying "Hey, there's no updates", and you're not charged for any reads.
As far as reducing the network activity, I think this is just how it works, and how you want it to work (its the actual listening part of the listener). If you don't want that, use a getDocument call, rather than a listener.
This is based on a few things:
When you look at snapshot.metadata.isFromCache (iOS SDK) for a query, the call will come twice. first from cache and then from the server. However, if you print a statement under snapshot.documentChanges, ONLY the cached call will print.
I did an experiment re-loading a view in iOS ~20 times. I also made on small change to my data. I then waited 5-10 minutes and looked at the usage on: https://console.cloud.google.com/firestore/usage. My total read count went up by 3, which means I was only charged for the update. Reloading the view a bunch of times was NOT charged.