I am currently using Firebase Firestore as a primary backend that retrieves data from a variety of sources. I also use Android's Room for my mobile backend. When the phone receives data it is stored in the Room database in the event the user will not go online again for days even weeks.
After looking through the device files, I see firestore saves the data in files under the /data/data/<your-app>/databases directory.
The file looks something like this
I have read the offline persistence docs on the firestore and there is no indication on how durable the offline persistence is It mentions that the data is cached but not for how long. My question is, what is the durability of Firestore's offline persistence. Would one recommend using it instead of having a fully-fledged local DB to store data that may not be synced over long periods of time (days,weeks)?
It seems to already handle syncing data well once a connection is re-established. Im just worried that after some point that file may be deleted by the system and the user loses everything.
On Android (as of this writing) Firestore uses SQLite as a persistence mechanism. So for intermittent periods of offline activity you should have no problems with performance or durability.
However if you are going to be offline for days or weeks (as you said) there are some things you should be aware of:
Performance
Because Cloud Firestore is meant to be used mostly online, pending writes that have not yet been synced to the server are held in a queue. If you do many pending writes without going online to resolve them, that queue will grow and it will slow down your overall read/write performance. Most of Cloud Firestore's performance guarantees come from indexing and replication on the backend, but most of those optimizations don't exist when you're operating offline-only.
Conflicts
Firestore's basic conflict resolution model is "last write wins". So if you have many offline clients writing to the same document, only the last one to come online will actually "win" and persist their change.
Features
Most of Firestore's features work offline with one major exception: transactions. Transactions can only execute when you are online. So if your app uses transactions it will not work properly offline without some special handling.
There is no indication in offical documentation on how durable the offline persistence is because it cannot be predicted. This question cannot have an exact answer, like 4 weeks or something like this because it depends on how many write operations take place while you are offline.
I recommend you not to use Cloud Firestore as an offline-only database. It's really designed as an online realtime database that can work for short to intermediate periods of being disconnected.
While offline, it will keep queue of all your write operations. As this queue grows, local operations and app startup will slow down. But you need to know that these operation will persist even if you restart the device. You not gonna lose any data.
Related
I am building an app for a gym which shows gym members info in a recyclerview at the app startup , the problem is at every fresh launch of the app the data is pulled from the firebase database and appears suddenly after 4-5 seconds in the recyclerview after the app starts, and i think this time will increase as the data grows.
If the data is same why pull it every time.
But i want to go one step further
what i want is when the user logs in for the first time all the data liked to that account should get pulled off and stored locally , i don't want to pull the data ever again even if the user is connected to internet, the changes user make should be made to realtime database as well as locally regardless how much data is created , so that i never need to pull data again once the user is logged in but just push the data every time it changes or grows or gets deleted, operations will happen to both local storage as well as at realtime database, so the user will always have local access to data he created.
And if he logs in again same thing happen again.
Is there a way to do it with firebase without implementing local sqlite database together with firebase realtime database.
This answer is for a use case where data is stored locally as well as on a remote server - similar to what's in the question. It may not specifically answer the question but may lead to a solution.
In the question, it appears there's local data and remotely stored data which are essentially the same - the OP is using sql to store locally and Firebase to store remotely. This is being done to 'speed up' the loading process and lower the amount of reads.
My suggestion is to use Firebase only. It's very fast and if coded correctly, the loading time should never be 4-5 seconds before the UI is available to the user (barring a bad internet connection if not using persistence).
Additionally, Firebase offers locally persisted data in two ways; cached data and persisted data (which are similar but read on)
Cached: Firebase apps automatically handle temporary network interruptions.
Cached data is available while offline and Firebase resends any writes
when network connectivity is restored.
and
Persisted: By enabling persistence, any data that the Firebase Realtime Database
client would sync while online persists to disk and is available
offline, even when the user or operating system restarts the app. This
means your app works as it would online by using the local data stored
in the cache. Listener callbacks will continue to fire for local
updates.
There's a lot more to it so please see iOS Offline Persistence and Enabling Offline Capabilities on Android
Your questions suggests that the Firebase SDKs provides this sort of functionality. It does not. You will have to implement something yourself.
The best way to tackle this in an easy and the most efficient way is to use the Room Persistence provided by AndroidX Architecture.
It is technically an ORM over SQLite. Please find the details here:
AndroidX Room Persistence
I'll like to know if it is possible to increase the size of the local cache of the firebase firestore database?
Will also like to deepen my knowledge on how firebase firestore offline
data persistence functions on mobile( android specifically).
Let me explain my use case and you evaluate if firebase firestore is the route to go for me.
I require a local database that can be synced averagely after every 24 hours and I require it to function offline and persists it's data. The main use case is this: I have a messaging feature integrated into the application and just as everyone expects to see his messages he's got to send and receive before he was disconnected that is, just as even when one is disconnected, he still can see his different conversations on whatsapp even after a phone reboot, that's how i expect my application to follow and I want to know if firebase firestore local cache offers me this possibility of persisting the cache even after phone reboot without having to connect to the internet? I do expect a lot of reads from the local cache but not that of writes while offline.
To summarize my question, can the local cache persists changes that have been made while the phone was offline even after phone reboots without any connection to the internet? Thank you all for your answers and time taken to read this in advance.
Please, do include some helpful links to tutorials showing how to use it if you do know any of them.
The size of the local cache depends on the size of the storage of your device. If you want to increase the size of your local storage, then you just need to free some space or buy a new hardware. There is no limitation regarding the maximum size that can be stored on your device.
Cloud Firestore supports offline data persistence. This feature caches a copy of the Cloud Firestore data that your app is actively using, so your app can access the data when the device is offline. You can write, read, listen to, and query the cached data. So, Cloud Firestore persists the data you write on a device to a local database. So the next time you start the application, it will indeed be able to read that same data, even when the device has never been connected to the Firebase servers. And as an answer to your question, yes it can.
But don't use Cloud Firestore as an offline-only database. It is really designed as an online database that came work for short to intermediate periods of being disconnected. While offline it will keep a queue of all your write operations. As this queue grows, local operations and app startup will slow down.
Nothing major, but over time these may add up. But remember, all these operation will persist even if you restart the device. You not gonna lose any data.
I'm having sync problems on my mobile application. The app works on-line and off-line.
The main issues consists on sync time, data crush and sync interrupts.
I have an REST API that serves data and business logic to mobile device. The Rest API has also the authentication mechanism of the app.
To sync, the mobile app has an background job (battery consumption problem) that runs every 30 seconds. This process the data stored on local database (SQLLite) and makes some rest calls for each operation. Some operations requires to write on different entities which it could originate several calls for each operation since the rest api it is entity oriented based on ODATA protocol.
At the moment, I'm studying new approaches to solve some of this problems.
I think, I want to entry on real-time database on the current the architecture...
I have some scenarios in mind, but my lack of knowledge on this kind of architectures makes me had some doubts about the best scenario.
My doubts are:
If I add an real-time database (like firebase, realm or others...),
should I keep the SQLLite database on the mobile app?
Since I already have an rest api to manage the data how this should communicate with the real-time db? should be my current rest
api "ask" to the real-time db the updates?
It is possible to implement an action on the real-time db (firebase or others) that notifies my current API?
Should the web evolve to use the real-time db also?
For what I found on the web, exists a lot of scenarios that can be followed depending on each case. However I don't find any clear scenario for my case.
In abstract, my app needs to work on-line and off-line and syncs fully with an server api.
The current implemented sync mechanism on the mobile app it may be not the best. Some operations has to make some calls to api and wait for the answer to process the next calls. Exists some business logic to process these calls. On server side some requests can be a little slow depending on the volume of the data to process.
Initial, my point is to improve the performance of the app with no sync issues.
I have been using SQLite as my storage solution for my android applications. I want to be able add synchronization functionality to one of my apps, Firebase looks like a good solution but the problem is that I need to know if I am to use Firebase to sync data with the SQLite database or Firebase can work and totally replace SQLite. I know Firebase has offline persistence but while offline can it hold as much data as SQLite and are the queries as powerful?
Clearly this depends on the business rules for which the options are considered. So there will not be the "right" answer to this.
At least one of our team members was thinking along these lines. Here is our solution:
Do all transactional data locally on the SQLite as we do not need transactions to be across devices.
Sync the rest using Firebase.
(1) makes sure that we don't misuse Firebase's non-persistent offline availability of its client.
The docs
Even with persistence enabled, transactions are not persisted across app restarts. So you cannot rely on transactions done offline being committed to your Firebase Realtime Database. To provide the best user experience, your app should show that a transaction has not been saved into your Firebase Realtime Database yet, or make sure your app remembers them manually and executes them again after an app restart.
(2) make sure that we use Firebase's persistent offline nature to synch non-transactional data across devices once connection is (re)established.
The docs, The Firebase Realtime Database client automatically keeps a queue of all write operations that are performed while your app is offline. When persistence is enabled, this queue is also persisted to disk so all of your writes are available when the user or operating system restarts the app. When the app regains connectivity, all of the operations are sent to the Firebase Realtime Database server.
This way we achieve a state acceptable within our use.
"can it hold as much data as SQLite and are the queries as powerful?" no and no if powerful is translated as "transaction across devices", by definition of Firebase being a remote database.
Consider a blogging application which allows offline mode in free version. And once user subscribes for paid version - the data is being synced to firebase.
The question is - since firebase has offline capabilities - can it work (like Parse) that the queries will explicitly use local storage when querying for data? (save/read). So that paid sync can be just a feature flag
Because from what I can tell by skimming the docs, the offline capability seems to be only for "store offline until I'm online" scenarios
Thanks
The Firebase Database is primarily an online database, that continues to work while the user is offline.
While the user is disconnected, Firebase queues the local writes operation in memory (and if you call setPersistenceEnabled(true) to disk). The way this works means that local-only performance will get worse as the local write queue grows.
So unless you have some reasonable maximum to this local number of write operations, your scenario may not work well on Firebase's offline architecture.