I'm building an Android application which has to work offline for weeks, but can sync immediately with a remote DB as it goes online.
My question is can Firestore be a good option for this? How long does Firestore keep its offline cache?
Firestore can be configured to persist data for such disconnected/offline usage. I recommend that you read the enable offline persistence section in the docs, which contains this sample of enabling this feature:
FirebaseFirestoreSettings settings = new FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(true)
.build();
db.setFirestoreSettings(settings);
This persistence is actually enabled by default on Android and iOS, so the call above is not needed.
Your Android code that interact with the database will be the same whether you're connected or not, since the SDK simply works the same. If you want to detect whether data is coming from the cache (and thus potentially stale), read the section Listen to offline data in the docs.
The data in the cache does not expire after a certain amount of time. The only two reasons data is removed from the cache:
The data has been removed from the server, in which case the client will remove it from the disk cache.
The client needs to purge its disk cache to make space for more recent data.
EDIT-25/5/2020: Francesco is correct, the docs link given in the comment does clarify that. It seems the cache size has been changed, by default it has been decreased to 40MB.
OLD: The following answer follows the official guide in the following link:
Handling Cache size
FirebaseFirestoreSettings settings = new FirebaseFirestoreSettings.Builder()
.setCacheSizeBytes(FirebaseFirestoreSettings.CACHE_SIZE_UNLIMITED)
.build();
db.setFirestoreSettings(settings);
The above code has a flag set for setCacheSize(), which will prevent your cache from
being cleared. You can also specify the same in size. If you do not set this by default the size is 100MB.
As per the guide, there is a method to check if the data you query came from cache or the firestore. Also the moment your device is back online the firestore refreshes the cache, and keeps the data synchronized.
To answer your question, as you have to work with offline data for weeks, i suggest every time the data is fetched to store it in json/xml formats, as storing huge amount of data in cache is not a really good approach when thought of in terms of performance.
I hope i helped you clear some things out.
If you listen to data in Cloud Firestore, you will get immediate snapshots of cached data and also updates when your app is able to connect online:
final DocumentReference docRef = db.collection("cities").document("SF");
docRef.addSnapshotListener(new EventListener<DocumentSnapshot>() {
#Override
public void onEvent(#Nullable DocumentSnapshot snapshot,
#Nullable FirebaseFirestoreException e) {
if (e != null) {
Log.w(TAG, "Listen failed.", e);
return;
}
// Determine if the data came from the server or from cache
String source = snapshot != null && snapshot.getMetadata().hasPendingWrites()
? "Local" : "Server";
// Read the data
if (snapshot != null && snapshot.exists()) {
Log.d(TAG, source + " data: " + snapshot.getData());
} else {
Log.d(TAG, source + " data: null");
}
}
});
Persistence is enabled by default so this behavior does not require any configuration.
Related
When the user opens the app, if auth.currentuser isn't null, then I need to query the user's username. I think a query listener makes sense because I don't want to keep pinging the server every time some code asks for the username. I looked at the Firebase docs they provide this example:
val docRef = db.collection("cities").document("SF")
docRef.addSnapshotListener { snapshot, e ->
if (e != null) {
Log.w(TAG, "Listen failed.", e)
return#addSnapshotListener
}
val source = if (snapshot != null && snapshot.metadata.hasPendingWrites())
"Local"
else
"Server"
if (snapshot != null && snapshot.exists()) {
Log.d(TAG, "$source data: ${snapshot.data}")
} else {
Log.d(TAG, "$source data: null")
}
}
However, I was advised to try to use as little client side code as possible to optimize security, so the aforementioned example worries me. Is it possible to use a query listener to listen to a server side function that can output the username so that I can still take advantage only running the query once, i.e. saving the username to the client cache?
Cloud Functions are short-lived, and you pay for the time that they're active. You cannot keep a connection from a Cloud Function open for a long-period of time, like the snapshot listeners that the Firestore server and SDK use.
While it is totally possible to wrap access to Firestore in Cloud Functions, this would follow a more traditional request/response model.
I recommend checking out:
Doug's blog post: Patterns for security with Firebase: offload client work to Cloud Functions.
Doug's blog post: Patterns for security with Firebase: combine rules with Cloud Functions for more flexibility
I am using Firebase Firestore and I'm facing a problem with the read operation:
I use an onCompleteListener, and inside there, I call different callbacks if the operation was successfull or not.
The problem is, if there is a network issue, the onCompleteListener is called, but task.isSuccessfull returns true!! So I get an empty result which I can't distinguish from a REAL empty result. Is there any way to distinguish a network issue from an empty read?
Thank you very much! My function is just below:
dataBase.collection(COLLECTION)
.whereEqualTo(FIELD, searched)
.get()
.addOnCompleteListener { task: Task<QuerySnapshot> ->
if (task.isSuccessful) {
listenerSuccess(task.result)
} else {
listenerError()
}
}
If you're offline, the client will first try to connect. Once it figures out that it can't connect, it will try to complete the get() based on the data in the local database. That's a valid action on Firestore, so that's why the task is completed successfully.
You can detect if the results came from the local cache vs which came straight from the server, by checking the metadata: querySnapshot.getMetadata().isFromCache(). From the docs:
true if the snapshot was created from cached data rather than guaranteed up-to-date server data. If your listener has opted into metadata updates (via MetadataChanges.INCLUDE) you will receive another snapshot with isFomCache() equal to false once the client has received up-to-date data from the backend.
I'm using Firestore with the Android SDK (11.6.2) and I'm hitting an exception when my device was offline and reconnects to the Internet.
When requesting a document, firestore fails with the following task exception :
com.google.firebase.firestore.FirebaseFirestoreException: Failed to get document because the client is offline.
However, the device is connected, I can make network requests beside using Firestore and they succeed.
This error is not consistent, sometimes the request will succeed right after reconnecting to the Internet. Sometimes, the request fails again and again, then succeeds, sometimes more than one minute after the device has been reconnected to the Internet.
Here is a sample request that produces the exception:
val docRef = firestore.collection("foo").document("bar")
docRef.get().addOnCompleteListener { task ->
if (task.isSuccessful) {
Log.d("FirestoreSourceSet", "Get document success")
} else {
Log.e("FirestoreSourceSet", "Get document error", task.exception)
}
}
I'm not using the offline capabilities of Firestore, thus the FirebaseFirestore instance is initialized the first time with the setPersistenceEnabled(false) flag:
val firestoreSettings = FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(false)
.build()
val firestore = FirebaseFirestore.getInstance().apply {
this.firestoreSettings = firestoreSettings
}
Why is Firestore returning this error even though the device is online? Am I missing something in the Firestore configuration that would avoid this error?
I tried upgrading Firebase to the 11.8.0 version, but I encounter the same behavior.
Logs
These are the logs while trying to fetch sync some data with Firestore (which begins with a document fetch) after leaving airplane mode: https://pastebin.com/xDMG2Pzj
The network is already available before the first Firestore call, as I waited for the Wifi to settle, and check it using the ConnectivityManager of Android before proceeding with Firestore.
The multiple calls are because I manually retry using a button each time I get the error until the document is successfully retrieved.
The first line of the log is when I turn the airplane mode one, which closes the stream of Firestore.
Edit: Issue
Firebase doesn't have a public tracker, but I reported the issue using their report tool, and made a repo that reproduces the issue.
They acknowledged the issue but could not provide an ETA, so we have to wait:
If we release the fix, we will announce it in our release notes page.
This is still an issue as of firestore-core 17.0.4
From the official documentation:
To use offline persistence, you don't need to make any changes to the code that you use to access Cloud Firestore data. With offline persistence enabled, the Cloud Firestore client library automatically manages online and offline data access and synchronizes local data when the device is back online.
When you initialize Cloud Firestore, you can enable or disable offline persistence. So, when using the following line of code:
val firestoreSettings = FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(false)
.build()
You actually set the persistence to false, you are disabling this feature. To solve this, just remove this line of code. Firestore has .setPersistenceEnabled(true) by default.
This issue was fixed by the release 17.1.5 of Cloud Firestore.
See the official changelog https://firebase.google.com/support/release-notes/android
Cloud Firestore now recovers more quickly from bad network states.
Using my reproduction project with the version 18.0.0, the issue is indeed not present.
I'm facing the same issue since 2018 and have not yet found a proper solution. It is still happening with version 22.1.2 of the Firestore library for Android. As commented in the source code of DocumentReference, this exception is thrown when the client is offline and the requested document is not available in the local cache. There is also a recent explanation of this behaviour by Denver Coneybeare.
This exception might occur for DocumentReference.get() calls. Here's an exemplary stack trace:
com.google.firebase.firestore.FirebaseFirestoreException: Failed to get document because the client is offline.
at com.google.firebase.firestore.DocumentReference.lambda$getViaSnapshotListener$1(DocumentReference.java:331)
at com.google.firebase.firestore.DocumentReference$$Lambda$2.onEvent(DocumentReference.java:8)
at com.google.firebase.firestore.DocumentReference.lambda$addSnapshotListenerInternal$2(DocumentReference.java:504)
at com.google.firebase.firestore.DocumentReference$$Lambda$3.onEvent(DocumentReference.java:16)
at com.google.firebase.firestore.core.AsyncEventListener.lambda$onEvent$0(AsyncEventListener.java:42)
at com.google.firebase.firestore.core.AsyncEventListener$$Lambda$1.run(AsyncEventListener.java:2)
at com.google.firebase.firestore.util.Executors$$Lambda$1.execute(Executors.java)
at com.google.firebase.firestore.core.AsyncEventListener.onEvent(AsyncEventListener.java:39)
at com.google.firebase.firestore.core.QueryListener.raiseInitialEvent(QueryListener.java:176)
at com.google.firebase.firestore.core.QueryListener.onViewSnapshot(QueryListener.java:95)
at com.google.firebase.firestore.core.EventManager.addQueryListener(EventManager.java:97)
at com.google.firebase.firestore.core.FirestoreClient.lambda$listen$6(FirestoreClient.java:160)
at com.google.firebase.firestore.core.FirestoreClient$$Lambda$6.run(FirestoreClient.java:12)
at com.google.firebase.firestore.util.AsyncQueue.lambda$enqueue$2(AsyncQueue.java:436)
at com.google.firebase.firestore.util.AsyncQueue$$Lambda$2.call(AsyncQueue.java:1)
at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor.lambda$executeAndReportResult$1(AsyncQueue.java:322)
at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$$Lambda$2.run(AsyncQueue.java:1)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$DelayedStartFactory.run(AsyncQueue.java:229)
at java.lang.Thread.run(Thread.java:919)
I'm sorry if this answer does not solve the problem, but this information would have been too long for a comment on the original post.
I also have the same problem. There is workaround - application waits some time and makes another attempt to get the data from server.
Here is mockup (this is Flutter/dart):
List<Course> items;
Widget build(BuildContext context) {
...
loadData();
...
//building th UI with data from items list
...
}
loadData() async {
//is not loaded but there is internet connection
if (!isLoaded&&isConnected) {
try {
querySnapshot = await query.getDocuments(source: Source.server);
}
catch (ex) {
print(ex);
Future.delayed(Duration(seconds: 10), () {
// setState to trigger build 10 seconds later to make one more attempt
setState(() {
isLoaded = false;
});
});
return;
}
// here goes the code to handle the result
items = querySnapshot.documents.map((s) => Course.fromDb(s)).toList();
}
this may help
Java:
FirebaseFirestore db = FirebaseFirestore.getInstance();
//to reconnect
db.terminate();
db = FirebaseFirestore.getInstance();
Kotlin:
var db = Firebase.firestore
//to reconnect
db.terminate()
db = Firebase.firestore
I'd like to use AWS AppSync for mobile development (Android/iOS) but I’m not sure about its offline capabilities.
According to the documentation the data will be accessible while being offline and synced automatically if the client gets online again. But I can't find any information about if the app client needs to connect to AWS first, before using AppSync to create and modify offline data.
I'm not familiar with the underlying technologies of AppSync (e.g. GraphQL) and I don't have access to the public preview version to test it myself.
I would like to enable privacy-sensitive users to use an app without connecting to AWS while still being able to use AppSync as an offline database. Only if a user later decides to use backup/sync data across devices he or she can opt-in to connect to AWS.
Will this use case be possible with AWS AppSync?
Without using any other local storage (like SharedPreferences, SQLite, Realm, etc.)
It should be possible with Firestore, AWS AppSync or your own Backend solution. Any approach you use, you will control when you want to start saving/syncing things online.
You need to handle all this while designing this app. Suggested approach
Let's take example of simple ToDo app
I will let User add & save Todos in app
All this data will be persisted on disk(using SQLLITE, Preferences or File etc.)
If User does clear data or reinstall app, all this data is lost
If User wants to go premium, I will let him sync this data with my Backend solution(any one of above-mentioned solution)
Example implementation using Android Shared preference as local storage
public void saveLocalTodo(String title, String details) {
ArrayList<Todo> todos;
Todo todo = new Todo(title, details);
String listOfTodo = sharedPreference.getString(TODOS_LIST, null);
if (listOfTodo == null)
todos = new ArrayList<Todo>();
else
todos = gson.fromJson(listOfTodo, new TypeToken<ArrayList<Todo>>() {
}.getType());
//save at 0th position, recent should always come first
todos.add(0, todo);
sharedPreference.edit().putString(TODOS_LIST, gson.toJson(todos)).apply();
}
public ArrayList<Todo> getLocalTodos() {
ArrayList<Todo> todos;
String listOfTodos = sharedPreference.getString(TODOS_LIST, null);
if (listOfTodos == null)
todos = new ArrayList<Todo>();
else
todos = gson.fromJson(listOfTodos, new TypeToken<ArrayList<Todo>>() {
}.getType());
return todos;
}
public void saveOnBackend() {
// Connect to Backend solution
// Get all local todos from preference
// Save all at once in batches
//OR
// Get all local todos from preference
// Save one by one
}
Use Realm Database to manage all offline and online data and save if the application uninstall
you can read
https://docs.aws.amazon.com/appsync/latest/devguide/building-a-client-app-reactnative.html
AWS AppSync support offline mode
and you can use data base for your app
I am working on firebase for the first time, read about offline capabilities of firebase , tested two scenarios :
scenario 1 (working):
offline mode, writing data to firebase database.
press back button(closed app)
went online, data got added to the firebase database.
scenario 2 ( not working):
offline mode, writing data to firebase database
close app
remove app from background(killed the application)
went online, data not getting added
I added this line:
Firebase.getDefaultConfig().setPersistenceEnabled(true);
how to handle scenario 2 ? Do I need to handle this scenario through local database?
Are you using Firebase.getDefaultConfig().setPersistenceEnabled(true); and keepSynced(true)?
Because in Firebase documentation says that keepSynced(true) it's who make the "magic" happens (together with setPersistenceEnabled(true)):
By calling keepSynced(true) on a location, the data for that location will automatically be downloaded and kept in sync, even when no listeners are attached for that location. Additionally, while a location is kept synced, it will not be evicted from the persistent disk cache.
So, if you're not using it, you're not persisting your database locally and then when you "kill" the application, there will not be any database to query from when your app is opened again.
I guess your using some service to sync the data, It will not work for 2nd scenario. For that when user turn on data services you will receive a broadcast receiver, from that check service is not running then start the service.
No need to handle scenario 2 using local database . Use Firebase.getDefaultConfig().setPersistenceEnabled(true) in application class and make android:name="yourapplicationclass" in manifest file.
to handle sync while change network ie online/offline use transaction handler to handle local sync to firebase database since some tome data is not pushed to firebase server.Like this I used inside network change method and solved this issue:
mDatabase.runTransaction(new Transaction.Handler() {
#Override
public Transaction.Result doTransaction(MutableData mutableData) {
if(mutableData.getValue() == null) {
mutableData.setValue(1);
} else {
mutableData.setValue((Long) mutableData.getValue() + 1);
}
return Transaction.success(mutableData); //we can also abort by calling Transaction.abort()
}
#Override
public void onComplete(DatabaseError databaseError, boolean b, DataSnapshot dataSnapshot) {
}
});