Develop Reference

Amplify.observe not updating based on changes to DynamoDB

I am currently making a call to my server to make a change in the Datastore and listening for another change to it. I do not receive any new changes from the DB despite the database being changed correctly on the backend.
I recently upgraded amplify from 1.4.2 to 1.6.8 and this is when the issue started to show up.
Amplify.DataStore.observe(Profile::class.java,
{ observationCancelable ->
//ERROR
},
{ changedItem ->
when (changedItem.item().state) {
//DO Business logic
}
},
{ exception ->
//ERROR
},
{
}
)
When I query the datastore I am getting outdated results that are not the same as the data on DynamoDB. Is there any way to figure out why the local datastore is not being updated? Could it be an issue with the configuration with the server? Or is there some setup step that I have missed?
Edit: Added some extra details below
So a few more details I wanted to add to this that seem way more relevant as well as updates based on my research. I am switching between endpoints(dev to testing environments). It would seem that the dev environment is working fine with the application, using the datastore correctly and doing the proper AppSync. But in the new testing environment, when the app starts, it gets the latest version of the datastore but it fails to do any AppSync despite being able to make changes to the backend.
Here is an error I am getting when the app attempts to subscribe to the Profile object in the back end
amplify:aws-datastore: Unauthorized failure for ON_CREATE Profile
amplify:aws-datastore: Releasing latch due to an error: Subscription error for Profile: [GraphQLResponse.Error{message='Not Authorized to access onCreateProfile on type Subscription', locations='null', path='null', extensions='{errorType=Unauthorized}'}]
Is there some config file that I should be looking at to compare with to make sure that the endpoints match or some access key that the app needs?

Use RxJava in queue-like manner and pause/start queue

I'm trying to use RxJava in my Android application, along with Retrofit, to interact with a RESTful API.
In my Android app I sent out a number of requests at various UX events. If one of the request returns an 'Invalid Token' error, I want to pause any other requests that get queued before they start so that I can renew the user's token, and then resume the paused requests.
Is this possible using RxJava? I'm just learning the library and am having trouble finding this functionality.
Thanks,

If these requests are sent in parallel, then you will likely get multiple "Invalid Token" errors. You would want to refresh the token for only the first instance of the error. To pause requests while the token is being refreshed, think about the source of the valid token being an observable.
Let's assume you have a network handling class that has methods:
Single<Response> makeNetworkRequest(Request request) { ... }
Single<Boolean> getToken() { ... }
It is implemented such that a value is emitted when a token is available, but otherwise waits until the token is refreshed.
Then, your observer chain will look something like:
observerChain
.flatMapSingle(request -> networkHandler.makeNetworkRequest(request)
.retryWhen(error -> error.flatMapSingle(e -> networkHandler.getToken()))
...
The retryWhen() operator recovers from the error by providing an observable that, when non-empty, will resubscribe to the upstream observable, in this case, the makeNetworkRequest().
Since you don't show any of your code that you are trying to adapt, you will have to make adjustments to the code above to get it to work with your application.

Firestore fails to retrieve data after reconnecting to the Internet

I'm using Firestore with the Android SDK (11.6.2) and I'm hitting an exception when my device was offline and reconnects to the Internet.
When requesting a document, firestore fails with the following task exception :
com.google.firebase.firestore.FirebaseFirestoreException: Failed to get document because the client is offline.
However, the device is connected, I can make network requests beside using Firestore and they succeed.
This error is not consistent, sometimes the request will succeed right after reconnecting to the Internet. Sometimes, the request fails again and again, then succeeds, sometimes more than one minute after the device has been reconnected to the Internet.
Here is a sample request that produces the exception:
val docRef = firestore.collection("foo").document("bar")
docRef.get().addOnCompleteListener { task ->
if (task.isSuccessful) {
Log.d("FirestoreSourceSet", "Get document success")
} else {
Log.e("FirestoreSourceSet", "Get document error", task.exception)
}
}
I'm not using the offline capabilities of Firestore, thus the FirebaseFirestore instance is initialized the first time with the setPersistenceEnabled(false) flag:
val firestoreSettings = FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(false)
.build()
val firestore = FirebaseFirestore.getInstance().apply {
this.firestoreSettings = firestoreSettings
}
Why is Firestore returning this error even though the device is online? Am I missing something in the Firestore configuration that would avoid this error?
I tried upgrading Firebase to the 11.8.0 version, but I encounter the same behavior.
Logs
These are the logs while trying to fetch sync some data with Firestore (which begins with a document fetch) after leaving airplane mode: https://pastebin.com/xDMG2Pzj
The network is already available before the first Firestore call, as I waited for the Wifi to settle, and check it using the ConnectivityManager of Android before proceeding with Firestore.
The multiple calls are because I manually retry using a button each time I get the error until the document is successfully retrieved.
The first line of the log is when I turn the airplane mode one, which closes the stream of Firestore.
Edit: Issue
Firebase doesn't have a public tracker, but I reported the issue using their report tool, and made a repo that reproduces the issue.
They acknowledged the issue but could not provide an ETA, so we have to wait:
If we release the fix, we will announce it in our release notes page.
This is still an issue as of firestore-core 17.0.4

From the official documentation:
To use offline persistence, you don't need to make any changes to the code that you use to access Cloud Firestore data. With offline persistence enabled, the Cloud Firestore client library automatically manages online and offline data access and synchronizes local data when the device is back online.
When you initialize Cloud Firestore, you can enable or disable offline persistence. So, when using the following line of code:
val firestoreSettings = FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(false)
.build()
You actually set the persistence to false, you are disabling this feature. To solve this, just remove this line of code. Firestore has .setPersistenceEnabled(true) by default.

This issue was fixed by the release 17.1.5 of Cloud Firestore.
See the official changelog https://firebase.google.com/support/release-notes/android
Cloud Firestore now recovers more quickly from bad network states.
Using my reproduction project with the version 18.0.0, the issue is indeed not present.

I'm facing the same issue since 2018 and have not yet found a proper solution. It is still happening with version 22.1.2 of the Firestore library for Android. As commented in the source code of DocumentReference, this exception is thrown when the client is offline and the requested document is not available in the local cache. There is also a recent explanation of this behaviour by Denver Coneybeare.
This exception might occur for DocumentReference.get() calls. Here's an exemplary stack trace:
com.google.firebase.firestore.FirebaseFirestoreException: Failed to get document because the client is offline.
at com.google.firebase.firestore.DocumentReference.lambda$getViaSnapshotListener$1(DocumentReference.java:331)
at com.google.firebase.firestore.DocumentReference$$Lambda$2.onEvent(DocumentReference.java:8)
at com.google.firebase.firestore.DocumentReference.lambda$addSnapshotListenerInternal$2(DocumentReference.java:504)
at com.google.firebase.firestore.DocumentReference$$Lambda$3.onEvent(DocumentReference.java:16)
at com.google.firebase.firestore.core.AsyncEventListener.lambda$onEvent$0(AsyncEventListener.java:42)
at com.google.firebase.firestore.core.AsyncEventListener$$Lambda$1.run(AsyncEventListener.java:2)
at com.google.firebase.firestore.util.Executors$$Lambda$1.execute(Executors.java)
at com.google.firebase.firestore.core.AsyncEventListener.onEvent(AsyncEventListener.java:39)
at com.google.firebase.firestore.core.QueryListener.raiseInitialEvent(QueryListener.java:176)
at com.google.firebase.firestore.core.QueryListener.onViewSnapshot(QueryListener.java:95)
at com.google.firebase.firestore.core.EventManager.addQueryListener(EventManager.java:97)
at com.google.firebase.firestore.core.FirestoreClient.lambda$listen$6(FirestoreClient.java:160)
at com.google.firebase.firestore.core.FirestoreClient$$Lambda$6.run(FirestoreClient.java:12)
at com.google.firebase.firestore.util.AsyncQueue.lambda$enqueue$2(AsyncQueue.java:436)
at com.google.firebase.firestore.util.AsyncQueue$$Lambda$2.call(AsyncQueue.java:1)
at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor.lambda$executeAndReportResult$1(AsyncQueue.java:322)
at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$$Lambda$2.run(AsyncQueue.java:1)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:462)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$DelayedStartFactory.run(AsyncQueue.java:229)
at java.lang.Thread.run(Thread.java:919)
I'm sorry if this answer does not solve the problem, but this information would have been too long for a comment on the original post.

I also have the same problem. There is workaround - application waits some time and makes another attempt to get the data from server.
Here is mockup (this is Flutter/dart):
List<Course> items;
Widget build(BuildContext context) {
...
loadData();
...
//building th UI with data from items list
...
}
loadData() async {
//is not loaded but there is internet connection
if (!isLoaded&&isConnected) {
try {
querySnapshot = await query.getDocuments(source: Source.server);
}
catch (ex) {
print(ex);
Future.delayed(Duration(seconds: 10), () {
// setState to trigger build 10 seconds later to make one more attempt
setState(() {
isLoaded = false;
});
});
return;
}
// here goes the code to handle the result
items = querySnapshot.documents.map((s) => Course.fromDb(s)).toList();
}

this may help
Java:
FirebaseFirestore db = FirebaseFirestore.getInstance();
//to reconnect
db.terminate();
db = FirebaseFirestore.getInstance();
Kotlin:
var db = Firebase.firestore
//to reconnect
db.terminate()
db = Firebase.firestore

Firestore offline cache

I'm building an Android application which has to work offline for weeks, but can sync immediately with a remote DB as it goes online.
My question is can Firestore be a good option for this? How long does Firestore keep its offline cache?

Firestore can be configured to persist data for such disconnected/offline usage. I recommend that you read the enable offline persistence section in the docs, which contains this sample of enabling this feature:
FirebaseFirestoreSettings settings = new FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(true)
.build();
db.setFirestoreSettings(settings);
This persistence is actually enabled by default on Android and iOS, so the call above is not needed.
Your Android code that interact with the database will be the same whether you're connected or not, since the SDK simply works the same. If you want to detect whether data is coming from the cache (and thus potentially stale), read the section Listen to offline data in the docs.
The data in the cache does not expire after a certain amount of time. The only two reasons data is removed from the cache:
The data has been removed from the server, in which case the client will remove it from the disk cache.
The client needs to purge its disk cache to make space for more recent data.

EDIT-25/5/2020: Francesco is correct, the docs link given in the comment does clarify that. It seems the cache size has been changed, by default it has been decreased to 40MB.
OLD: The following answer follows the official guide in the following link:
Handling Cache size
FirebaseFirestoreSettings settings = new FirebaseFirestoreSettings.Builder()
.setCacheSizeBytes(FirebaseFirestoreSettings.CACHE_SIZE_UNLIMITED)
.build();
db.setFirestoreSettings(settings);
The above code has a flag set for setCacheSize(), which will prevent your cache from
being cleared. You can also specify the same in size. If you do not set this by default the size is 100MB.
As per the guide, there is a method to check if the data you query came from cache or the firestore. Also the moment your device is back online the firestore refreshes the cache, and keeps the data synchronized.
To answer your question, as you have to work with offline data for weeks, i suggest every time the data is fetched to store it in json/xml formats, as storing huge amount of data in cache is not a really good approach when thought of in terms of performance.
I hope i helped you clear some things out.

If you listen to data in Cloud Firestore, you will get immediate snapshots of cached data and also updates when your app is able to connect online:
final DocumentReference docRef = db.collection("cities").document("SF");
docRef.addSnapshotListener(new EventListener<DocumentSnapshot>() {
#Override
public void onEvent(#Nullable DocumentSnapshot snapshot,
#Nullable FirebaseFirestoreException e) {
if (e != null) {
Log.w(TAG, "Listen failed.", e);
return;
}
// Determine if the data came from the server or from cache
String source = snapshot != null && snapshot.getMetadata().hasPendingWrites()
? "Local" : "Server";
// Read the data
if (snapshot != null && snapshot.exists()) {
Log.d(TAG, source + " data: " + snapshot.getData());
} else {
Log.d(TAG, source + " data: null");
}
}
});
Persistence is enabled by default so this behavior does not require any configuration.

Firebase RemoteConfig fetch calling each time not after cache expired

Maybe I am missing something, but shouldn't the fetch() be called only if the cached values are older than the cached value?
Having this code called from activity's onCreate
firebaseRC = FirebaseRemoteConfig.getInstance()
firebaseRC.fetch(3600L). .addOnCompleteListener(this, new OnCompleteListener<Void>() {
#Override
public void onComplete(#NonNull Task<Void> task) {
if (task.isSuccessful()) {
//THE TASK IS ALWAYS SUCCESSFUL
firebaseRC.activateFetched();
}
}
});
If I start the app on my emulator a few times in a row, the fetch completes successfully each time.
Shouldn't the fetch actually complete with success only if my data is older than 3600 seconds? So starting the app a second time, fetch should complete without onComplete
The docs say:
Remote Config caches values locally after the first successful fetch.
By default the cache expires after 12 hours, but you can change the
cache expiration for a specific fetch by passing the desired cache
expiration to the fetch method. If the values in the cache are older
than the desired cache expiration, Remote Config will request fresh
config values from the service. If your app requests fresh values
using fetch several times, requests are throttled and your app is
provided with cached values.
It doesn't say how onComplete will trigger... should I use activateFetched() each time as it has the same value?

The API documentation for fetch() says that the results will come from either cache or server. It doesn't say that a cached result it considered a failure. The fetch will only fail if it can't give you any results at all, which means your app is probably offline and has no previously successful fetched values in cache.