I have been using Firebase Database in my Android app for almost a year now and it works pretty nice. Unfortunately the data stops being synced to the could after some time. It is just never synced/stored to the cloud. Only local. So when user reinstalls the app, it only contains the data which was stored in the cloud. So to the user it looks like the data was removed, but actually is was never stored. I checked and the data is not visible in the firebase-console. Because it happens after a reinstall I guess it has something to do with the syncing. Users report losing data of about 2-3 months.
I'm using the following singleton helper class. Note I use the setPersistenceEnabled(true) and keepSynced(true).
public class FirebaseHelper{
protected FirebaseHelper(Context c) {
this.c = c.getApplicationContext();
FirebaseDatabase.getInstance().setPersistenceEnabled(true);
mAuth = FirebaseAuth.getInstance();
this.userRef = FirebaseDatabase.getInstance().getReference().child(((BuildConfig.DEBUG ? "debug" : "release"))).child("users").child(getUID());
this.userRef.keepSynced(true);
this.path1 = userRef.child("path1");
this.path2 = userRef.child("path2");
this.path3 = userRef.child("path3");
this.path4 = userRef.child("path4");
}
public static FirebaseHelper getInstance(Context c) {
if (instance == null) {
instance = new FirebaseHelper(c);
}
return instance;
}
public String insertObject(MyObject obj) {
DatabaseReference newItem = this.path1.push();
String pushID = newItem.getKey();
obj.id = pushID;
newItem.setValue(obj.getObject());
return pushID;
}
public void updateData(...){}
...other methods
}
What could possibly be the cause of this?
There are only three reasons for this to happen to the best of my knowledge.
1) The method getUID()
Somehow the getUID() method is returning a null or invalid value which leads the data to be stored to somewhere else or it is not getting stored at all.
You are using simply getUid() instead of FirebaseAuth.getInstance().getCurrentUser().getUid(). So it must be a user defined method.
I think your getUid() does something like this.
String getUid() {
try{
return FirebaseAuth.getInstance().getCurrentUser().getUid() ;
} catch (NullPointerException e) {
return null;
}
}
FirebaseAuth.getInstance().getCurrentUser() will return null if cache is cleared. It will lead your data to be lost.
2) Redundant data
Since Marshmallow, data is backed up to the cloud including shared preference. If you are checking shared preference to decide if user is logged in, user will be gone taken inside after reinstalling the app, skipping the login page. But actually he is not logged in which means FirebaseAuth.getInstance().getCurrentUser() returns null and any attempt to access the database will fail (depends on your database rules).
Solution: use FirebaseAuth.getInstance().getCurrentUser()==null to check if user is enabled.
You can alternatively set backup to false. But I prefer first method.
3) An internal bug in Firebase SDK
Unfortunately there is nothing much we can do with it. Try narrowing it down and find a scenario by which the issue can be reproduced and report it to Firebase.
By the way, child(((BuildConfig.DEBUG ? "debug" : "release"))) is really smart. I am going to adopt it.
Well, if your database has been syncing and you have not made any changes to the code whatsoever, it means that this is a firebase error, particularly related to the mobile phone the user is using.
Most developers who use firebase find problems querying the database when certain carriers are used. I have researched into this issue but i have not yet resolved it yet. If you happen to be using mobile data, actions like authenticating a user may not work.
Solution
Use a different internet source to test your code. Try using wifi instead of mobile data while debugging or testing your app.
if I find any helpful work around, I will file it on a firebase project experiencing the problem on open source Lucem
Throwing this out as a guess as well:
At some point you distributed an app where BuildConfig.DEBUG = true, so users that install an updated version "lose" their data. Doesn't explain why other users haven't reported shorter losses though...
The solution would be a data migration, checking which has newer data and then copying the data if DEBUG is newer.
Related
After deleting data from my Firestore Database, it takes my Android app some time to realize that the data was deleted, and I assume that it's happening due the auto data cache. My app has nothing to do with offline usage and I'd like to disable this feature...
I have added this in my custom Application Class:
import android.app.Application;
import com.google.firebase.firestore.FirebaseFirestore;
import com.google.firebase.firestore.FirebaseFirestoreSettings;
public class ApplicationClass extends Application {
#Override
public void onCreate() {
super.onCreate();
FirebaseFirestore db=FirebaseFirestore.getInstance();
FirebaseFirestoreSettings settings = new FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(false)
.build();
db.setFirestoreSettings(settings);
}
}
The problem occurs after turning off the internet connection and than turning it back on (while the app is still running, in the background or not)- the Firestore module seems to lose connection to the server, and it makes the opposite operation than the intended one - instead of stop taking data from the cache, it takes data from the cache only.
For example, debugging this code will always show that isFromCache is true and documentSnapshot is empty (even though that on the server side - it's not empty):
usersRef.document(loggedEmail).collection("challenges_received").get().addOnSuccessListener(new OnSuccessListener<QuerySnapshot>() {
#Override
public void onSuccess(QuerySnapshot documentSnapshots) {
boolean isFromCache=documentSnapshots.getMetadata().isFromCache();
if (!documentSnapshots.isEmpty()) {
}
}
});
Is this normal behavior?
Is there another way to disable the data cache in Cloud Firestore?
EDIT:
Adding: FirebaseFirestore.setLoggingEnabled(flase); (instead of the code above) in the custom Application Class gives the same result.
According to Cloud Firestore 16.0.0 SDK update, there is now a solution to this problem:
You are now able to choose if you would like to fetch your data from the server only, or from the cache only, like this (an example for server only):
DocumentReference documentReference= FirebaseFirestore.getInstance().document("example");
documentReference.get(Source.SERVER).addOnSuccessListener(new OnSuccessListener<DocumentSnapshot>() {
#Override
public void onSuccess(DocumentSnapshot documentSnapshot) {
//...
}
});
For cache only, just change the code above to Source.CACHE.
By default, both methods still attempt server and fall back to the cache.
I just ran a few tests in an Android application to see how this works. Because Firestore is currently still in beta release and the product might suffer changes any time, i cannot guarantee that this behaviour will still hold in the future.
db.collection("tests").document("fOpCiqmUjAzjnZimjd5c").get().addOnCompleteListener(new OnCompleteListener<DocumentSnapshot>() {
#Override
public void onComplete(#NonNull Task<DocumentSnapshot> task) {
DocumentSnapshot documentSnapshot = task.getResult();
System.out.println("isFromCache: " + documentSnapshot.getMetadata().isFromCache());
}
});
Regarding the code, is the same no matter if we're getting the data from the cache or you are connected to the servers.
When I'm online it prints:
isFromCache: false
When I'm offline, it prints:
isFromCache: true
So, for the moment, there is no way to stop the retrieval of the data from the cache while you are not connected to the server, as you cannot force the retrieval of the data from the cache while you're connected to the server.
If instead I use a listener:
db.collection("tests").document("fOpCiqmUjAzjnZimjd5c").addSnapshotListener(new DocumentListenOptions().includeMetadataChanges(), new EventListener<DocumentSnapshot>() {
#Override
public void onEvent(DocumentSnapshot documentSnapshot, FirebaseFirestoreException e) {
System.out.println("listener.isFromCache: " + documentSnapshot.getMetadata().isFromCache());
}
});
I get two prints when I'm online:
listener.isFromCache: true
listener.isFromCache: false
Firestore is desinged to retrieve data from the chache when the device is permanently offline or while your application temporarily loses its network connection and for the moment you cannot change this behaviour.
As a concusion, an API that does something like this, currently doesn't exist yet.
Edit: Unlike in Firebase, where to enable the offline persistence you need use this line of code:
FirebaseDatabase.getInstance().setPersistenceEnabled(true);
In Firestore, for Android and iOS, offline persistence is enabled by default.
Using the above line of code, means that you tell Firebase to create a local (internal) copy of your database so that your app can work even if it temporarily loses its network connection.
In Firestore we find the opposite, to disable persistence, we need to set the PersistenceEnabled option to false. This means that you tell Firestore not to create a local copy of your database on user device, which in term means that you'll not be able to query your database unless your are connected to Firebase servers. So without having a local copy of your database and if beeing disconected, an Exception will be thrown. That's why is a good practice to use the OnFailureListener.
Update (2018-06-13): As also #TalBarda mentioned in his answer this is now possible starting with the 16.0.0 SDK version update. So we can achieve this with the help of the DocumentReference.get(Source source) and Query.get(Source source) methods.
By default, get() attempts to provide up-to-date data when possible by waiting for data from the server, but it may return cached data or fail if you are offline and the server cannot be reached. This behavior can be altered via the Source parameter.
So we can now pass as an argument to the DocumentReference or to the Query the source so we can force the retrieval of data from the server only, chache only or attempt server and fall back to the cache.
So something like this is now possible:
FirebaseFirestore db = FirebaseFirestore.getInstance();
DocumentReference docIdRef = db.collection("tests").document("fOpCiqmUjAzjnZimjd5c");
docIdRef.get(Source.SERVER).addOnSuccessListener(new OnSuccessListener<DocumentSnapshot>() {
#Override
public void onSuccess(DocumentSnapshot documentSnapshot) {
//Get data from the documentSnapshot object
}
});
In this case, we force the data to be retrieved from the server only. If you want to force the data to be retrieved from the cache only, you should pass as an argument to the get() method, Source.CACHE. More informations here.
FirebaseFirestoreSettings settings = new FirebaseFirestoreSettings.Builder()
.setPersistenceEnabled(false)
.build();
dbEventHome.setFirestoreSettings(settings);
By setting this it is fetching from server always.
In Kotlin:
val db:FirebaseFirestore = Firebase.firestore
val settings = firestoreSettings {
isPersistenceEnabled = false
}
db.firestoreSettings = settings
// Enable Firestore logging
FirebaseFirestore.setLoggingEnabled(flase);
// Firestore
mFirestore = FirebaseFirestore.getInstance();
In general: the Firebase client tries to minimize the number of times it downloads data. But it also tries to minimize the amount of memory/disk space it uses.
The exact behavior depends on many things, such as whether the another listener has remained active on that location and whether you're using disk persistence. If you have two listeners for the same (or overlapping) data, updates will only be downloaded once. But if you remove the last listener for a location, the data for that location is removed from the (memory and/or disk) cache.
Without seeing a complete piece of code, it's hard to tell what will happen in your case.
Alternatively: you can check for yourself by enabling Firebase's logging [Firebase setLoggingEnabled:YES];
try this For FireBase DataBase
mDatabase.getReference().keepSynced(false);
FirebaseDatabase.getInstance().setPersistenceEnabled(false);
In Kotlin;
val settings = FirebaseFirestoreSettings.Builder()
with(settings){
isPersistenceEnabled = false
}
Firebase.firestore.firestoreSettings = settings.build()
What is a proper, solid way of having getToken() return with a token id?
I have seen a couple of attempts using
while-loop, calling getToken() until it's not null
Local BroadcastReceiver
Timer
The while loop seem really risky in terms of ANRs and having a loop running indefinitely.
The local BroadcastReceiver isn't readily applicable to my application, because even though there is a main activity in my app, there is nothing forcing the user to interact with it. Users can accomplish tasks in the app without going through that main activity.
The timer seem fragile. How long do you have to wait for? Seconds, minutes, hours?
The Firebase quickstart sample code does not provide an example to handle the situation when getToken() is null, and I don't see what a proper and solid implementation would be to ensure there's a token being returned, without any of the nasty side effects of the above mentioned implementations would bring.
Having a callback method to hook onto would've made this a non-issue, but since that isn't available, I just have to run this through StackOverflow to get an idea of how people resolve this issue.
Note: I have implemented onTokenRefresh() and that works for the specific situations stated in the docs, but I can't use that alone, because the app is only upgraded on a lot of devices, and this method isn't called during an app upgrade. I have the null issues on (Nexus) hardware devices, I am not using the emulator at all and don't intend to use it.
As it seems there is noone adding other approaches, than the ones I mentioned before, I'm now going to answer my own question.
The "solution" I have implemented to having a token being generated so that you can handle it more consistently, and store it for later reference, is this;
The following class follows the quickstart almost to the letter. But instead of sending the token away to a server, at the exact time it is accessible, I instead store it for later use:
public class FirebaseIdService extends FirebaseInstanceIdService {
#Override
public void onTokenRefresh() {
// Get updated InstanceID token.
String refreshedToken = FirebaseInstanceId.getInstance().getToken();
if(refreshedToken != null) {
Log.d("TOKENTAG", "Refreshed token: " + refreshedToken);
SharedPreferences prefs = getApplicationContext().getSharedPreferences("pref_id", 0);
prefs.edit().putString("share_pref_token", refreshedToken).apply();
}
}
}
When I want/need to send the token I check if there's already one available in the SharedPreferences and if not, I simply delete the Firebase instance, and call getToken(), to have another one generated. What this essentially does is to tell Firebase that "I want to invalidate the current instance", and then, by calling getToken() afterwards, have it generate a new one, essentially making onTokenRefresh() to be called, whenever a token has been generated and is available:
SharedPreferences prefs = getApplicationContext().getSharedPreferences("pref_id", 0);
if(prefs.getString("share_pref_token", "").isEmpty()) {
Log.d("TOKENTAG", "token not available, so let's force one to be generated");
try {
FirebaseInstanceId.getInstance().deleteInstanceId();
} catch (IOException e) {
e.printStackTrace();
}
FirebaseInstanceId.getInstance().getToken();
}
I am pretty sure this is a sub optimal solution, and I'm welcoming suggestions to make this suggestion better, but at this point this is basically the only way that seem to cover the bases of combining onTokenRefresh() and getToken() to more consistently get a token without any of the afformentioned drawbacks.
I'm building an android app using the Android Parse SDK, which gets all data from Parse at initialisation and stores it locally. Later, it will only update those entities (ParseObjects) which need so. I'm not getting any return from some Pin() operations, and similarly no callback when I use PinInBackground() and variants. Same happens with Unpin().
My code is something like the following. I have a list of ControlDate, a ParseObject which contains updated_at and updated_locally_at for each Parse data table. I use it to decide if I should query a given table (reducing number of queries). I iterate over this list when I perform a data update, in an IntentService, like this:
protected void onHandleIntent(Intent intent) {
for(ControlDate d : mUpdateQueue) { // mUpdateQueue is the list of ControlDate
if(d.entityNeedsUpdate()) { // returns true if updated_at > updated_locally_at
updateEntity(d);
}
}
private boolean updateEntity(ControlDate d) {
String tableName = d.getTableName();
Date lastLocalUpdate = d.getLastLocalUpdate();
ParseQuery<ParseObject> qParse = ParseQuery.getQuery(tableName);
qParse.whereGreaterThan("updated_at", lastLocalUpdate);
try {
// update the entities
List<ParseObject> entities = qParse.find();
ParseObject.pinAll(entities); // SOMETIMES GETS STUCK (no return)
// update the associated ControlDate
d.setLastLocalUpdate(new Date()); // updated_locally_at = now
d.pin(); // SOMETIMES GETS STUCK (no return)
}
catch(ParseException e) {
// error
}
}
}
Those operations SOMETIMES do not return. I'm trying to find a pattern but still no luck, apparently it started happening when I added pointer arrays to some of the entities. Thus, I think it may be due to the recursive nature of pin(). However it is strange that it sometimes also gets stuck with ParseObjects which do not reference any others - as it is the case with d.pin().
Things I've tried:
changing the for loop to a ListIterator (as I am changing the list of ControlDates, but I don't think this is necessary);
using the threaded variants (eg.: PinInBackground()) - no callback;
pinning each entity individually (in a loop, doing pin()) - a lot slower, still got stuck;
debugging - the thread just blocks here: http://i.imgur.com/oBDjpCw.png?1
I'm going crazy with this, help would be much appreciated!
PS.: I found this https://github.com/BoltsFramework/Bolts-Android/issues/48
Its an open issue on the bolts library, which is used in the Android SDK and may be causing this (maybe?). Anyway I cannot see how I could overcome my problem even though the cause for the pin() not returning could be an "unobserved exception" leading to a deadlock.
I'm getting the following error when using the encrypted SQLCipher database in my Android app, but only off and on:
net.sqlcipher.database.SQLiteException: not an error
at net.sqlcipher.database.SQLiteDatabase.dbopen(Native Method)
at net.sqlcipher.database.SQLiteDatabase.<init>(SQLiteDatabase.java:1950)
at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:900)
at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:947)
at net.sqlcipher.database.SQLiteOpenHelper.getReadableDatabase(SQLiteOpenHelper.java:195)
at com.android.storage.DatabaseHelper.getReadable(DatabaseHelper.java:99)
...
I've got the proper files in the assets/ and libs/ folders because the database works fine most of the time. However, every once in awhile I'll see this error. I've seen this twice now on my phone and it's always been after resuming the app after hours of inactivity (I check for user's oauth token in db if it gets cleared from memory).
I call "SQLiteDatabase.loadLibs(this)" only from the Application::onCreate() method so my hunch is that this isn't getting called on a resume and is throwing the error. Does this sound possible? If so, where should I call loadLibs? A user could enter the app in any activity and I access the db if the token isn't in memory. I see my options as either calling loadLibs on each Activity::onCreate or calling it each time I attempt to open the db. Would it cause any harm or performance issues if I called it multiple times like this?
You might consider moving the SQLiteDatabase.loadLibs(this); to your application subclass of net.sqlcipher.database.SQLiteOpenHelper. You can then pass the static instance of your Application subclass as its argument. Something like the following might be an example:
public class SchemaManager extends net.sqlcipher.database.SQLiteOpenHelper {
private static SchemaManager instance;
public static synchronized SchemaManager getInstance() {
if(instance == null) {
SQLiteDatabase.loadLibs(YourApplication.getInstance());
instance = new SchemaManager(…)
}
return instance;
}
}
With regard to the exception that was provided, the Java routine calls into a JNI layer that calls sqlite3_open_v2, setting the soft heap limit and setting the busy timeout. I would suggest adding logging locally to verify you are passing a valid path and a non null passphrase when attempting to acquire the SQLiteDatabase instance when you get a crash. Calling SQLiteDatabase.loadLibs(this); multiple times shouldn't cause a noticeable performance impact, much of what occurs are calls to System.loadLibrary(…) which get mapped into Runtime.getRuntime().loadLibrary(…), once a dynamic library has been loaded, subsequent calls are ignored.
We have a strange bug. Our App will loose it's data (stored in SQLite) if the battery runs out but it doesn't if we kill the application forcefully and then the battery runs out.
We're not sure what could cause this.
EDIT 1:
WHAT DOES IT MEAN TO LOOSE THE DATA?
When the user registers we save his username etc.
When the app is closed, forced-closed and the user opens the app again the username is there
When the battery runs out and the app is running in foreground. After the battery is recharged and the user opens the app again, the username is gone.
How are objects stored:
public Void perform(SQLiteDatabase db) {
final ConfigEntry entry = new ConfigEntry(property, value);
if(contains(property)) {
db.update(ConfigEntry.TABLE_NAME, databaseAdapter.convertToContentValues(entry), ConfigEntry.COLUMN_NAME + " = ?", new String[] { property });
} else {
db.insertOrThrow(ConfigEntry.TABLE_NAME, null, databaseAdapter.convertToContentValues(entry));
}
return null;
}
});
So basically db.insertOrThrow is called.
This is how we initialize the database:
public S doInTransaction(TransactionTask task) {
DatabaseHelper mDbHelper = DatabaseHelper.getInstance(mContext);
SQLiteDatabase mDb = mDbHelper.getWritableDatabase();
mDb.setLockingEnabled(true);
mDb.beginTransaction();
try {
S result = task.perform(mDb);
mDb.setTransactionSuccessful();
return result;
} finally {
mDb.endTransaction();
}
}
I am not sure but I hope I can provide some questions or inspiration to dig deeper:
Have you monitored if your database goes through the complete initial creation process or are the tables there but only the username missing? Can you download the database file from the device and check the content in an SQLite tool on your PC?
You open the database inside doInTransaction() but you don't close it there:
public S doInTransaction(TransactionTask task) {
DatabaseHelper mDbHelper = DatabaseHelper.getInstance(mContext);
SQLiteDatabase mDb = mDbHelper.getWritableDatabase();
//mDb.setLockingEnabled(true); // true is default, so it can be removed
mDb.beginTransaction();
S result = null;
try {
result = task.perform(mDb);
mDb.setTransactionSuccessful();
} finally {
// rolls back if setTransactionSuccessful() wasn't called
mDb.endTransaction();
mDb.close();
}
return result;
}
Are you completely sure that you use the same database? I ask because I saw it once that someone used a timestamp as part of the database name.
You are not creating an in-memory database by mistake?
There are multiple different ways for a device to die of low battery: It might just drop dead or it manages to actually make an emergency shut down. Which one is the case? Have you any logging that might provide some inside of what your app does when it goes off?
To sum up: I doubt that the reason is in the code you provided and as you mentioned, the code is only executed when a button is pressed. That leaves the above questions for further investigation (there are, of course, some I might have missed...).
The problem is you are using beginTransaction(). If you have call beginTransaction() you need to call:
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
below or your data Will NOT be saved. The reason is here in the sqlite manual
And the reason why your data isn't saving is, if you started transaction and db.setTransactionSuccessful();wasn't called because of the system tried to close it,then data will not be saved. However finally{} might still be called but that doesn't change the fact that db.setTransactionSuccessful(); wasn't called.
Thanks a lot for all the Help, we finally found the answer.
We are encrypting our database using a user password and salting the password with the MAC adddress of the device.
What happened is that sometimes when the user ran out of battery after the boot, when our app was ran and we requested for the MAC address it failed because the network card it's still disabled or something. In that case we created a new salt which when trying to read the DB would fail.