We have a strange bug. Our App will loose it's data (stored in SQLite) if the battery runs out but it doesn't if we kill the application forcefully and then the battery runs out.
We're not sure what could cause this.
EDIT 1:
WHAT DOES IT MEAN TO LOOSE THE DATA?
When the user registers we save his username etc.
When the app is closed, forced-closed and the user opens the app again the username is there
When the battery runs out and the app is running in foreground. After the battery is recharged and the user opens the app again, the username is gone.
How are objects stored:
public Void perform(SQLiteDatabase db) {
final ConfigEntry entry = new ConfigEntry(property, value);
if(contains(property)) {
db.update(ConfigEntry.TABLE_NAME, databaseAdapter.convertToContentValues(entry), ConfigEntry.COLUMN_NAME + " = ?", new String[] { property });
} else {
db.insertOrThrow(ConfigEntry.TABLE_NAME, null, databaseAdapter.convertToContentValues(entry));
}
return null;
}
});
So basically db.insertOrThrow is called.
This is how we initialize the database:
public S doInTransaction(TransactionTask task) {
DatabaseHelper mDbHelper = DatabaseHelper.getInstance(mContext);
SQLiteDatabase mDb = mDbHelper.getWritableDatabase();
mDb.setLockingEnabled(true);
mDb.beginTransaction();
try {
S result = task.perform(mDb);
mDb.setTransactionSuccessful();
return result;
} finally {
mDb.endTransaction();
}
}
I am not sure but I hope I can provide some questions or inspiration to dig deeper:
Have you monitored if your database goes through the complete initial creation process or are the tables there but only the username missing? Can you download the database file from the device and check the content in an SQLite tool on your PC?
You open the database inside doInTransaction() but you don't close it there:
public S doInTransaction(TransactionTask task) {
DatabaseHelper mDbHelper = DatabaseHelper.getInstance(mContext);
SQLiteDatabase mDb = mDbHelper.getWritableDatabase();
//mDb.setLockingEnabled(true); // true is default, so it can be removed
mDb.beginTransaction();
S result = null;
try {
result = task.perform(mDb);
mDb.setTransactionSuccessful();
} finally {
// rolls back if setTransactionSuccessful() wasn't called
mDb.endTransaction();
mDb.close();
}
return result;
}
Are you completely sure that you use the same database? I ask because I saw it once that someone used a timestamp as part of the database name.
You are not creating an in-memory database by mistake?
There are multiple different ways for a device to die of low battery: It might just drop dead or it manages to actually make an emergency shut down. Which one is the case? Have you any logging that might provide some inside of what your app does when it goes off?
To sum up: I doubt that the reason is in the code you provided and as you mentioned, the code is only executed when a button is pressed. That leaves the above questions for further investigation (there are, of course, some I might have missed...).
The problem is you are using beginTransaction(). If you have call beginTransaction() you need to call:
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
below or your data Will NOT be saved. The reason is here in the sqlite manual
And the reason why your data isn't saving is, if you started transaction and db.setTransactionSuccessful();wasn't called because of the system tried to close it,then data will not be saved. However finally{} might still be called but that doesn't change the fact that db.setTransactionSuccessful(); wasn't called.
Thanks a lot for all the Help, we finally found the answer.
We are encrypting our database using a user password and salting the password with the MAC adddress of the device.
What happened is that sometimes when the user ran out of battery after the boot, when our app was ran and we requested for the MAC address it failed because the network card it's still disabled or something. In that case we created a new salt which when trying to read the DB would fail.
Related
I have been using Firebase Database in my Android app for almost a year now and it works pretty nice. Unfortunately the data stops being synced to the could after some time. It is just never synced/stored to the cloud. Only local. So when user reinstalls the app, it only contains the data which was stored in the cloud. So to the user it looks like the data was removed, but actually is was never stored. I checked and the data is not visible in the firebase-console. Because it happens after a reinstall I guess it has something to do with the syncing. Users report losing data of about 2-3 months.
I'm using the following singleton helper class. Note I use the setPersistenceEnabled(true) and keepSynced(true).
public class FirebaseHelper{
protected FirebaseHelper(Context c) {
this.c = c.getApplicationContext();
FirebaseDatabase.getInstance().setPersistenceEnabled(true);
mAuth = FirebaseAuth.getInstance();
this.userRef = FirebaseDatabase.getInstance().getReference().child(((BuildConfig.DEBUG ? "debug" : "release"))).child("users").child(getUID());
this.userRef.keepSynced(true);
this.path1 = userRef.child("path1");
this.path2 = userRef.child("path2");
this.path3 = userRef.child("path3");
this.path4 = userRef.child("path4");
}
public static FirebaseHelper getInstance(Context c) {
if (instance == null) {
instance = new FirebaseHelper(c);
}
return instance;
}
public String insertObject(MyObject obj) {
DatabaseReference newItem = this.path1.push();
String pushID = newItem.getKey();
obj.id = pushID;
newItem.setValue(obj.getObject());
return pushID;
}
public void updateData(...){}
...other methods
}
What could possibly be the cause of this?
There are only three reasons for this to happen to the best of my knowledge.
1) The method getUID()
Somehow the getUID() method is returning a null or invalid value which leads the data to be stored to somewhere else or it is not getting stored at all.
You are using simply getUid() instead of FirebaseAuth.getInstance().getCurrentUser().getUid(). So it must be a user defined method.
I think your getUid() does something like this.
String getUid() {
try{
return FirebaseAuth.getInstance().getCurrentUser().getUid() ;
} catch (NullPointerException e) {
return null;
}
}
FirebaseAuth.getInstance().getCurrentUser() will return null if cache is cleared. It will lead your data to be lost.
2) Redundant data
Since Marshmallow, data is backed up to the cloud including shared preference. If you are checking shared preference to decide if user is logged in, user will be gone taken inside after reinstalling the app, skipping the login page. But actually he is not logged in which means FirebaseAuth.getInstance().getCurrentUser() returns null and any attempt to access the database will fail (depends on your database rules).
Solution: use FirebaseAuth.getInstance().getCurrentUser()==null to check if user is enabled.
You can alternatively set backup to false. But I prefer first method.
3) An internal bug in Firebase SDK
Unfortunately there is nothing much we can do with it. Try narrowing it down and find a scenario by which the issue can be reproduced and report it to Firebase.
By the way, child(((BuildConfig.DEBUG ? "debug" : "release"))) is really smart. I am going to adopt it.
Well, if your database has been syncing and you have not made any changes to the code whatsoever, it means that this is a firebase error, particularly related to the mobile phone the user is using.
Most developers who use firebase find problems querying the database when certain carriers are used. I have researched into this issue but i have not yet resolved it yet. If you happen to be using mobile data, actions like authenticating a user may not work.
Solution
Use a different internet source to test your code. Try using wifi instead of mobile data while debugging or testing your app.
if I find any helpful work around, I will file it on a firebase project experiencing the problem on open source Lucem
Throwing this out as a guess as well:
At some point you distributed an app where BuildConfig.DEBUG = true, so users that install an updated version "lose" their data. Doesn't explain why other users haven't reported shorter losses though...
The solution would be a data migration, checking which has newer data and then copying the data if DEBUG is newer.
I'm having SQLite trouble in a multithreaded application. I have an Activity which uses a subclass of AsyncTaskLoader to perform some data import from a file (specified by an Uri as it comes from Android Storage Access Framework), and when the loader is started and does its work (it writes to the database) and the device is rotated, I get a 'android.database.sqlite.SQLiteDatabaseLockedException: database is locked (code 5)' exception. I know what the problem is (multiple SQLiteOpenHelpers accessing a database), but I am not sure how I'm supposed to fix this. Here is my code:
Activity:
private void importTests(Uri uri) {
Bundle loaderArgs = new Bundle();
loaderArgs.putParcelable(URI_IMPORTER_LOADER_ARG, uri);
getLoaderManager().initLoader(0, loaderArgs, this).forceLoad();
}
#Override
public Loader<Exception> onCreateLoader(int id, Bundle args) {
Uri uri = args.getParcelable(URI_IMPORTER_LOADER_ARG);
return new ImporterLoader(this, uri, dbHelper);
}
#Override
public void onLoadFinished(Loader<Exception> loader, Exception exception) {
getLoaderManager().destroyLoader(loader.getId());
if (exception != null) {
// import failed, show toast
} else {
// init Ui
}
}
#Override
public void onLoaderReset(Loader<Exception> loader) {
// nothing
}
(Note: I don't create the Loader in onCreate; rather, I do it on demand when the import functionality is invoked, and destroy it as soon as it is ready. I'm not sure if this is a correct way of using loaders.)
Now more detailed information about the problem:
when the activity is started, it creates a DbHelper (which is a SQLiteOpenHelper subclass), stores it in a member field, and reads the database (using getReadableDatabase(), but most likely it will be writable anyways) to initialize the Ui (show a list of items etc.)
if an import is triggered, a Loader is created and it gets the DbHelper from the activity; this import reads a Uri using its InputStreams, does a bit of parsing and writes rows to the database
when, during import, the device is rotated, the following happens: the loader is still going on (I don't want to destroy it, I want it to finish its task and trigger updating the Ui when it's done, possibly in the new, rotated, activity - that's the very reason I use a Loader), and it still uses its instance of DbHelper and its open connection; however, the activity is destroyed and then created again, creating another instance of DbHelper, which again tries to read the database to initialize the Ui
as a result, there are 2 DbHelpers with one open connection each, and the second one from the new activity instance throws the SQLiteDatabaseLockedException from getReadableDatabase()
Until now, I've been using a new DbHelper in every activity in the app as it wasn't possible to use the db from multiple threads, but now that I implemented the first background use case, all hell breaks loose, of course. So actually, it seems to be possible to have multiple connections opened at the same time, as long as they are not accessed simultaneously, as I had many activities stack on top of each other, each having its own helper, and (wrongly?) didn't close them in onPause and open in onResume.
So, the question is if I'm doing anything fundamentally wrong here? Based on my research, there seems to be only 2 solutions: create a ContentProvider which will manage the database (I would like not to be forced to do this as I really don't need no CP), or somehow keep only one DbHelper/connection for the whole app. How I do it is unimportant, but as of now my favorite would probably be a singleton (yuck) initialized in a custom Application subclass. In the future I would like to try Dagger so I would probably make it a #Singleton, but not yet.
Edit: unfortunately, my ui still blocks - the importer transaction blocks any db reads from the ui initialization methods. I guess I need to figure out a way to start reader transactions which are not blocked by the single write tx.
Edit 2: I was able to successfully unblock readers while a writer is working. To do this:
I call
setWriteAheadLoggingEnabled(true);
in the DbHelper constructor
use
db.beginTransactionNonExclusive();
instead of
db.beginTransaction();
in the ImporterLoader (the thing which performs the writes)
I'm using GreenDao to store a lot of data, coming from a REST service.
A lot of my entities are connected with relations.
Everything works great, but tomorrow I have to implement a rocksolid workflow.
When I load my data I have to check if an error occurs.
If so, I have to make sure nothing is stored in the SQLite DB.
Normally I would work with transactions and rollback in case of an exception,
otherwise commit to the db.
For now I just use insertordelete to save an entity, everytime I created an object.
What would be the way to implement this?
On inserts and updates Greendao checks if there is a ongoing transaction. If that is the case greendao will not start a new transaction.
So the only thing to do is to start a transaction on your database and commit/rollback after your work is done or you notice an error. All inserts and updates will be in the same transaction which has benefits concerning data consistency and also on performance, since greendao will start new transactions with commit/rollback for every insert and update operation.
Summarized you can use code like this:
SQLiteDatabase db = dao.getDatabase();
db.beginTransaction();
try {
// do all your inserts and so on here.
db.setTransactionSuccessful();
} catch (Exception ex) {
} finally {
db.endTransaction();
}
I also tweaked my greendao a bit so that it doesn't cache inserted objects to get further performance and memoryusage benefits (since I insert a lot of data once and I only use very few data during runtime depending on user input). See this post.
I'm getting an NullPointerException when I insert values into to my SQLite table on Android and I don't understand why. I'm testing ContentValues and the database instance for null.
This is the insertion code:
public void insertOrIgnore(ContentValues values) {
SQLiteDatabase db = this.dbHelper.getWritableDatabase();
try {
//I added these null value checks to stop NPE, but doesn't help.
if (values != null && db != null) {
db.insertWithOnConflict(TABLE, null, values, SQLiteDatabase.CONFLICT_IGNORE);
}
} catch (SQLiteException e) {
} finally {
if (db != null) {
db.close();
}
}
}
where
public static final String TABLE = "albums";
Most of the time this code works with the data added to the database as expected. However, it sometimes and rarely generates the below error. The stack trace is from ACRA and I have not been able to isolate under what conditions this error occurs. I'm looking for pointers as to why this happens and what the conditions are. My knowledge of SQLite is beginner level.
java.lang.NullPointerException
at android.database.sqlite.SQLiteStatement.releaseAndUnlock(SQLiteStatement.java:290)
at android.database.sqlite.SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:96)
at android.database.sqlite.SQLiteDatabase.executeSql(SQLiteDatabase.java:2025)
at android.database.sqlite.SQLiteDatabase.execSQL(SQLiteDatabase.java:1965)
at android.database.sqlite.SQLiteDatabase.beginTransaction(SQLiteDatabase.java:690)
at android.database.sqlite.SQLiteDatabase.beginTransactionNonExclusive(SQLiteDatabase.java:605)
at android.database.sqlite.SQLiteStatement.acquireAndLock(SQLiteStatement.java:247)
at android.database.sqlite.SQLiteStatement.executeInsert(SQLiteStatement.java:112)
at android.database.sqlite.SQLiteDatabase.insertWithOnConflict(SQLiteDatabase.java:1844)
at com.mydomain.myapp.albums.AlbumsData.insertOrIgnore(AlbumsData.java:89)
Line 89 is the db.insertWithOnConflict(...) call shown above.
I'm not looking for an answer with complete code necessarily but rather a pointer and explanation as to what's going wrong so I can begin to fix it myself.
EDIT:
The stack trace shows the NPE originates from line 290 of SQLiteStatement (v 4.03):
setNativeHandle(mDatabase.mNativeHandle);
So it seems the database instance is null. How can it become null during a transaction when I tested for null at the beginning of the transaction?
As mentioned here SQLiteDatabase close() function causing NullPointerException when multiple threads
The reason for your bug could be that you close the database at some point. Probably concurrently while the task that fails was not finished.
I've followed the stacktrace a bit and this is what roughly happens:
AlbumsData.insertOrIgnore(AlbumsData.java:89)
You call insertWithOnConflict, which builds the resulting sql string ("INSERT OR IGNORE INTO...") then wraps that together with the values from your ContentValues into a SQLiteStatement.
SQLiteDatabase.insertWithOnConflict(SQLiteDatabase.java:1844) - The resulting statement is to be executed now
SQLiteStatement.executeInsert(SQLiteStatement.java:112) - before the actual insert can happen, the database needs to acquire a lock.
SQLiteStatement.acquireAndLock(SQLiteStatement.java:247) - some checks happen here, the database object is as far as I can see not null at that point. Code decides that it has to start a transaction. The database object itself is as far as I can see not locked at that point.
SQLiteDatabase.beginTransactionNonExclusive(SQLiteDatabase.java:605) - just forwarding
SQLiteDatabase.beginTransaction(SQLiteDatabase.java:690) - after some checks (not sure if database has to exist here) it will try to execute execSQL("BEGIN IMMEDIATE;")
SQLiteDatabase.execSQL(SQLiteDatabase.java:1965) - just forward
SQLiteDatabase.executeSql(SQLiteDatabase.java:2025) - builds another SQLiteStatement out of "BEGIN IMMEDIATE;. This one should be executed now
SQLiteStatement.executeUpdateDelete(SQLiteStatement.java:96) - starts with checking the database lock, this seems to be okay and the database should not be null here. The statement is then executed and finally the database is to be unlocked again.
SQLiteStatement.releaseAndUnlock(SQLiteStatement.java:290) - cleans up some stuff and in the end fails with NPE because the database is null.
Line numbers don't match so there are probably vendor modifications / additions in that code.
As you can see, the code crashes before actually using the data you supplied. It was about to do
BEGIN TRANSACTION IMMEDIATE; -- crash
INSERT INTO table (...) VALUES (...);
-- (end transaction)
That makes it in my opinion a framework bug. The database object that is internally handled there should not be able to be null somewhere down the line, especially when it seems that it was not null further up in the stack.
I also think that it is possible that another hidden exception could be the root cause for this. There are a lot of try { /* do stuff */ } finally { /* clean up */ } blocks within the code and the finally part will be executed even if the try part throws an exception. Now the finally block could cause another exception and the result is AFAIK that the original exception is replaced by the new exception from the finally block.
Especially executeUpdateDelete() is like
try {
acquireAndLock(WRITE);
// actual statement execution
} finally {
releaseAndUnlock();
}
if the database is closed at that point, acquireAndLock or any code in the try part could fail and that could leave the database object at null which causes releaseAndUnlock to fail again. You should get the same stacktrace.
Apart from that, don't do empty catch blocks like catch (SQLiteException e) { /* empty */ }. Log them with ACRA if possible / you don't do that already.
This NPE appears to be from a custom ROM as the Android source code is pointing to different Methods than the ones you receive in the LogCat. What I do for such cases is that: if the rate of these exceptions is very rare, I ignore them as it is difficult to know what custom ROM is running on the phone and more difficult to get the source code of this custom ROM to know where the problem is.
Not many users are using custom ROMs, so if you extensively tested your App on different phones with different SDKs and the rate of the Exceptions you get is not that significant, you can ignore them. Otherwise, you can take a shoot in the dark and speculate what can be in this custom ROM that is causing NPE (personally, I think it is not worth the effort).
I use sqlite transaction in Android:
SQLiteDatabase database = sqlite_helper.getWritableDatabase();
database.beginTransaction();
...
database.setTransactionSuccessful();
database.endTransaction();
My questions are :
1. Should I place endTransaction() in finally code block like this：
try {
database.beginTransaction();
...
database.setTransactionSuccessful();
}
finally {
database.endTransaction();
}
If there are exepctions during database operations, will the database be rolled back automatically without using "finally"?
When the transaction is not ended, can other threads read or write the same database? I hear sqlite in Android is threading safe, but I are not sure with it. I guess there will be some problems during transaction. Is there an error raised if another thread writes the same database with the same connection?
I ever found this error in my app, but I don't know whether it's related to the threading safe problem:
android.database.sqlite.SQLiteMisuseException: library routine called out of sequence:
, while compiling
Does anyone help me to answer these questions? Thanks a lot!
1.you should always place endTransaction in finally block
2.transaction in SQLite is thread safe,see the doc http://www.sqlite.org/atomiccommit.html
You should always put endTransaction() into a finally block (also see the docs).
Otherwise, the database would not be able to notice than an exception has happened.
The only other way to end a transaction would be to close the connection, in which case SQLite automatically rolls back any active transaction.
As long as one connection writes to the database (which means that a transaction is active), no other connections can read or write. Therefore, you should take care not to forget to end transactions.
You should never write from multiple threads; what would happen if one threads ends the transaction while the other one is still writing?
Your SQLiteMisuseException might be related, or not; that's impossible to say without seeing the code.
Yes, you should use the finally block. Here is a simple, THREAD SAFE method I use:
/**
* Call for multiple DB insertion transactions. It is thread safe and fast!
*/
private synchronized void writeTransaction(Runnable task) {
try {
db.beginTransaction();
task.run();
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
}
the synchronized keyword locks the method with its containing object, thus making it thread safe...