We are getting a large number of SQLiteDiskIOException errors in our Android app, with stack traces similar to the following:
E/AndroidRuntime( 2252): Caused by: android.database.sqlite.SQLiteDiskIOException: disk I/O error
E/AndroidRuntime( 2252): at android.database.sqlite.SQLiteQuery.native_fill_window(Native Method)
E/AndroidRuntime( 2252): at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:75)
E/AndroidRuntime( 2252): at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:288)
E/AndroidRuntime( 2252): at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:269)
E/AndroidRuntime( 2252): at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:171)
E/AndroidRuntime( 2252): at android.database.AbstractCursor.moveToFirst(AbstractCursor.java:248)
E/AndroidRuntime( 2252): at com.company.android.CActivity$QueryTask.doInBackground(CActivity.java:1660)
This recently starting happening a few weeks ago, but no significant database changes took place in the exact release in which reporting of this issue started. Despite having (I believe) appropriate indices, this exception has been reported more than we are comfortable with on cursor.moveToFirst() calls after the query completes, as you can see from the stack trace. So far, we have been completely unable to reproduce.
The query is somewhat complicated, with three left joins. Below is a representative query, with some identifying columns changed to protect the innocent.
select distinct c._id as _id,c.type as type,c.name as name,c.slug as slug,c.description as description,c.extra1 as extra1,c.extra2 as extra2,c.extra3 as extra3,c.extra4 as extra4,c.extra5,c.extra6 as extra6,c.extra7 as extra7,c.extra8 as extra8, c.extra9 as extra9,c.sslug as sslug,
c2.name as sname,
p.type as prel,
em.dS as dS,em.eS as eS
from cse as c
left join cse as c2 on c.sslug=c2.slug
left join emed as em on c.slug=em.slug
left join pre as p on c.sslug=p.slug
where c.pslug='slug' AND c.user='user' AND c.csource='csource'
order by c.type asc, c.extra6 desc, c.sortorder asc
Other sources have suggested that we are trying to pull out too much data, but this is simply not the case. The three user cases where we have been able to get full database rowcounts show:
cse with < 2000 entries,
emed with <150 entries, and
pre with either 0 or 7 entries.
Furthermore 'explain query plan' on the query indicates all joins are done against indexed columns.
In every case we have seen this, the user was running Android 2.1 on a variety of devices (DROID, Hero, EVO, possibly others). Notably, we have not seen this on a couple G1 devices we have, even when they are loaded down with other apps.
Finally, uninstalling and reinstalling has proven successful in cleaning up the issue, though possibly only temporarily.
I fear this problem is a result of data corruption in Android 2.1.
Does anyone have possible suggestions of what to look into?
Could this be related to this Android bug on SQLiteDatabaseCorruptException
Guidance and solutions are much appreciated.
It seems that you have multi threading issue, one thread tries to get the data while another one or more than one are trying to insert some data into your tables,because the exception is thrown from method (getCount).
Also don't forget; SQLite cursor is not internally synchronized, so if you use this cursor from multiple threads you should perform your own sync mechanism.
We have similar problem. When our application starts souch error occurs.
In onCreate method we check is there any database in application path. If any database occurs, we invoke such code
public boolean createDataBase() throws IOException
{
boolean dbExist = checkDataBase();
if(dbExist)
{
//do nothing - database already exist
}
else
{
//By calling this method and empty database will be created into the default system path
//of your application so we are gonna be able to overwrite that database with our database.
this.getReadableDatabase();
}
return dbExist;
}
private boolean checkDataBase()
{
SQLiteDatabase checkDB = null;
try
{
String myPath = DATABASE_PATH + DATABASE_NAME;
checkDB = SQLiteDatabase.openDatabase(myPath, null, SQLiteDatabase.OPEN_READONLY);
}
catch(SQLiteException e)
{
}
if(checkDB != null)
{
checkDB.close();
}
return checkDB != null ? true : false;
}
And if there is no database(first run of application) we copy it from resources.
But some times occurs SQLiteDiskIOException and database calls from resources.
one reason is as Bassel Kh stated, multi thread problom, the another is the db is unavailable, ie. db is deleted or sdcard is unavaliable.
Related
Im searching for some days one solution for my problems with sqlite and android. I read a lot of articles about sqlite and android. I see people saying that you need close CURSOR, and others saying that you need close DATABASE, and others that you need close BOTH, and OTHERS that you dont need close NOTHING, because one Android Engineering post.
In my apps we have too many uses of sqlite.
1 - We store things to be synced.
2 - We add/delete things in background.
3 - We have +/- 10 background services that are started by Alarm Manager to sync somethings.
Everything running in same time, multiple threads and more.
When i get the app, we have a lot and a lot and a lot of crashes from closing the cursor/database while other cursor is opened at the same time. A lot of confusion.
So, after reading, a google engineer post (i dont remember what google group), an answer that you dont need close sqlite cursor or database (i dont remember and i dont know if make difference).
So now in my background services that sync things, i get errors about it:
Fatal Exception: android.database.CursorWindowAllocationException
Cursor window allocation of 2048 kb failed. # Open Cursors=593 (# cursors opened by this proc=593)
So what i need do? What is the correct way? What will work with a lot of threads and IntentServices using it?
My sample code today:
public static MyInternalObject getFirstAudience() {
SQLiteDatabase db = UbookProvider.getInstance().getReadableDatabase();
String sql = "SELECT * FROM " + UbookContract.MyInternalObject.TABLE_NAME + " LIMIT 1";
Cursor cursor = db.rawQuery(sql, null);
if (cursor != null) {
if (cursor.getCount() > 0) {
cursor.moveToFirst();
do {
return MyInternalObjectService.createFromCursor(cursor);
} while (cursor.moveToNext());
}
}
return null;
}
Im on dark, i dont know a decent way to do it to solve my problems with database in android.
Anyone can help me?
My github (github.com/prsolucoes) have a lot of open-source projects too. I do a lot of things to community, but today is i that need.
Yes, you need to close your cursors. One way to do this is to wrap all of your queries in a try/finally statement.
Cursor c = null;
try {
c = db.query(...)
} finally {
if (c != null) {
c.close();
}
}
If you are using CursorLoaders (which I highly recommend), then you don't need to close your cursors, because the Loader framework manages it for you.
I have a read-only database connection. Sometimes, when reading data from the database with a SELECT query, it throws a SQLiteReadOnlyDatabaseException.
I open the connection like this:
return SQLiteDatabase.openDatabase(path, null, SQLiteDatabase.OPEN_READONLY);
The query is:
Select * FROM BudgetVersions WHERE entityId = ?
I read data from the database using db.rawQuery(), like this:
String query = ...;
Cursor c = db.rawQuery(query, new String[]{ activeBudgetId });
try {
if (c.moveToFirst()) {
bv.versionName = c.getString(c.getColumnIndexOrThrow("versionName"));
return bv;
} else {
return null;
}
} finally {
c.close();
}
Very rarely, I get a crash like this, inside the call to c.moveToFirst():
Caused by: android.database.sqlite.SQLiteReadOnlyDatabaseException: attempt to write a readonly database (code 776)
at android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(Native Method)
at android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:845)
at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:836)
at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62)
at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:144)
at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:133)
at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:197)
at android.database.AbstractCursor.moveToFirst(AbstractCursor.java:237)
As a workaround, I could try using a writable database connection instead, but I'd like to know why the crash is happening.
The table I'm reading from is a standard SQLite table:
CREATE TABLE BudgetVersions (
entityId VARCHAR PRIMARY KEY NOT NULL UNIQUE,
budgetId VARCHAR NOT NULL,
versionName VARCHAR NOT NULL,
dateFormat VARCHAR,
currencyFormat VARCHAR,
lastAccessedOn DATETIME,
isTombstone BOOL NOT NULL,
deviceKnowledge NUMERIC NOT NULL
);
I've seen the crash happen on both a KitKat emulator and a device running Lollipop.
There is a separate writeable connection open to the same database at the same time, owned by a WebView. The database is being updated by Javascript code in the WebView, and read from in the native Android/Java layer with this read-only connection.
I expect this may prove to be the ultimate cause of the problem, but I'd like to understand in detail why a read-only connection would interfere with a separate writeable connection.
I am well aware that the general advice is to use a single connection to the database, but since the writeable connection is owned by the WebView, I don't have easy access to it from the Java code.
Solved by changing it to a writeable database connection. The clue was in the documentation for the 776 error code:
(776) SQLITE_READONLY_ROLLBACK
The SQLITE_READONLY_ROLLBACK error code is an extended error code for
SQLITE_READONLY. The SQLITE_READONLY_ROLLBACK error code indicates
that a database cannot be opened because it has a hot journal that
needs to be rolled back but cannot because the database is readonly.
During development, I am frequently interrupting the currently-running app to install and run a new version. This causes the currently-running app to be force-stopped by the system. If the Javascript code in the WebView is in the middle of writing to the database via its separate writeable connection when the app is nuked, then a hot journal will be left behind.
When the new version of the app starts up, the read-only database connection in the native Java code is opened. When this connection spots the journal, it tries to roll back the journal. And because it's a read-only connection, it fails.
(This fits with the crash being observed immediately on startup after I've made a change.)
The correct fix is therefore to make the Java connection a writeable connection. This connection never attempts a write during normal operation, but it must write when recovering from a previous interrupted write through the WebView's writeable connection.
I'm trying to fix a DB issue I had in Ankidroid. The following statement was executed:
SELECT count(*) FROM cards c WHERE type = 1 AND combinedDue <
1335153600.000000
It was generated there. The code executed is the following (github link):
cursor = mDatabase.rawQuery(query, null);
if (!cursor.moveToNext()) {
throw new SQLException("No result for query: " + query);
}
I can't understand how I can get no record, I should get either 1 or 0. The call stack in the log was the following:
at com.ichi2.anki.Deck.rebuildRevCount(Deck.java:1351)
Caused by: android.database.SQLException: No result for query: ……
at com.ichi2.anki.AnkiDb.queryScalar(AnkiDb.java:129)
at com.ichi2.anki.Deck._rebuildRevCount(Deck.java:1621)
Any idea?
In fact, I was more looking at a similar case. We have a table «Stats» with a column "global", in which there can be one global record and many daily records. If we don't find (a simple select) the global record, we create it. On my phone, it seems that sometime the global record is not found, so we create an additional one, which break things.
It really look like the case I showed above.
Edit:
I found why. In another thread, an AsyncTask closes the DB at the same time the query is being made (because a lot of processing triggered by the GUI is done asynchronously). And it returns a cursor with no record.
I saw that by adding traces to a file.
Try if (cursor.moveToFirst == false) instead of using the ! operator. I think that the way you are currently doing it is always going to.pass you into the body of the if. And the cursor isn' t properly initialized.without a.moveToFirst.
I have an app that uses a database with 3 tables in it. Those 3 tables have data read from and written to them by activities and services.
Having gotten a few "android.database.sqlite.SQLiteException: database is locked" crashes, I went in to the database adapter class and wrapped every write, update, or delete function with a synchronized statement, so like:
public int deleteExpiredAlarms() {
String whereClause = FIELD_EXPIRED + " = 1";
int val = 0;
synchronized(dbWriteLock) {
val = db.delete(ALARM_DATABASE_TABLE, whereClause, null);
}
return val;
}
That seemed to make it better. But lately it's gotten bad again as I've added more services that read and write to different tables.
Do I need to synchronize ALL db access statements, including queries?
The exception is occurring on the attempt to open the writable database via the open helper...should I synchronize that act also?
I've heard that I should only be using one db helper so that there won't be issues with multiple threads accessing the db. How do I use only one db helper? Every example I've seen so far has the db helper as an instantiated value inside the db adapter....so wouldn't that be a separate db helper per db adapter instantiated (one in an activity, one in a service running,etc)
I've looked at using a content provider instead, as it's been claimed to solve problems like this, but it's really more work than I want to do if I should be able to have direct db access without locking issues. And I do not plan to make this db accessible to other apps.
Thanks for the help.
This link describes my problem exactly: http://old.nabble.com/Android-database-corruption-td28044218.html#a28044218
There are about 300 people using my Android App right now and every once and while I get a crash report to the server with this stack trace:
android.database.sqlite.SQLiteDatabaseCorruptException: database disk image is malformed
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2596)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2621)
at android.app.ActivityThread.access$2200(ActivityThread.java:126)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1932)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:123)
at android.app.ActivityThread.main(ActivityThread.java:4595)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:521)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:860)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:618)
at dalvik.system.NativeStart.main(Native Method) Caused by: android.database.sqlite.SQLiteDatabaseCorruptException: database disk image is malformed
at android.database.sqlite.SQLiteQuery.native_fill_window(Native Method)
at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:75)
at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:295)
at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:276)
at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:171)
at android.database.AbstractCursor.moveToFirst(AbstractCursor.java:248)
The result is the app crashing and all the data in the DB being lost.
One thing to note is that every time I read or write to the database I get a new SQLiteDatabase and close it as soon as I'm done. I did this in an attempt to prevent these kind of corruption errors.
I also tried synchronizing all DB reads and writes using a single static object and that didn't seem to help.
Is it possible this is just a SQLite bug?
I found a similar bug with the built-in email app here: http://code.google.com/p/android/issues/detail?id=5610.
Here is my code:
public class KeyValueTableAdapter extends BaseTableAdapter {
private String tableName;
private String keyColumnName;
private String valueColumnName;
public KeyValueTableAdapter(Context context, String tableName, String keyColumnName, String valueColumnName) {
super(context);
this.tableName = tableName;
this.keyColumnName = keyColumnName;
this.valueColumnName = valueColumnName;
}
protected String getStringValue(int key) {
Cursor cursor = null;
SQLiteDatabase db = null;
String value;
try {
db = dbOpenHelper.getReadableDatabase();
cursor = db.query(true, tableName, new String[] { valueColumnName }, keyColumnName + "=" + key, null, null, null, null, null);
if ((cursor.getCount() == 0) || !cursor.moveToFirst()) {
value = null;
} else {
value = cursor.getString(0);
}
} finally {
if (cursor != null) cursor.close();
if (db != null) db.close();
dbOpenHelper.close();
}
return value;
}
}
public abstract class BaseTableAdapter {
protected DbOpenHelper dbOpenHelper;
public BaseTableAdapter(Context context) {
this.dbOpenHelper = new DbOpenHelper(context, DatabaseSettings.DATABASE_NAME, null, DatabaseSettings.DATABASE_VERSION);
}
}
Most likely the database process(es) is getting killed during an I/O. For example by a task killer, or if you're allowing db write operations to continue during a time when the app should shutdown or sleep...
See if you can reproduce the issue by putting your app in a DB write loop and using a task killer on it.
Scenario: 32 bytes being written to the database, the writer task gets killed after only writing 10, result: database left in inconsistent and possibly corrupt state.
Also see:
Android process killer
EDIT: opening and closing the DB for each read/write? stop that! :)
Using multiple instances of SQLiteDatabase could be causing your problem if you have two instances updating the same database file at the same time.
Implement a backup db process each day, and if the DB gets corrupted you just replace the database with a backup. You can use simple file copy paste methods to maintain a backup each day.
Since I can't comment -yet- on Brad's post.
I have to agree with the extra overhead.
I have a HTC Magic as my daily phone, and ram is always an issue.
Android phones are at very different ends of the $$$
Some are super cheap and some are super expensive, this basically comes down to ram, and cpu.
People who run task killers do ruin their android phones.
As a developer you should suggest people not to use them, or just deny support to people who use task killers, since android doesn't need these "improvements" (ask steve (cyanogen))
Also the new statement in android is very expensive.
You want to limit the amount of new calls when programming for android.
Programming for android is all about reuse of the precious memory. (HTC Magics/Dreams only have 96MB available to applications, and most of it is already in use)
As for your SQLiteDB...the API says that your SQLiteDB is private to your application.
I don't see why you need to open and close a NEW connection to it each time you want to read or write to it.
I would rather keep the connection open until the user looses focus from it.
However, if you are writing a content provider, that is a different story.
"the DB holds session information so
it's not very feasible to do a backup.
The data changes by the minute"
You should try using SharedPreferences: it stores key-value pairs (in the background, it uses a file).
Storing values:
SharedPreferences sp=MyActivity.getSharedPreferences("Name", Context.MODE_PRIVATE);
SharedPreferences.Editor editor = sp.edit();
editor.putString("key", value);
editor.putBoolean("another", true);
editor.commit();
Retrieving data:
sp.getString("key", "Not found");
// "Not found" is the default value
// if sp does not contain the specified key
sp.getBoolean("another", false);
// false is the default value
// if sp does not contain the specified key
See getSharedPreferences and SharedPreferences for a more detailed description.