Im searching for some days one solution for my problems with sqlite and android. I read a lot of articles about sqlite and android. I see people saying that you need close CURSOR, and others saying that you need close DATABASE, and others that you need close BOTH, and OTHERS that you dont need close NOTHING, because one Android Engineering post.
In my apps we have too many uses of sqlite.
1 - We store things to be synced.
2 - We add/delete things in background.
3 - We have +/- 10 background services that are started by Alarm Manager to sync somethings.
Everything running in same time, multiple threads and more.
When i get the app, we have a lot and a lot and a lot of crashes from closing the cursor/database while other cursor is opened at the same time. A lot of confusion.
So, after reading, a google engineer post (i dont remember what google group), an answer that you dont need close sqlite cursor or database (i dont remember and i dont know if make difference).
So now in my background services that sync things, i get errors about it:
Fatal Exception: android.database.CursorWindowAllocationException
Cursor window allocation of 2048 kb failed. # Open Cursors=593 (# cursors opened by this proc=593)
So what i need do? What is the correct way? What will work with a lot of threads and IntentServices using it?
My sample code today:
public static MyInternalObject getFirstAudience() {
SQLiteDatabase db = UbookProvider.getInstance().getReadableDatabase();
String sql = "SELECT * FROM " + UbookContract.MyInternalObject.TABLE_NAME + " LIMIT 1";
Cursor cursor = db.rawQuery(sql, null);
if (cursor != null) {
if (cursor.getCount() > 0) {
cursor.moveToFirst();
do {
return MyInternalObjectService.createFromCursor(cursor);
} while (cursor.moveToNext());
}
}
return null;
}
Im on dark, i dont know a decent way to do it to solve my problems with database in android.
Anyone can help me?
My github (github.com/prsolucoes) have a lot of open-source projects too. I do a lot of things to community, but today is i that need.
Yes, you need to close your cursors. One way to do this is to wrap all of your queries in a try/finally statement.
Cursor c = null;
try {
c = db.query(...)
} finally {
if (c != null) {
c.close();
}
}
If you are using CursorLoaders (which I highly recommend), then you don't need to close your cursors, because the Loader framework manages it for you.
Related
I sometime get this error in my logcat.
Failed to read row 0, column 0 from a CursorWindow which has 0 rows, 64 columns.
First, a bit of a back story. I have an app that runs on many devices in our organisation. Primarily, it currently runs on about 20 x Samsung Note 8 devices, 2 x Samsung Note 10.1 devices and a couple of other ones. So far, the problem has only been happening on 2 of the Note 8 devices. On all of the other devices it seems to work just fine.
How the app works is that users use the app to gather information, text, photos, signature etc... and all of this is then stored/inserted into the SQLite database as a row in the submissions table. The submissions table has 64 columns to cater for each field being collected. There is a sync method that is always running (it is an AsyncTask that is executed inside of a runnable thread, so even if the app is closed, it still syncs the data in the background, unless you swipe close it from the android task manager), that syncs the data to a remote server and that checks each 10 seconds if a sync is needed, for example if a new submission was inserted, it will then start syncing that submission. When a submission is finished syncing with the server and receives a success response, it is then deleted from the device's database. So far it has been working great and thousands of submissions have been synced successfully etc. however, every now and again, I get this one error from one or two specific Note 8 tablets that reproduce this problem. I have tried many times to re create the error but it always works when I test it and I have tried every type of scenario to test it.
My code is thousands of lines, so I will try to keep it as relevant as possible. First, here is the relevant code for the runnable:
public Runnable myRunnable = new Runnable()
{
#Override
public void run()
{
count ++;
if(count >= 10)
{
android.util.Log.w(" SYNC ", "---------------");
android.util.Log.w(" SYNC ", "Checking if sync method is busy");
if(!syncBusy)
{
android.util.Log.w(" SYNC ", "Sync method OPEN");
doSync();//This will start doing sync.
count = 0;
}
else
{
android.util.Log.w(" SYNC ", "Sync method BUSY, will try again in 10 seconds");
android.util.Log.w(" SYNC ", "---------------");
}
}
if(count == 1 && !syncBusy)
{
checkSubmissionsLefttoSync();
}
mHandler.postDelayed(myRunnable, 1000);
}
};
Then, the doSync() method is where I do the upload and also where I get the error. It's over a thousands lines long, so I don't really want to post the code here. What I can say is that it works 100% for all devices except the one or two exceptions that produce the above mentioned error.
Also, I will post the part where I actually traverse the database inside of the sync method:
databaseHelper = new Handler_Database(this);
Cursor cursor2 = databaseHelper.getAllUnsyncedSubmissions();
if(cursor2.getCount() > 0)
{
if(cursor2.moveToFirst())
{
do
{
try
{
//doing lookups here and populating sync arrays
}
catch (IllegalStateException e)
{
//Do Nothing
}
catch (NullPointerException e)
{
//Do Nothing
}
}
while(cursor2.moveToNext());
}
else
{
}
}
else
{
}
cursor2.close();
databaseHelper.close();
I have noticed something else and I am not sure if it's a problem, however, when I run the app, logcat outputs the following line about 6 or 7 times:
12:48:11.115 7416 #7416 DEBUG SQLiteOpenHelper DB version : 60
I have my own warning messages when the app starts up, and those ones only display once. However, the db version and other info is being written multiple times to the logcat. Is this a problem ? Does this mean my database has some sort of bug that it creates multiple instances ?
All the apps are updated with signed apk's from the playstore. My database onUpgrade method looks like this:
#Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion)
{
// Drop older table if existed
db.execSQL("DROP TABLE IF EXISTS " + TABLE_SUBMISSIONS);
// Create tables again
onCreate(db);
}
This works fine for all the other tablets and device. They all update automatically from the playstore and continue to work as they should. However, why is logcat outputting those lines multiple times ?
Could this be part of the problem why I am getting the error as mentioned in the beginning of this post ?
Any possible insight would be gladly appreciated. I have been pulling my hair out for weeks and cannot find any fault in my code.
UPDATE 1:
I just saw this warning:
08:30:58.710 16745 #16745 WARN CursorWindow Window is full: requested allocation 307333 bytes, free space 249426 bytes, window size 2097152 bytes
I guess that this might be the root of my problems. Any ideas on how to solve this efficiently ?
UPDATE 2:
I think a solution might be to limit the columns that I retrieve from with the cursor. E.g., all the small text values/columns. Then afterwards, create a new query for each of the columns that I know takes up too much space. Do you think this is a solution ? Anyone ?
UPDATE 3:
This might work, I think I will do the large fields in separate cursors after the initial one is recycled. Something like this (I just wrote some psuedo code):
Cursor c = db.rawQuery("SELECT Column1, Column2 .... FROM table " + ... , ...);
Solution:
See my solution here ! https://stackoverflow.com/a/26797130/1518916
I finished my solution and it works fine now. Basically, the bottom line is that it is not ideal to store large data in a sqlite database on android. When working with images, rather store just the URI and keep the image on the device. HOWEVER, if you must, then I recommend to not load all image data into the same cursor, split them up and do multiple queries.
Please see my answer here https://stackoverflow.com/a/26797130/1518916
This error occurs when your images are of large size. Either store the url of the image or compress the image before storing.
I know this question has asked many times in SO,but i couldn't figure out my exact problem.
I am using the following code to get the data from the database(Table1) and update another Table2 based on retrieval value. Its working fine in some android versions but when i gone to test with Android 4.0.3. I am geting this java.lang.IllegalStateException:?.attempt to re-open an already-closed object at sum_cursor.moveToNext();.
I am using this code in AsyncTask.
/** Sum of total matched values*/
Cursor sum_cursor = db.gettotalMatchvalue(this);
if(sum_cursor!=null)
{
sum_cursor.moveToFirst();
for(int j=0; j<sum_cursor.getCount();j++)
{
float totalmatchedscore = sum_cursor.getInt(0);
float totalingredients = Float.parseFloat(sum_cursor.getString(sum_cursor.getColumnIndex(APPDatabase.CK_TOTALINCREDIENTS)));
/**average = totalscore/totalingredients*/
double average = totalmatchedscore/totalingredients;
int id = Integer.parseInt(sum_cursor.getString(sum_cursor.getColumnIndex(APPDatabase.CK_ID)));
db.updateAverage(id, average);
sum_cursor.moveToNext(); //Here is the problem
}
}
db.close();
My update method coding
/** Update average */
public void updateAverage(int id,double average)
{
SQLiteDatabase db = getWritableDatabase();
ContentValues values = new ContentValues();
values.put(CK_FINALVALUE,average);
db.update(TABLE, values,CK_ID+" = "+id , null);
}
What i am doing wrong here?
I know many of you come across this situation. Could you help me guys.
Thanks for your help.
You can't update a table while iterating over the results of a query. There are good reasons for this; what if the data you are adding would cause a change to the data you're iterating over? Your cursor would not return valid data.
Trying to store data back into the table in updateAverage() is causing the problem. You should just remember the average value during your loop, and then update it once at the end after you've finished looping over your cursor.
To further explain the exact error you're getting: the act of inserting new data is causing the database to close all the cursors which are currently open, as a safety measure. So when you call sum_cursor.moveToNext() after updating the average, the cursor is a bit surprised to find that it's already been closed.
What if you comment out db.updateAverage(id, average) ?
You can achieve your goal with pure SQL it is faster and better from architecture point of View because all logic will be in one SQL transaction
REPLACE INTO table_where_to_put SELECT *, (totalmatchedscore/totalingredients) as average FROM table_with_input_data
Use REPLACE OR UPDATE
I'm trying to fix a DB issue I had in Ankidroid. The following statement was executed:
SELECT count(*) FROM cards c WHERE type = 1 AND combinedDue <
1335153600.000000
It was generated there. The code executed is the following (github link):
cursor = mDatabase.rawQuery(query, null);
if (!cursor.moveToNext()) {
throw new SQLException("No result for query: " + query);
}
I can't understand how I can get no record, I should get either 1 or 0. The call stack in the log was the following:
at com.ichi2.anki.Deck.rebuildRevCount(Deck.java:1351)
Caused by: android.database.SQLException: No result for query: ……
at com.ichi2.anki.AnkiDb.queryScalar(AnkiDb.java:129)
at com.ichi2.anki.Deck._rebuildRevCount(Deck.java:1621)
Any idea?
In fact, I was more looking at a similar case. We have a table «Stats» with a column "global", in which there can be one global record and many daily records. If we don't find (a simple select) the global record, we create it. On my phone, it seems that sometime the global record is not found, so we create an additional one, which break things.
It really look like the case I showed above.
Edit:
I found why. In another thread, an AsyncTask closes the DB at the same time the query is being made (because a lot of processing triggered by the GUI is done asynchronously). And it returns a cursor with no record.
I saw that by adding traces to a file.
Try if (cursor.moveToFirst == false) instead of using the ! operator. I think that the way you are currently doing it is always going to.pass you into the body of the if. And the cursor isn' t properly initialized.without a.moveToFirst.
I use the following method for reading/writing db:
Database is located at /data/data/{packagename}/databases/Database.db
Since the database is greater than 3Mb we found a specific solution to have it copied there and to have it populated with appropriate data.
Following is the class implementing the task to get the opened database. This class is a singleton.
public class DatabaseHelper extends SQLiteOpenHelper
to open the database we use the following method:
SQLiteDatabase db = DatabaseHelper.getInsance().getReadableDatabase();
Then rawquery is used for querying the db:
Cursor cursor = db.rawQuery(query, null);
Then best fitting to our purposes we fetch the database data into memory in different resultset instances:
while (cursor.moveToNext()) {
ResultSet rs = new ResultSet();
rs.setThis(cursor.getInt(0));
rs.setThat(cursor.getString(1));
// and so on.. this is just an example
ResultList.add(rs);
}
Finally:
cursor.close();
db.close();
Let mention, if necessary, transaction is used also, but using transaction didn't lead to speed-up.
For every single query the pattern above is (quite) followed. But unfortunately this solution seems very slow. So some method profiling is made and it came to clear, that sqlite setlocale is always run at getReadableDatabase() (which is created! don't forget) and that method takes the most of the time. Meanly 40% alone..
Please advice how to solve this problem! Or maybe please offer an other pattern to satisfy our needs!
Thanks in advance
Szia!
Funniest thing is, native_setLocale (which is causing the slow DB open) apparently doesn't even work: http://code.google.com/p/android/issues/detail?id=2625
6) Finally:
cursor.close();
db.close();
It's not possible to keep the database open between queries?
As with the question posed here (SQLCipher for Android getReadableDatabase() Overherad) the performance issue you are seeing is likely due to SQLCipher key derivation. Performance for opening a database is deliberately slow due to key derivation. You should cache the database connection so that it can be used multiple times without having to open and key the database repeatedly. If this is possible, opening the database once during startup is the preferred course of action. Subsequent access on the same database handle will not trigger key derivation, so performance will be much faster.
Not a total show stopper for my project, but I'm a little concerned at the performance of SQLITE when I am deleting a specific row from the database.. Sometimes it's taking up to 5 seconds which seems too long and I can imagine users may thing the application has crashed.
My code is very straightforward, the delete call is simply :
boolean result;
result = mDb.delete(ACCIDENTS_MEDIA_TABLE, ACCIDENTS_MEDIA_KEY_ROWID + "=" + rowId, null) > 0;
So nothing odd there, the table is relative small in terms of fields (5) but 2 of those are blob fields so I'm wondering if that can be the reason.
If anyone has experience with this & suggestions how to improve my performance so it's not taking up to 5 seconds to delete a row, that would be appreciated.. Thanks.
Rgds,
Using an ASyncTask, or a seperate Runnable/Thread with a handler is, as suggested the best way to approach these type of issues.