Develop Reference

Android sqlite unable to read after updating a blob

I want to store an image (size approx. 10MB) in the SQLite database. For that I created a DB helper, a Dao. Everything works fine, I can create several records and read them without a problem, I can even update the blob in the latest record without a problem.
But if I go back to an older record and update the blob, I cannot load this record with the blob any longer.
I have a list view where I show all the records, and for that I use a select that doesn't return the blob. This list works fine, but when I click on an item in the list, I try to load the record with the blob, the cursor returns 0 rows.
public void save(Bill aBill) {
ContentValues values = new ContentValues();
values.put(DatabaseHelper.BILL_NAME_COLUMN, aBill.getName());
values.put(DatabaseHelper.BILL_DUE_DATE_COLUMN, getContentValue(aBill.getDueDate()));
values.put(DatabaseHelper.BILL_IMAGE_COLUMN, aBill.getImage());
if (!aBill.isPersistent()) {
aBill.setId(database.insert(DatabaseHelper.BILL_TABLE, null, values));
aBill.setPersistent(true);
} else {
database.update(DatabaseHelper.BILL_TABLE, values, DatabaseHelper.BILL_ID_COLUMN + "=?", new String[]{String.valueOf(aBill.getId())});
}
}
// fails after updating the blob
public Bill get(long id) {
Cursor cursor = database.query(DatabaseHelper.BILL_TABLE,
new String[]{DatabaseHelper.BILL_ID_COLUMN, DatabaseHelper.BILL_NAME_COLUMN, DatabaseHelper.BILL_DUE_DATE_COLUMN, DatabaseHelper.BILL_IMAGE_COLUMN}, "id = ?", new String[] {String.valueOf(id)}, null,
null, DatabaseHelper.BILL_DUE_DATE_COLUMN);
Bill bill = null;
while (cursor.moveToNext()) {
bill = new Bill();
bill.setPersistent(true);
bill.setId(cursor.getLong(cursor.getColumnIndex(DatabaseHelper.BILL_ID_COLUMN)));
bill.setName(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_NAME_COLUMN)));
bill.setDueDate(getDate(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_DUE_DATE_COLUMN))));
bill.setImage(cursor.getBlob(cursor.getColumnIndex(DatabaseHelper.BILL_IMAGE_COLUMN)));
}
cursor.close();
return bill;
}
//works fine after updating the blob
public List findAll() {
List bills = new ArrayList();
Cursor cursor = database.query(DatabaseHelper.BILL_TABLE,
new String[]{DatabaseHelper.BILL_ID_COLUMN, DatabaseHelper.BILL_NAME_COLUMN, DatabaseHelper.BILL_DUE_DATE_COLUMN}, null, null, null,
null, DatabaseHelper.BILL_DUE_DATE_COLUMN);
while (cursor.moveToNext()) {
Bill bill = new Bill();
bill.setPersistent(true);
bill.setId(cursor.getLong(cursor.getColumnIndex(DatabaseHelper.BILL_ID_COLUMN)));
bill.setName(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_NAME_COLUMN)));
bill.setDueDate(getDate(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_DUE_DATE_COLUMN))));
bills.add(bill);
}
cursor.close();
return bills;
}
Here is the exception:
java.lang.IllegalStateException: Couldn't read row 0, col 0 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
at android.database.CursorWindow.nativeGetLong(Native Method)
at android.database.CursorWindow.getLong(CursorWindow.java:511)
at android.database.AbstractWindowedCursor.getLong(AbstractWindowedCursor.java:75)
at net.rka.android.billreminder.BillDao.get(BillDao.java:106)
I suspect that updating a blob in a row corrupts the database somehow.
Did anybody run into a similar problem? If so how did you solve it?

Your issue is very likely due to the size of the image(s) and a quirk, for want of a better term, that you can store large BLOB's without issue, but due to the size limitations of an Android's Cursor Window of 2m, that you may not be able to retrieve the BLOB. This sometimes compounded by some of the SQLiteDatabase/Cursor (The Cursor getBlob() or it's underlying methods in this case) methods that basically hide underlying failures, in order to provide what is often a simpler development experience.
If you used the SQLiteDatabase DatabaseUtils.dumpCursor this may highlight the issue(s) that may have been hidden by the SQLiteDatabase query convenience method. So adding :-
DatabaseUtils.dumpCursor(cursor); //<<<< ADDED
while (cursor.moveToNext()) { ........
May provide clues.
I can think of 3 options :-
Rather than store the files as BLOBS, store files as files on disk and store the path in the Database.
Significantly reduce the size of the images.
Look into using C++ and the native SQLIte3 libraries to retrieve the BLOBS into a suitably sized container.
Perhaps the may be some libraries that do this. However, I don't recall any being mentioned.

Cursor Index Issue

Can anyone please explain why this method in my database helper class is not working properly. I query a table named Tutorial with a layout column.
public String getLayout(int layout_position)
{
String SQL = "SELECT * FROM Tutorial";
Cursor cursor = database.rawQuery(SQL, null);
cursor.moveToFirst();
cursor.move(layout_position);
String layout = cursor.getString(cursor.getColumnIndex("layout"));
System.out.println(">>>DBhelper->getLayout: "+ layout);
return (layout);
}
The cursor is moved by int layout_position, which is incremented every time before I call the method. Everything actually works just fine until layout_position reaches 2. Then I get this error from logcat
07-20 09:35:41.479: E/AndroidRuntime(4630): android.database.CursorIndexOutOfBoundsException: Index 2 requested, with a size of 2
The Tutorial table is pre-populated with 4 rows, so I'm not sure what is happening here?

cursor.move moves the cursors position by the amount specified in the argument. You are most likely looking for cursor.moveToPosition(layout_position)

Why is such DELETE query not working?

This is my method to delete a row from the database where appointment_date is equal to a date that was passed in
public void deleteAllAppointments(String date) {
SQLiteDatabase db = this.getWritableDatabase();
String deleteAllQuery = "DELETE FROM " + TABLE_APPOINTMENTS + " WHERE appointment_date = '" + date + "'";
db.rawQuery(deleteAllQuery, null);
Log.d("Query: ", deleteAllQuery);
}
I then use it like this
//Database (DatabaseHandler is the one that contains all database methods)
final DatabaseHandler database = new DatabaseHandler(this);
//This happens when button is clicked, it is tested an executes with every chick,
//#param selectedDate is a string like "18/03/2014"
database.deleteAllAppointments(selectedDate);
It executes and query looks like this
DELETE FROM appointments WHERE appointment_date = '18/03/2014'
However row with appointment_date = '18/03/2014' is not deleted.
I'm sure database is set up correctly as I have working methods with it and all information is received from there in correct format.
NOTE: Adding "*" to "DELETE * FROM..." returns a fatal syntax error.

rawQuery() just compiles the SQL but does not run it. To actually run it, use either execSQL() or call one of the moveTo...() methods on the cursor returned by rawQuery().
For further info, see What is the correct way to do inserts/updates/deletes in Android SQLiteDatabase using a query string?

For tasks such as insert or delete there are really great "convenience methods" like the [delete method](http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html#delete(java.lang.String, java.lang.String, java.lang.String[])) already built in to the database.
public int delete (String table, String whereClause, String[] whereArgs)
As to why your current approach would fail, it could be something as simple the format of the column you're trying to delete not matching (e.g. you have created the table as a date value and not a string).
In any case, using the built in delete method is easier because it will notify you when it fails by returning the number of rows affected by the delete. rawQuery just returns a cursor, which you would then have to get the result from to see if it worked.

Are you sure your data value is in European format of day/month/year ala your query value of 18/03/2014 and maybe its not US style of month/day/year: 03/18/2014.
Not trying to be US-centric but that was my first thought.
Otherwise, definitely look at SQLiteDatabase.delete:
http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html#delete(java.lang.String, java.lang.String, java.lang.String[])

Quering SqlLite database where clause Android

Hi I am developing an android app.I am trying to query from the database. I need to fetch everything from the table TASK where dbDate = AlarmDate and dbdTime = AlarmTime.
c = db.rawQuery("SELECT * FROM TASK WHERE dbDate = '"+AlarmDate+"' AND dbTime= '"+Alarmtime+"'", null);
The problem is ,the cursor c is null.
I am not sure where I am going wrong in the query. Please Help.
Thanks!

Android has binding method to avoid sql inject. You can use the second parameter to provide the variables of SQL.
Cursor cur = db.rawQuery("SELECT * FROM TASK WHERE dbDate = ? AND dbTime = ? ", new String[]{AlarmDate, AlarmDate});

Going by your comment 'I have used db = openOrCreateDatabase("Globus", 0, null); where Globus is the db name', you are not using SQLite properly with android.
What you should be doing is creating class which extends SQLiteOpenHelper, then make sure you override the onCreate and onUpgrade methods, these are the methods where you create tables and make changes, it has been said a hundred times on here so I will provide a link to a tutorial: http://www.codeproject.com/Articles/119293/Using-SQLite-Database-with-Android
When you do database operations, on the class call getWritableDatabase (http://developer.android.com/reference/android/database/sqlite/SQLiteOpenHelper.html#getWritableDatabase())
I say call getWritableDatabase because that way you don't need to worry if you can write to it, a writable database is also readable. Just FYI. Ask away for more details.
This should be the process of reading (writing is the same, just use what method you want instead of query):
SQLiteDatabase db = dbHelper.getWritableDatabase();
db.beginTransaction();//this should lock the tables you are reading
Cursor c = db.rawQuery("select 1 where 1=?", new String[]{"1"});
if(c.moveToFirst()){
do{
//Do what you want with the row
}while(c.moveToNext());
}
c.close();
db.setTransactionSuccessful();
db.endTransaction();
db.close();
Here is the source code of a database helper I wrote, maybe it will help, read through it, understand how it works. https://bitbucket.org/FabianCCook/dbhelper/src/af7a8eba8d1a3f139e4170bbef9f1a2d3fdf1b47/src/nz/smartlemon/DatabaseHelper/ApplicationDataDbHelper.java?at=master
And if you want to know the reason the open methods exist read through this code
(This class was made from the help of someone elses code)
https://bitbucket.org/FabianCCook/dbhelper/src/af7a8eba8d1a3f139e4170bbef9f1a2d3fdf1b47/src/nz/smartlemon/DatabaseHelper/SDCardSQLiteOpenHelper.java?at=master

SQLiteDatabase db = getReadableDatabase();
Cursor cur = db.rawQuery("SELECT * FROM TASK WHERE dbDate = '"+AlarmDate+"' AND dbTime = '"+AlarmTime+"'",new String [] {});
Make sure you have gotten a readable database for 'db' or it will return null everytime.
Also change the end of your raw query to new String [] {}
Hope this helps, this is what I use in my applications.

Android/SQLite: Insert-Update table columns to keep the identifier

Currently, I am using the following statement to create a table in an SQLite database on an Android device.
CREATE TABLE IF NOT EXISTS 'locations' (
'_id' INTEGER PRIMARY KEY AUTOINCREMENT, 'name' TEXT,
'latitude' REAL, 'longitude' REAL,
UNIQUE ( 'latitude', 'longitude' )
ON CONFLICT REPLACE );
The conflict-clause at the end causes that rows are dropped when new inserts are done that come with the same coordinates. The SQLite documentation contains further information about the conflict-clause.
Instead, I would like to keep the former rows and just update their columns. What is the most efficient way to do this in a Android/SQLite environment?
As a conflict-clause in the CREATE TABLE statement.
As an INSERT trigger.
As a conditional clause in the ContentProvider#insert method.
... any better you can think off
I would think it is more performant to handle such conflicts within the database. Also, I find it hard to rewrite the ContentProvider#insert method to consider the insert-update scenario. Here is code of the insert method:
public Uri insert(Uri uri, ContentValues values) {
final SQLiteDatabase db = mOpenHelper.getWritableDatabase();
long id = db.insert(DatabaseProperties.TABLE_NAME, null, values);
return ContentUris.withAppendedId(uri, id);
}
When data arrives from the backend all I do is inserting the data as follows.
getContentResolver.insert(CustomContract.Locations.CONTENT_URI, contentValues);
I have problems figuring out how to apply an alternative call to ContentProvider#update here. Additionally, this is not my favored solution anyways.
Edit:
#CommonsWare: I tried to implement your suggestion to use INSERT OR REPLACE. I came up with this ugly piece of code.
private static long insertOrReplace(SQLiteDatabase db, ContentValues values, String tableName) {
final String COMMA_SPACE = ", ";
StringBuilder columnsBuilder = new StringBuilder();
StringBuilder placeholdersBuilder = new StringBuilder();
List<Object> pureValues = new ArrayList<Object>(values.size());
Iterator<Entry<String, Object>> iterator = values.valueSet().iterator();
while (iterator.hasNext()) {
Entry<String, Object> pair = iterator.next();
String column = pair.getKey();
columnsBuilder.append(column).append(COMMA_SPACE);
placeholdersBuilder.append("?").append(COMMA_SPACE);
Object value = pair.getValue();
pureValues.add(value);
}
final String columns = columnsBuilder.substring(0, columnsBuilder.length() - COMMA_SPACE.length());
final String placeholders = placeholderBuilder.substring(0, placeholdersBuilder.length() - COMMA_SPACE.length());
db.execSQL("INSERT OR REPLACE INTO " + tableName + "(" + columns + ") VALUES (" + placeholders + ")", pureValues.toArray());
// The last insert id retrieved here is not safe. Some other inserts can happen inbetween.
Cursor cursor = db.rawQuery("SELECT * from SQLITE_SEQUENCE;", null);
long lastId = INVALID_LAST_ID;
if (cursor != null && cursor.getCount() > 0 && cursor.moveToFirst()) {
lastId = cursor.getLong(cursor.getColumnIndex("seq"));
}
cursor.close();
return lastId;
}
When I check the SQLite database, however, equal columns are still removed and inserted with new ids. I do not understand why this happens and thought the reason is my conflict-clause. But the documentation states the opposite.
The algorithm specified in the OR clause of an INSERT or UPDATE
overrides any algorithm specified in a CREATE TABLE. If no algorithm
is specified anywhere, the ABORT algorithm is used.
Another disadvantage of this attempt is that you loose the value of the id which is return by an insert statement. To compensate this, I finally found an option to ask for the last_insert_rowid. It is as explained in the posts of dtmilano and swiz. I am, however, not sure if this is safe since another insert can happen inbetween.

I can understand the perceived notion that it is best for performance to do all this logic in SQL, but perhaps the simplest (least code) solution is the best one in this case? Why not attempt the update first, and then use insertWithOnConflict() with CONFLICT_IGNORE to do the insert (if necessary) and get the row id you need:
public Uri insert(Uri uri, ContentValues values) {
final SQLiteDatabase db = mOpenHelper.getWritableDatabase();
String selection = "latitude=? AND longitude=?";
String[] selectionArgs = new String[] {values.getAsString("latitude"),
values.getAsString("longitude")};
//Do an update if the constraints match
db.update(DatabaseProperties.TABLE_NAME, values, selection, null);
//This will return the id of the newly inserted row if no conflict
//It will also return the offending row without modifying it if in conflict
long id = db.insertWithOnConflict(DatabaseProperties.TABLE_NAME, null, values, CONFLICT_IGNORE);
return ContentUris.withAppendedId(uri, id);
}
A simpler solution would be to check the return value of update() and only do the insert if the affected count was zero, but then there would be a case where you could not obtain the id of the existing row without an additional select. This form of insert will always return to you the correct id to pass back in the Uri, and won't modify the database more than necessary.
If you want to do a large number of these at once, you might look at the bulkInsert() method on your provider, where you can run multiple inserts inside a single transaction. In this case, since you don't need to return the id of the updated record, the "simpler" solution should work just fine:
public int bulkInsert(Uri uri, ContentValues[] values) {
final SQLiteDatabase db = mOpenHelper.getWritableDatabase();
String selection = "latitude=? AND longitude=?";
String[] selectionArgs = null;
int rowsAdded = 0;
long rowId;
db.beginTransaction();
try {
for (ContentValues cv : values) {
selectionArgs = new String[] {cv.getAsString("latitude"),
cv.getAsString("longitude")};
int affected = db.update(DatabaseProperties.TABLE_NAME,
cv, selection, selectionArgs);
if (affected == 0) {
rowId = db.insert(DatabaseProperties.TABLE_NAME, null, cv);
if (rowId > 0) rowsAdded++;
}
}
db.setTransactionSuccessful();
} catch (SQLException ex) {
Log.w(TAG, ex);
} finally {
db.endTransaction();
}
return rowsAdded;
}
In truth, the transaction code is what makes things faster by minimizing the number of times the database memory is written to the file, bulkInsert() just allows multiple ContentValues to be passed in with a single call to the provider.

One solution is to create a view for the locations table with a INSTEAD OF trigger on the view, then insert into the view. Here's what that would look like:
View:
CREATE VIEW locations_view AS SELECT * FROM locations;
Trigger:
CREATE TRIGGER update_location INSTEAD OF INSERT ON locations_view FOR EACH ROW
BEGIN
INSERT OR REPLACE INTO locations (_id, name, latitude, longitude) VALUES (
COALESCE(NEW._id,
(SELECT _id FROM locations WHERE latitude = NEW.latitude AND longitude = NEW.longitude)),
NEW.name,
NEW.latitude,
NEW.longitude
);
END;
Instead of inserting into the locations table, you insert into the locations_view view. The trigger will take care of providing the correct _id value by using the sub-select. If, for some reason, the insert already contains an _id the COALESCE will keep it and override an existing one in the table.
You'll probably want to check how much the sub-select affects performance and compare that to other possible changes you could make, but it does allow you keep this logic out of your code.
I tried some other solutions involving triggers on the table itself based on INSERT OR IGNORE, but it seems that BEFORE and AFTER triggers only trigger if it will actually insert into the table.
You might find this answer helpful, which is the basis for the trigger.
Edit: Due to BEFORE and AFTER triggers not firing when an insert is ignored (which could then have been updated instead), we need to rewrite the insert with an INSTEAD OF trigger. Unfortunately, those don't work with tables - we have to create a view to use it.

INSERT OR REPLACE works just like ON CONFLICT REPLACE. It will delete the row if the row with the unique column already exists and than it does an insert. It never does update.
I would recommend you stick with your current solution, you create table with ON CONFLICT clausule, but every time you insert a row and the constraint violation occurs, your new row will have new _id as origin row will be deleted.
Or you can create table without ON CONFLICT clausule and use INSERT OR REPLACE, you can use insertWithOnConflict() method for that, but it is available since API level 8, requires more coding and leads to the same solution as table with ON CONFLICT clausule.
If you still want to keep your origin row, it means you want to keep the same _id you will have to make two queries, first one for inserting a row, second to update a row if insertion failed (or vice versa). To preserve consistency, you have to execute queries in a transaction.
db.beginTransaction();
try {
long rowId = db.insert(table, null, values);
if (rowId == -1) {
// insertion failed
String whereClause = "latitude=? AND longitude=?";
String[] whereArgs = new String[] {values.getAsString("latitude"),
values.getAsString("longitude")};
db.update(table, values, whereClause, whereArgs);
// now you have to get rowId so you can return correct Uri from insert()
// method of your content provider, so another db.query() is required
}
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}

Use insertWithOnConflict and set the last parameter (conflictAlgorithm) to CONFLICT_REPLACE.
Read more at the following links:
insertWithOnConflict documentation
CONFLICT_REPLACE flag

for me, none of the approaches are work if I don't have "_id"
you should first call update, if the affected rows are zero, then insert it with ignore:
String selection = MessageDetailTable.SMS_ID+" =?";
String[] selectionArgs = new String[] { String.valueOf(md.getSmsId())};
int affectedRows = db.update(MessageDetailTable.TABLE_NAME, values, selection,selectionArgs);
if(affectedRows<=0) {
long id = db.insertWithOnConflict(MessageDetailTable.TABLE_NAME, null, values, SQLiteDatabase.CONFLICT_IGNORE);
}

Use INSERT OR REPLACE.
This is the correct way to do it.