A very pertinent question (at least I think it is) came to me while developing an Android app.
Example
We are inserting 10000 rows on a db (at once).
db.beginTransaction();
try{
for(Object toInsert: listOfObjects) {
ContentValues values = new ContentValues();
//put the values on the object
values.put(key, toInsert.getValue());
db.insert(columnName, null, values);
}
db.setTransactionSuccessful();
} catch(Exception e) {
//handle exception
} finally {
db.endTransaction();
}
We are creating 10000 new ContentValue objects on the loop. And object creation is very expensive to the VM.
And if we modify this a little bit?
Different Approach
ContentValues values, hack = new ContentValues();
db.beginTransaction();
try{
for(Object toInsert: listOfObjects) {
values = hack;
//put the values on the object
values.put(key, toInsert.getValue());
db.insert(columnName, null, values);
}
db.setTransactionSuccessful();
} catch(Exception e) {
//handle exception
} finally {
db.endTransaction();
}
In this second example, we are making a 'reset' to the value object, because that will be used in every single row.
And so, my question is: Am I doing this right? With the second approach I'm optimizing the process without leaving a big memory footprint? If not, why? Have you some suggestions/thoughts about this?
You are doing it wrong with the two variables.
Consider the following case:
In first iteration, values = new instance, hack = new instance. OK.
after you do values = hack. values and hack both refer to the same memory location now. So there is no point in creating two variables.
You could simply do following:
ContentValues values = new ContentValues();
db.beginTransaction();
try{
for(Object toInsert: listOfObjects) {
//put the values on the object
values.put(key, toInsert.getValue());
db.insert(columnName, null, values);
}
db.setTransactionSuccessful();
} catch(Exception e) {
//handle exception
} finally {
db.endTransaction();
}
Related
ATM when I write to the SQLite in my android app, I do it this way:
try {
for (User user: users) {
ContentValues values = new ContentValues();
databaseManager.database.beginTransaction();
values.put("user_name", user.getName());
values.put("user_email", user.getEmail());
databaseManager.database.insert("users", null, values);
}
databaseManager.database.setTransactionSuccessful();
} catch (Exception ex) {
ex.printStackTrace();
} finally {
databaseManager.database.endTransaction();
}
But when I read from the DB, I dont use begin, setsuccessful and end:
Cursor cursor = databaseManager.database.rawQuery(SQLQueries.getUsers(), null);
if (cursor.moveToFirst()) {
if (cursor!=null) {
do {
User user = new User();
try {
user.setName(cursor.getString(cursor.getColumnIndexOrThrow("user_name")));
user.setEmail(cursor.getString(cursor.getColumnIndexOrThrow("user_email")));
} catch (Exception ex) {
ex.printStackTrace();
}
users.add(user);
} while (cursor.moveToNext());
}
}
if (cursor != null && !cursor.isClosed()) {
cursor.close();
cursor = null;
}
Should I add beginTransaction, setTransactionSuccessful and endTransaction to the read operations as well? Im pretty sure I shouldnt, but I need to be 100% on this one.
When you are not using explicit transactions, SQLite will automatically wrap a transaction around every statement.
When you write to the database, every insert/update/delete call is a single statement.
If you are doing multiple such operations, you use transactions to avoid paying the transaction overhead for each of them.
A query (query or rawQuery) is a single statement, even if it returns multiple rows.
Therefore, using a transaction around a single query does not make any difference.
(If you have multiple queries, you could use a transaction to ensure that their results are consistent with each other even if another thread attempts to change the database between them.)
I am using transactions to insert record to my database. Can you please tell me whether this is the right way to get total inserted record (return by numrow)? Also, in the code below, if some insert fails, will it continue to next insertion or will exit (I didn't use endTransaction in Catch block)?
int numrow = 0;
try{
db.beginTransaction();
for(mylibman cn : insertlist){
ContentValues values = new ContentValues();
values.put(KEY_LIBID, cn.getLibid());
values.put(KEY_NAME, cn.getBookname());
db.insertWithOnConflict(TABLE_NAME, null, values,SQLiteDatabase.CONFLICT_IGNORE);
numrow++;
}
db.setTransactionSuccessful();
}catch (Exception e) {
} finally {
db.endTransaction();
}
return numrow;
you should check like this
int numrow = 0;
try{
db.beginTransaction();
for(mylibman cn : insertlist){
ContentValues values = new ContentValues();
values.put(KEY_LIBID, cn.getLibid());
values.put(KEY_NAME, cn.getBookname());
//do like this
long insertedId=db.insertWithOnConflict(TABLE_NAME, null, values,SQLiteDatabase.CONFLICT_IGNORE);
if(insertedId!=-1)
{
numrow++;
}
}
db.setTransactionSuccessful();
}catch (Exception e) {
} finally {
db.endTransaction();
}
return numrow;
Your code will also count records that were ignored.
insertWithOnConflict returns the rowid of the inserted record, or -1, so you have to check for that:
if (db.insertWithOnConflict(TABLE_NAME, null, values,
SQLiteDatabase.CONFLICT_IGNORE) != -1)
numrow++;
The insertWithOnConflict function does not throw an exception if a record is not inserted due to a conflict. However, it will throw if there is some other error, such as an unknown column name, or a read-only database.
You should not blindly ignore exceptions; just use try/finally:
db.beginTransaction();
try {
for (...) {
...
}
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
Non of the answers here reflect the documentation:
"returns:
the row ID of the newly inserted row OR the primary key of the existing row if the input param 'conflictAlgorithm' = CONFLICT_IGNORE OR -1 if any error"
Since the author wants to know the 'inserted' columns, both answers are wrong. The number inserted rows could be 0, as they already exist. However "CONFLICT_IGNORE" will make insert return the existing primary key, so the numrow will be the rows you 'tried' to insert and didn't fail, but not the rows 'inserted'.
You might have to use the flag "SQLiteDatabase.CONFLICT_ABORT" or "SQLiteDatabase.CONFLICT_FAIL" depending on what and how you insert.
Source: http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html#insertWithOnConflict%28java.lang.String,%20java.lang.String,%20android.content.ContentValues,%20int%29
im trying to do a batch insert of about 700 floats. The method i'm using is below and as well as the content provider's bulkInsert. The issue is that when i put all the floating point values into the ContentValues nothing happens. What's a better way to insert those floating point values into the ContentValues object?
private void saveToDatabase( float[] tempValues )
{
ContentValues values = new ContentValues();
// WM: TODO: add patient id and sensor type
for (float tempVal : tempValues){
values.put( DataTable.COLUMN_DATA, tempVal );
}
ContentValues[] cvArray = new ContentValues[1];
cvArray[0] = values;
ContentResolver resolver = getContentResolver();
resolver.bulkInsert( HealthDevContentProvider.CONTENT_URI_DATA, cvArray);
public int bulkInsert(Uri uri, ContentValues[] values){
int numInserted = 0;
String table = null;
int uriType = sURIMatcher.match(uri);
switch (uriType) {
case RAWINPUT_TABLE:
table = RAWINPUT_TABLE_PATH;
break;
}
db.beginTransaction();
try {
for (ContentValues cv : values) {
long newID = db.insertOrThrow(table, null, cv);
if (newID <= 0) {
throw new SQLException("Failed to insert row into " + uri);
}
}
db.setTransactionSuccessful();
getContext().getContentResolver().notifyChange(uri, null);
numInserted = values.length;
} finally {
db.endTransaction();
}
return numInserted;
}
If you want each float to have it's own record in your database, you need an instance of ContentValues for each new record. Right now you have one instance of ContentValues and you are writing the same key to it (meaning you are writing over the value) 700 times.
private void saveToDatabase( float[] tempValues ) {
final int count = tempValues.legnth;
ContentValues[] cvArray = new ContentValues[count];
for (int i = 0; i < count; i++) {
float tempVal = tempValues[i];
ContentValues values = new ContentValues();
values.put( DataTable.COLUMN_DATA, tempVal );
cvArray[i] = values;
}
/* all the rest */
}
I know that this will be rude, but just throw away this code. Providers have primary methods to deal with most SQLite operations and you tried to blend three of them (insert(), bulkInsert(), and applyBatch()) into some kind of Frankenstein. Here are the main mistakes:
1) This line values.put(DataTable.COLUMN_DATA, tempVal) is not inserting new entries at each iteration; it is overriding them. After all iterations, values contains only the 700th float value of your array.
2) As #Karakuri remembered, there is only one ContentValues instance inside cvArray. bulkInsert() doc states about its second parameter:
An array of sets of column_name/value pairs to add to the database. This must not be null.
So cvArray must contain a ContentValues instance (a set) for every entry you want to insert into the database.
3) Not exactly an error, but something you should watch out. There are no guarantees that mTables will exist, and trying to make operations without specifying a table will throw a SQLException.
4) These three lines are basically useless:
if (newId <= 0) {
throw new SQLException("Failed to insert row into " + uri);
}
insertOrThrow() already throws an exception if some error happens during the insert operation. If you want to check manually for an error, try insert() or insertWithOnConflict() (or add a catch to your try block and deal with the exception there).
5) And finally, there is the problem about numInserted #petey pointed (and there's no need to repeat).
One last advice: forget that bulkInsert() exists. I know that this will require more lines of code, but using applyBatch() you can achieve better results (and more easily, since you do not have to implement it). Wolfram Rittmeyer wrote a series of excellent articles about transactions, check if you have any doubt.
Last but not least (yes, I'm in a good mood today), this is how I would do a basic implementation of your code:
#Override
public Uri insert(Uri uri, ContentValues values) {
final SQLiteDatabase db // TODO: retrieve writable database
final int match = matcher.match(uri);
switch(match) {
case RAWINPUT_TABLE:
long id = db.insert(RAWINPUT_TABLE, null, values); // TODO: add catch block to deal.
getContext().getContentResolver().notifyChange(uri, null, false);
return ContentUris.withAppendedId(uri, id);
default:
throw new UnsupportedOperationException("Unknown uri: " + uri);
}
}
private void saveToDatabase( float[] tempValues ) {
ArrayList<ContentProviderOperation> operations = new ArrayList<ContentProviderOperation>();
for (float tempVal : tempValues){
operations.add(ContentProviderOperation
.newInsert(HealthDevContentProvider.CONTENT_URI_DATA)
.withValue(DataTable.COLUMN_DATA, tempVal).build();
.withValue() // TODO: add patient id
.withValue() // TODO: add sensor type);
}
// WARNING!! Provider operations (except query if you are using loaders) happen by default in the main thread!!
getContentResolver().applyBatch(operations);
}
I use batch inserts, not sure what the difference between bulk and batch is but all I do is this
ArrayList<ContentProviderOperation> operations = new ArrayList<ContentProviderOperation>();
for(int j=0;j<locationAry.length;j++){
ContentValues values2 = new ContentValues();
values2.put(MapPoints.ELEMENT_ECM2ID, ecm2id);
values2.put(MapPoints.ELEMENT_ID, newElementId);
values2.put(MapPoints.LATITUDE, locationAry[j+1]);
values2.put(MapPoints.LONGITUDE, locationAry[j]);
values2.put(MapPoints.LAYER_ID, layerID);
operations2.add(ContentProviderOperation.newInsert(MapPoints.CONTENT_URI).withValues(values2).build());
}
getContentResolver().applyBatch(MapElements.AUTHORITY, operations);
did you override the bulkInsert method in your ContentProvider?
If one insert fails, your whole transaction fails. Without seeing your table create statement for unique keys, try a replace after your insert fails.. Also your numInserted will always be the same as values.length no matter what insert/replace fails. this doesnt seem correct either.
...
db.beginTransaction();
int numInserted = 0;
try {
for (ContentValues cv : values) {
long newID;
try {
newID = database.insertOrThrow(table, null, cv);
} catch (SQLException ignore) {
newID = database.replace(table, null, cv);
}
if (newID <= 0) {
Log.e("TAG, "Failed to insert or replace row into " + uri);
} else {
// you are good...increment numInserted
numInserted++;
}
}
db.setTransactionSuccessful();
getContext().getContentResolver().notifyChange(uri, null);
} finally {
db.endTransaction();
}
return numInserted;
i am working on sqlite insertion using contentvalues with transaction . The following code does not generate any exception however the data is not inserted.
Did i miss somethings ? Thanks.
public boolean addRecord(Rec rec) {
SQLiteDatabase db = this.getWritableDatabase();
db.beginTransaction();
ContentValues values = new ContentValues();
values.put(KEY_ID, rec.get());
// Inserting Row
try {
db.insertOrThrow(TABLE_RECORDS, null, values);
} catch (SQLException e) {
e.printStackTrace();
}
db.endTransaction();
db.close();
return true;
}
After calling beginTransaction, you must call setTransactionSuccessful to ensure that the transaction gets committed. Without that call, any changes in the transaction are rolled back.
db.beginTransaction();
try {
...
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
This particular construction ensures that any exception in the actual database code will result in an rollback.
(If you get an exception, it is a bad idea to just swallow it; the entire function must fail.)
I want to insert multiple sms in "content://sms/" table. let's say 500 sms.
my code is :
ContentValues [500] valuesarray = new ContentValues[];
for(int i=0;i<values.size();i++){
valuesarray[i] = values.get(i);
}
getContentResolver().bulkInsert(Uri.parse("content://sms/"), valuesarray);
It works, but it is extremely slow, and it makes no difference with insert() method. I serched on the net, and found methods like :
try {
database.beginTransaction();
for (ContentValues initialValues : allValues) {
values = initialValues == null ? new ContentValues() : new ContentValues(initialValues);
rowId = insertEvent(database, values);
if (rowId > 0)
rowsAdded++;
}
database.setTransactionSuccessful();
} catch (SQLException ex) {
} finally {
database.endTransaction();
}
But this is for personnal databases. How can I use a method like this with Android's "content://sms/" provider?
How can I use a method like this with Android's "content://sms/" provider?
You can't, sorry.
(besides, that provider is not part of the Android SDK and may not exist on all devices, anyway, so you should not be using it)