Android insert multiple sms in "content://sms/" with contentresolver.bulkInsert() : too slow - android

I want to insert multiple sms in "content://sms/" table. let's say 500 sms.
my code is :
ContentValues [500] valuesarray = new ContentValues[];
for(int i=0;i<values.size();i++){
valuesarray[i] = values.get(i);
}
getContentResolver().bulkInsert(Uri.parse("content://sms/"), valuesarray);
It works, but it is extremely slow, and it makes no difference with insert() method. I serched on the net, and found methods like :
try {
database.beginTransaction();
for (ContentValues initialValues : allValues) {
values = initialValues == null ? new ContentValues() : new ContentValues(initialValues);
rowId = insertEvent(database, values);
if (rowId > 0)
rowsAdded++;
}
database.setTransactionSuccessful();
} catch (SQLException ex) {
} finally {
database.endTransaction();
}
But this is for personnal databases. How can I use a method like this with Android's "content://sms/" provider?

How can I use a method like this with Android's "content://sms/" provider?
You can't, sorry.
(besides, that provider is not part of the Android SDK and may not exist on all devices, anyway, so you should not be using it)

Related

Android - How can I pass data related to two tables to a the insert method of a Content Provider

I need to insert data related to an Order and its corresponding Detail.
Without a ContentProvider I would do something like this:
public boolean insertOrder(Order order, ArrayList<OrderDetail> items) {
boolean wasSuccessful = false;
ContentValues cvOrder = new ContentValues();
cvPedido.put(ORDER_CUSTOMER_ID, order.getCustomerId());
cvPedido.put(ORDER_CUSTOMER_NAME, order.getCustomerName());
String insertQuery = "INSERT INTO " + ORDER_DETAIL_TABLE
+ " VALUES (?,?)";
//...
try {
this.openWriteableDB();
SQLiteStatement statement = db.compileStatement(insertQuery);
db.beginTransaction();
long idOrder = db.insertOrThrow(ORDER_TABLE, null, cvOrder);
if (idOrder > 0) {
for (int i = 0; i < items.size(); i++) {
OrderDetail detail=items.get(i);
statement.clearBindings();
statement.bindString(1, detail.getDescription);
statement.bindDouble(2, detail.getPrice);
//...
statement.execute();
}
db.setTransactionSuccessful();
wasSuccessful = true;
}
} finally {
db.endTransaction();
this.closeDB();
}
return wasSuccessful;
}
The problem is that now I want to use a ContentProvider and I don't know what to do with this kind of cases where data about two or more tables must be passed to a single CRUD operation, knowing that a insert operation only accepts two parameters :
#Override
public Uri insert(Uri uri, ContentValues values) {
}
What do you do in a ContentProvider when you have to insert relational data in a transaction?
Thanks in advance.
You should use ContentProviderOperation. Since it's your ContentProvider you can assure that applyBatch() will execute all operations within a transaction. All standard content providers also ensure that that's the case.
See my blog post about ContentProviderOperation in general and my other post about how to use withBackReference() to access results of previous operations - which you need to access the orderId.
One important caveat: All ContentProviderOperations of one batch must use the same authority - but can use different URIs! In your case that should be no problem.
You can put all the data into a ContentValues and have a provider. You'll have to get a little creative with the order details.
Below psuedo code I create a key "DETAIL" on the fly with a integer then the item.
ContentValues values = new ContentValues();
values.put(ORDER_ID,orderid);
for (int i = 0; i < items.size(); i++) {
values.put("DETAIL" + Integer.ToString(i),items.get(i));
}
Uri uri = context.getContentResolver().insert(
ORDER_URI, values);
Then in content provider you sort it out.
#Override
public Uri insert(Uri uri, ContentValues values) {
int uriType = sURIMatcher.match(uri);
SQLiteDatabase sqlDB = database.getWritableDatabase();
long id = 0;
switch (uriType) {
case ORDER:
// trim name and description
trimNameDescriptions(values);
try {
id = sqlDB.insertOrThrow(ORDERS_TABLE,
null, values);
Integer i =0;
do (values.containsKey("DETAIL" + i.toString()){
ContentValues v = new ContentValues();
v.put("DETAIL",values.get("Detail" + i.toString()));
v.put("ORDERID",id);
//ACTUALLY CALL THE INSERT METHOD OF THE PROVIDER
insert(DETAIL_URI,v);
i+=1;
}
You can design content provider to mirror your SQLite tables, and use insertOrder code same as above.
Just use insert of content providers for each table(uri), to perform similar operations as in your insertOrder method
Another option is to define your content provider URI to take combination of your Order and items , and implement the parsing yourself in the content provider before committing to underlying data model.

Importing CSV file to android sqlite datase on phone itself [duplicate]

I need to parse a fairly large XML file (varying between about a hundred kilobytes and several hundred kilobytes), which I'm doing using Xml#parse(String, ContentHandler). I'm currently testing this with a 152KB file.
During parsing, I also insert the data in an SQLite database using calls similar to the following: getWritableDatabase().insert(TABLE_NAME, "_id", values). All of this together takes about 80 seconds for the 152KB test file (which comes down to inserting roughly 200 rows).
When I comment out all insert statements (but leave in everything else, such as creating ContentValues etc.) the same file takes only 23 seconds.
Is it normal for the database operations to have such a big overhead? Can I do anything about that?
You should do batch inserts.
Pseudocode:
db.beginTransaction();
for (entry : listOfEntries) {
db.insert(entry);
}
db.setTransactionSuccessful();
db.endTransaction();
That increased the speed of inserts in my apps extremely.
Update:
#Yuku provided a very interesting blog post: Android using inserthelper for faster insertions into sqlite database
Since the InsertHelper mentioned by Yuku and Brett is deprecated now (API level 17), it seems the right alternative recommended by Google is using SQLiteStatement.
I used the database insert method like this:
database.insert(table, null, values);
After I also experienced some serious performance issues, the following code speeded my 500 inserts up from 14.5 sec to only 270 ms, amazing!
Here is how I used SQLiteStatement:
private void insertTestData() {
String sql = "insert into producttable (name, description, price, stock_available) values (?, ?, ?, ?);";
dbHandler.getWritableDatabase();
database.beginTransaction();
SQLiteStatement stmt = database.compileStatement(sql);
for (int i = 0; i < NUMBER_OF_ROWS; i++) {
//generate some values
stmt.bindString(1, randomName);
stmt.bindString(2, randomDescription);
stmt.bindDouble(3, randomPrice);
stmt.bindLong(4, randomNumber);
long entryID = stmt.executeInsert();
stmt.clearBindings();
}
database.setTransactionSuccessful();
database.endTransaction();
dbHandler.close();
}
Compiling the sql insert statement helps speed things up. It can also require more effort to shore everything up and prevent possible injection since it's now all on your shoulders.
Another approach which can also speed things up is the under-documented android.database.DatabaseUtils.InsertHelper class. My understanding is that it actually wraps compiled insert statements. Going from non-compiled transacted inserts to compiled transacted inserts was about a 3x gain in speed (2ms per insert to .6ms per insert) for my large (200K+ entries) but simple SQLite inserts.
Sample code:
SQLiteDatabse db = getWriteableDatabase();
//use the db you would normally use for db.insert, and the "table_name"
//is the same one you would use in db.insert()
InsertHelper iHelp = new InsertHelper(db, "table_name");
//Get the indices you need to bind data to
//Similar to Cursor.getColumnIndex("col_name");
int first_index = iHelp.getColumnIndex("first");
int last_index = iHelp.getColumnIndex("last");
try
{
db.beginTransaction();
for(int i=0 ; i<num_things ; ++i)
{
//need to tell the helper you are inserting (rather than replacing)
iHelp.prepareForInsert();
//do the equivalent of ContentValues.put("field","value") here
iHelp.bind(first_index, thing_1);
iHelp.bind(last_index, thing_2);
//the db.insert() equilvalent
iHelp.execute();
}
db.setTransactionSuccessful();
}
finally
{
db.endTransaction();
}
db.close();
If the table has an index on it, consider dropping it prior to inserting the records and then adding it back after you've commited your records.
If using a ContentProvider:
#Override
public int bulkInsert(Uri uri, ContentValues[] bulkinsertvalues) {
int QueryType = sUriMatcher.match(uri);
int returnValue=0;
SQLiteDatabase db = mOpenHelper.getWritableDatabase();
switch (QueryType) {
case SOME_URI_IM_LOOKING_FOR: //replace this with your real URI
db.beginTransaction();
for (int i = 0; i < bulkinsertvalues.length; i++) {
//get an individual result from the array of ContentValues
ContentValues values = bulkinsertvalues[i];
//insert this record into the local SQLite database using a private function you create, "insertIndividualRecord" (replace with a better function name)
insertIndividualRecord(uri, values);
}
db.setTransactionSuccessful();
db.endTransaction();
break;
default:
throw new IllegalArgumentException("Unknown URI " + uri);
}
return returnValue;
}
Then the private function to perform the insert (still inside your content provider):
private Uri insertIndividualRecord(Uri uri, ContentValues values){
//see content provider documentation if this is confusing
if (sUriMatcher.match(uri) != THE_CONSTANT_IM_LOOKING_FOR) {
throw new IllegalArgumentException("Unknown URI " + uri);
}
//example validation if you have a field called "name" in your database
if (values.containsKey(YOUR_CONSTANT_FOR_NAME) == false) {
values.put(YOUR_CONSTANT_FOR_NAME, "");
}
//******add all your other validations
//**********
//time to insert records into your local SQLite database
SQLiteDatabase db = mOpenHelper.getWritableDatabase();
long rowId = db.insert(YOUR_TABLE_NAME, null, values);
if (rowId > 0) {
Uri myUri = ContentUris.withAppendedId(MY_INSERT_URI, rowId);
getContext().getContentResolver().notifyChange(myUri, null);
return myUri;
}
throw new SQLException("Failed to insert row into " + uri);
}

Bulk inserting using an array of ContentValues

im trying to do a batch insert of about 700 floats. The method i'm using is below and as well as the content provider's bulkInsert. The issue is that when i put all the floating point values into the ContentValues nothing happens. What's a better way to insert those floating point values into the ContentValues object?
private void saveToDatabase( float[] tempValues )
{
ContentValues values = new ContentValues();
// WM: TODO: add patient id and sensor type
for (float tempVal : tempValues){
values.put( DataTable.COLUMN_DATA, tempVal );
}
ContentValues[] cvArray = new ContentValues[1];
cvArray[0] = values;
ContentResolver resolver = getContentResolver();
resolver.bulkInsert( HealthDevContentProvider.CONTENT_URI_DATA, cvArray);
public int bulkInsert(Uri uri, ContentValues[] values){
int numInserted = 0;
String table = null;
int uriType = sURIMatcher.match(uri);
switch (uriType) {
case RAWINPUT_TABLE:
table = RAWINPUT_TABLE_PATH;
break;
}
db.beginTransaction();
try {
for (ContentValues cv : values) {
long newID = db.insertOrThrow(table, null, cv);
if (newID <= 0) {
throw new SQLException("Failed to insert row into " + uri);
}
}
db.setTransactionSuccessful();
getContext().getContentResolver().notifyChange(uri, null);
numInserted = values.length;
} finally {
db.endTransaction();
}
return numInserted;
}
If you want each float to have it's own record in your database, you need an instance of ContentValues for each new record. Right now you have one instance of ContentValues and you are writing the same key to it (meaning you are writing over the value) 700 times.
private void saveToDatabase( float[] tempValues ) {
final int count = tempValues.legnth;
ContentValues[] cvArray = new ContentValues[count];
for (int i = 0; i < count; i++) {
float tempVal = tempValues[i];
ContentValues values = new ContentValues();
values.put( DataTable.COLUMN_DATA, tempVal );
cvArray[i] = values;
}
/* all the rest */
}
I know that this will be rude, but just throw away this code. Providers have primary methods to deal with most SQLite operations and you tried to blend three of them (insert(), bulkInsert(), and applyBatch()) into some kind of Frankenstein. Here are the main mistakes:
1) This line values.put(DataTable.COLUMN_DATA, tempVal) is not inserting new entries at each iteration; it is overriding them. After all iterations, values contains only the 700th float value of your array.
2) As #Karakuri remembered, there is only one ContentValues instance inside cvArray. bulkInsert() doc states about its second parameter:
An array of sets of column_name/value pairs to add to the database. This must not be null.
So cvArray must contain a ContentValues instance (a set) for every entry you want to insert into the database.
3) Not exactly an error, but something you should watch out. There are no guarantees that mTables will exist, and trying to make operations without specifying a table will throw a SQLException.
4) These three lines are basically useless:
if (newId <= 0) {
throw new SQLException("Failed to insert row into " + uri);
}
insertOrThrow() already throws an exception if some error happens during the insert operation. If you want to check manually for an error, try insert() or insertWithOnConflict() (or add a catch to your try block and deal with the exception there).
5) And finally, there is the problem about numInserted #petey pointed (and there's no need to repeat).
One last advice: forget that bulkInsert() exists. I know that this will require more lines of code, but using applyBatch() you can achieve better results (and more easily, since you do not have to implement it). Wolfram Rittmeyer wrote a series of excellent articles about transactions, check if you have any doubt.
Last but not least (yes, I'm in a good mood today), this is how I would do a basic implementation of your code:
#Override
public Uri insert(Uri uri, ContentValues values) {
final SQLiteDatabase db // TODO: retrieve writable database
final int match = matcher.match(uri);
switch(match) {
case RAWINPUT_TABLE:
long id = db.insert(RAWINPUT_TABLE, null, values); // TODO: add catch block to deal.
getContext().getContentResolver().notifyChange(uri, null, false);
return ContentUris.withAppendedId(uri, id);
default:
throw new UnsupportedOperationException("Unknown uri: " + uri);
}
}
private void saveToDatabase( float[] tempValues ) {
ArrayList<ContentProviderOperation> operations = new ArrayList<ContentProviderOperation>();
for (float tempVal : tempValues){
operations.add(ContentProviderOperation
.newInsert(HealthDevContentProvider.CONTENT_URI_DATA)
.withValue(DataTable.COLUMN_DATA, tempVal).build();
.withValue() // TODO: add patient id
.withValue() // TODO: add sensor type);
}
// WARNING!! Provider operations (except query if you are using loaders) happen by default in the main thread!!
getContentResolver().applyBatch(operations);
}
I use batch inserts, not sure what the difference between bulk and batch is but all I do is this
ArrayList<ContentProviderOperation> operations = new ArrayList<ContentProviderOperation>();
for(int j=0;j<locationAry.length;j++){
ContentValues values2 = new ContentValues();
values2.put(MapPoints.ELEMENT_ECM2ID, ecm2id);
values2.put(MapPoints.ELEMENT_ID, newElementId);
values2.put(MapPoints.LATITUDE, locationAry[j+1]);
values2.put(MapPoints.LONGITUDE, locationAry[j]);
values2.put(MapPoints.LAYER_ID, layerID);
operations2.add(ContentProviderOperation.newInsert(MapPoints.CONTENT_URI).withValues(values2).build());
}
getContentResolver().applyBatch(MapElements.AUTHORITY, operations);
did you override the bulkInsert method in your ContentProvider?
If one insert fails, your whole transaction fails. Without seeing your table create statement for unique keys, try a replace after your insert fails.. Also your numInserted will always be the same as values.length no matter what insert/replace fails. this doesnt seem correct either.
...
db.beginTransaction();
int numInserted = 0;
try {
for (ContentValues cv : values) {
long newID;
try {
newID = database.insertOrThrow(table, null, cv);
} catch (SQLException ignore) {
newID = database.replace(table, null, cv);
}
if (newID <= 0) {
Log.e("TAG, "Failed to insert or replace row into " + uri);
} else {
// you are good...increment numInserted
numInserted++;
}
}
db.setTransactionSuccessful();
getContext().getContentResolver().notifyChange(uri, null);
} finally {
db.endTransaction();
}
return numInserted;

writing exception to parcel exception while updating contact name in android?

I need to update a column (cached_name) of table callog.calls in sql lite database in android.what I can't figure out is how to use update statement.I am unable to find anything regarding update command using cursor.kindly help.thanks in advance.
//number1 is the phone number against which I need to run update query in db.
//name is the string which I used to insert against number1..
String name=edittext.gettext().tostring();
ContentResolver cr = getContentResolver();
Cursor cur = cr.query(CallLog.Calls.CONTENT_URI,
null, null, null, null);
ContentValues value=new ContentValues();
value.put("CallLog.Calls.CACHED_NAME",name);
cur.moveToFirst();
while(!cur.isLast())
{
String number=cur.getString(cur.getColumnIndex(CallLog.Calls.NUMBER));
if((number.equals(number1)==true)
{
try{
cr.update(CallLog.Calls.CONTENT_URI,value,CallLog.Calls.CACHED_NAME+"=?",null);
}
catch(Exception e)
{
e.printStackTrace();
System.out.println(e.getStackTrace());
}
}//if
cur.moveToNext();
}//while
Basically, the only line you actually need is the update :
String name = edittext.getText().toString();
ContentResolver cr = getContentResolver();
ContentValues value = new ContentValues();
// Note that there is no "" here. The value is a constant
value.put(CallLog.Calls.CACHED_NAME, name);
cr.update(CallLog.Calls.CONTENT_URI, values, CallLog.Calls.NUMBER+"=?",
new String[] { number1 });
This is equivalent to this raw SQL query:
UPDATE call_log SET cached_name = name WHERE number = number1;
There is no need for iterating over all the calls, that's what the where clause is for.
Also,
while(!cur.isLast())
prevents you from actually reading the last matching value. Don't do that. Use
while (cur.moveToNext())
when you need to iterate over a cursor (which you don't, here)
I strongly suggest you take a look at how cursors work, as well as how sql works.
(Also, out of curiosity, why do you need to modify the callLog table?)

Android SQLite database: slow insertion

I need to parse a fairly large XML file (varying between about a hundred kilobytes and several hundred kilobytes), which I'm doing using Xml#parse(String, ContentHandler). I'm currently testing this with a 152KB file.
During parsing, I also insert the data in an SQLite database using calls similar to the following: getWritableDatabase().insert(TABLE_NAME, "_id", values). All of this together takes about 80 seconds for the 152KB test file (which comes down to inserting roughly 200 rows).
When I comment out all insert statements (but leave in everything else, such as creating ContentValues etc.) the same file takes only 23 seconds.
Is it normal for the database operations to have such a big overhead? Can I do anything about that?
You should do batch inserts.
Pseudocode:
db.beginTransaction();
for (entry : listOfEntries) {
db.insert(entry);
}
db.setTransactionSuccessful();
db.endTransaction();
That increased the speed of inserts in my apps extremely.
Update:
#Yuku provided a very interesting blog post: Android using inserthelper for faster insertions into sqlite database
Since the InsertHelper mentioned by Yuku and Brett is deprecated now (API level 17), it seems the right alternative recommended by Google is using SQLiteStatement.
I used the database insert method like this:
database.insert(table, null, values);
After I also experienced some serious performance issues, the following code speeded my 500 inserts up from 14.5 sec to only 270 ms, amazing!
Here is how I used SQLiteStatement:
private void insertTestData() {
String sql = "insert into producttable (name, description, price, stock_available) values (?, ?, ?, ?);";
dbHandler.getWritableDatabase();
database.beginTransaction();
SQLiteStatement stmt = database.compileStatement(sql);
for (int i = 0; i < NUMBER_OF_ROWS; i++) {
//generate some values
stmt.bindString(1, randomName);
stmt.bindString(2, randomDescription);
stmt.bindDouble(3, randomPrice);
stmt.bindLong(4, randomNumber);
long entryID = stmt.executeInsert();
stmt.clearBindings();
}
database.setTransactionSuccessful();
database.endTransaction();
dbHandler.close();
}
Compiling the sql insert statement helps speed things up. It can also require more effort to shore everything up and prevent possible injection since it's now all on your shoulders.
Another approach which can also speed things up is the under-documented android.database.DatabaseUtils.InsertHelper class. My understanding is that it actually wraps compiled insert statements. Going from non-compiled transacted inserts to compiled transacted inserts was about a 3x gain in speed (2ms per insert to .6ms per insert) for my large (200K+ entries) but simple SQLite inserts.
Sample code:
SQLiteDatabse db = getWriteableDatabase();
//use the db you would normally use for db.insert, and the "table_name"
//is the same one you would use in db.insert()
InsertHelper iHelp = new InsertHelper(db, "table_name");
//Get the indices you need to bind data to
//Similar to Cursor.getColumnIndex("col_name");
int first_index = iHelp.getColumnIndex("first");
int last_index = iHelp.getColumnIndex("last");
try
{
db.beginTransaction();
for(int i=0 ; i<num_things ; ++i)
{
//need to tell the helper you are inserting (rather than replacing)
iHelp.prepareForInsert();
//do the equivalent of ContentValues.put("field","value") here
iHelp.bind(first_index, thing_1);
iHelp.bind(last_index, thing_2);
//the db.insert() equilvalent
iHelp.execute();
}
db.setTransactionSuccessful();
}
finally
{
db.endTransaction();
}
db.close();
If the table has an index on it, consider dropping it prior to inserting the records and then adding it back after you've commited your records.
If using a ContentProvider:
#Override
public int bulkInsert(Uri uri, ContentValues[] bulkinsertvalues) {
int QueryType = sUriMatcher.match(uri);
int returnValue=0;
SQLiteDatabase db = mOpenHelper.getWritableDatabase();
switch (QueryType) {
case SOME_URI_IM_LOOKING_FOR: //replace this with your real URI
db.beginTransaction();
for (int i = 0; i < bulkinsertvalues.length; i++) {
//get an individual result from the array of ContentValues
ContentValues values = bulkinsertvalues[i];
//insert this record into the local SQLite database using a private function you create, "insertIndividualRecord" (replace with a better function name)
insertIndividualRecord(uri, values);
}
db.setTransactionSuccessful();
db.endTransaction();
break;
default:
throw new IllegalArgumentException("Unknown URI " + uri);
}
return returnValue;
}
Then the private function to perform the insert (still inside your content provider):
private Uri insertIndividualRecord(Uri uri, ContentValues values){
//see content provider documentation if this is confusing
if (sUriMatcher.match(uri) != THE_CONSTANT_IM_LOOKING_FOR) {
throw new IllegalArgumentException("Unknown URI " + uri);
}
//example validation if you have a field called "name" in your database
if (values.containsKey(YOUR_CONSTANT_FOR_NAME) == false) {
values.put(YOUR_CONSTANT_FOR_NAME, "");
}
//******add all your other validations
//**********
//time to insert records into your local SQLite database
SQLiteDatabase db = mOpenHelper.getWritableDatabase();
long rowId = db.insert(YOUR_TABLE_NAME, null, values);
if (rowId > 0) {
Uri myUri = ContentUris.withAppendedId(MY_INSERT_URI, rowId);
getContext().getContentResolver().notifyChange(myUri, null);
return myUri;
}
throw new SQLException("Failed to insert row into " + uri);
}

Categories

Resources