I am new to android and maybe its a silly question but i am not getting it. See i am designing a game in which we give scores to some persons. So i want to store the names of the persons in a database while installation and then their scores set to 0 initially which will be updated according to what the users select. Here i am not able to figure out that how should i enter the data as it will be around 100 names and their scores. Using INSERT INTO() statement will make it like 100 statements. So is there any short method like can we do it through strings or something. Just guessing though. Any help would be appreciated.
You don't hard-code names or scores into your SQL statements. Instead, you use parameters.
var command = new SQLiteCommand()
command.CommandText = "INSERT INTO Scores (name, score) VALUES(#name, #score)";
command.CommandType = CommandType.Text;
foreach (var item in data)
{
command.Parameters.Add(new SQLiteParameter("#name", item.Name));
command.Parameters.Add(new SQLiteParameter("#score", item.Score));
command.ExecuteNonQuery();
}
and then just loop through all of the names and scores.
I recommend you using a transaction.
You can archive this stating you want to use a transaction with beginTransaction(), do all the inserts on makeAllInserts() with a loop and if everything works then call setTransactionSuccessful() to do it in a batch operation. If something goes wrong, on the finally section you will call endTransaction() without setting the success, this will execute a rollback.
db.beginTransaction();
try {
makeAllInserts();
db.setTransactionSuccessful();
}catch {
//Error in between database transaction
}finally {
db.endTransaction();
}
For the makeAllInserts function, something like this could work out:
public void makeAllInserts() {
for(int i = 0; i < myData.size(); i++) {
myDataBase = openDatabase();
ContentValues values = new ContentValues();
values.put("name", myData.get(i).getName());
values.put("score", myData.get(i).getScore());
myDataBase.insert("MYTABLE", nullColumnHack, values);
}
}
If you also want to know about the nullColumnHack here you have a good link -> https://stackoverflow.com/a/2663620/709671
Hope it helps.
Related
This is my first Application with database, I hope that someone can help me to understand this problem. I have this insert method:
public long insertData(String name, int password){
....
contentValues.put(KEY_NAME, name);
contentValues.put(KEY_PASSWORD, password);
return db.insert(DBHelper.TABle_NAME, null, contentValues);
}
I can insert few data with this method, but what about if I have thousands of rows? how can I insert all these data into database? where can I write all these data, in extra class or what?
As others have said, you'll need to do some sort of iteration.
Efficiency can be gained by performing a bulk transaction. Here's an example:
public int bulkInsert(#NonNull ContentValues[] values) {
int insertCount = 0;
SQLiteDatabase db = mSqlHelper.getWritableDatabase();
try {
db.beginTransaction();
for (ContentValues value : values) {
if (db.insertOrThrow(tableName, null, value) == -1) {
throw new Exception("Unknown error while inserting entry in database.");
}
insertCount++;
}
db.setTransactionSuccessful();
} catch (Exception e) {
Log.e(LOG_TAG, "An error occurred while bulk-inserting database entries.\n" + e.getMessage(), e);
} finally {
db.endTransaction();
}
return insertCount;
}
There is no 'bulk load' facility that I'm aware of.
You'd just have to spin through the list, and insert the items.
You might want to think about why you're potentially trying to insert thousands of items into a database on a hardware-limited device like a phone or a tablet.
Might it be better to put the data on a server, and create an API that you can use to load data (for display) by pages?
you can do it the same way, that you do with few data, you only need to catch the thousands rows to insert into your database using your method, you can use asyntask, or a service to do that
You can use the same method to insert any amount of records, whether it's 1 or 1,000. Use a loop to call your insert method and add your records to your database. Consider putting your database executions in an AsyncTask to prevent your UI thread from hanging.
Your data can come from anywhere, as long as it's formatted to fit your function parameters String, int
Hello I am pretty new with SQLite and I am trying to deal with some database manipulation in my project.
I have a table with almost 4000 rows and this is the format of every row:
problem_id (string)
problem_no (string)
problem_title (string)
dacu (int)
I need to query a bunch of problem_no based on the problem_id. The quantity of query is almost 1000 at a time. So I wrote a query code like this:
Set<Integer> getProblemsTitle(HashSet<String> problemsIDs) {
SQLiteDatabase db = this.getReadableDatabase();
HashSet<Integer> problemNo = new HashSet<Integer>();
Cursor cursor = null;
for (Iterator<String> iterator = problemsIDs.iterator(); iterator.hasNext();) {
cursor = db.query(CommonUtils.PROBLEM_TABLE, new String[] {
CommonUtils.KEY_PROBLEM_NO },
CommonUtils.KEY_PROBLEM_ID + "=?",
new String[] { iterator.next() }, null, null, null, null);
if (cursor != null && cursor.moveToFirst()) {
problemNo.add(cursor.getInt(0));
}
cursor.close();
}
db.close();
Set<Integer> set = new TreeSet<Integer>(problemNo);
return set;
}
I know this is not a optimized snippet. And I need to optimize it a lot to reduce the execution time of the query. I did it inside AsyncTask but it is taking too much time.
How can I do this efficiently with faster performance?
You might want to consider taking this out of the database. If you just grabbed all the problems, you could add them all in code. Running one SELECT with 4000 results is still going to be much faster than a thousand SELECT statements.
The approach would be to grab them all, but sorted(ORDER BY problem_id). You could then just check each item in problemIDs against it, and add when you get a match.
You could also use the IN operator as Mathew suggests, but I don't know how efficient that will be with 1000 items in the set.
Don't iterate over a collection of IDs, but use the IN operator in a WHERE condition.
SELECT * FROM Table WHERE problem_id IN (1,2,3,4,5)
This will return all the records in the set. Whereas you are querying them one at a time.
You could try compiling a query, and maybe you can try to load the database into memory before reading.
Create an index on the problem_id column.
I am testing the capabilities of the device -- to show the customer the size of data that can be stored inside the device, how fast it can be retrieved, how fast the search works, etc.
I am using my content provider to access the product database table with few columns. I have already moved the code to the content provider to avoid the extra communication when inserting the test records. The following code is called via menu from an activity to fill the table with the test content
Uri uri = Uri.parse(DemoContentProvider.PRODUCTS_CONTENT_URI + "/insertdemo");
getContentResolver().insert(uri, null);
The URI is recognized in the .insert method of the content provider and the following private method (of the same content provider) is called to fill the table (notice the 100 thousands of items):
private void insertDemoProducts() {
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
}
The inner insertProduct() looks like that:
private void insertProduct(String code, String name, String stock,
String price, String listprice) {
SQLiteDatabase sqlDB = database.getWritableDatabase();
ContentValues values = new ContentValues();
values.put(ProductTable.COLUMN_CODE, code);
values.put(ProductTable.COLUMN_NAME, name);
values.put(ProductTable.COLUMN_STOCK, stock);
values.put(ProductTable.COLUMN_PRICE, price);
values.put(ProductTable.COLUMN_LISTPRICE, listprice);
sqlDB.insert(ProductTable.TABLE_PRODUCT, null, values);
}
It works, but it takes "forever". How can I make it faster? What is the fastest method you know to fill the table?
Just some numbers to consider: 1000 items takes about 20 seconds to be created.
You need to use transactions when writing to a sqlite-database, otherwise it will persist the data for every insert i.e save it to sd which will take "forever".
for instance, make insertProduct take a list of products and save them in one transaction:
private void insertProducts(List<Product> products) {
try {
db.beginTransaction();
for(Product product : products) {
insertProduct(...);
}
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
}
This is how you can implement it in your existing code:
private void insertDemoProducts() {
SQLiteDatabase sqlDB = database.getWritableDatabase();
try {
sqlDB.beginTransaction();
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
sqlDB.setTransactionSuccessful();
} finally {
sqlDB.endTransaction();
}
}
Anyway, I am not completely satisfied with the accepted question because I do not understand the reason why adding the transaction makes it faster.
Looking at the Android sources, I have found that the sqlDB.insert(...) calls insertWithOnConflict(...) and that one construct the string for the SQL command using the StringBuilder class (with questionmarks as placeholders for the inserted values). Only then the string is passed to the SQLiteStatement constructor together with array of the inserted values. This means that string with the SQL command is being built again and again.
Further, a string representation of an SQL command template can be precompiled thus avoiding also the repeated compilation of the command. Then .bindXxx and .execute methods can be used for inserting the wanted records into the table. When put together, I did use the followig code (iside the outer transaction as Dean suggested):
StringBuilder sql = new StringBuilder();
sql.append("INSERT INTO ");
sql.append(ProductTable.TABLENAME);
sql.append("(");
sql.append(ProductTable.COLUMN_CODE);
sql.append(",");
sql.append(ProductTable.COLUMN_NAME);
sql.append(",");
sql.append(ProductTable.COLUMN_STOCK);
sql.append(",");
sql.append(ProductTable.COLUMN_PRICE);
sql.append(",");
sql.append(ProductTable.COLUMN_LISTPRICE);
sql.append(") VALUES (?, ?, 100, 75.50, 70.27)");
SQLiteStatement insert = sqldb.compileStatement(sql.toString());
for (int i = 1; i <= 300000; ++i) {
insert.bindLong(1, i);
insert.bindString(2, "Test product " + i);
insert.execute();
}
When compared with adding the transaction only, the result is about 3-times faster. The 300 thousands records were inserted in about 3 minutes and 15 seconds on Nexus 7.
I am trying to build a sqlite database. It has 2800 entries in them. When i try to insert, it takes a minute or so and later gives the system error message. The respective codes are given below.
In the create database java file,
ContentValues cv4 = new ContentValues();
public long createVariantEntry(String varid, String makeid, String modelid, String varname, String posteddate) {
cv4.put(VARIANT_ID, varid);
cv4.put(VARIANT_MAKE_ID, makeid);
cv4.put(VARIANT_MODEL_ID, modelid);
cv4.put(VARIANT_NAME, varname);
cv4.put(VARIANT_POSTED_DATE, posteddate);
return Database.insert(VARIANT_TABLE_NAME, null, cv4);
}
In the mainActivity,
for(int i = 0; i<build_emp_database.size();i++)
{
md.createVariantEntry(build_emp_database.get(i).get(0), build_emp_database.get(i).get(1), build_emp_database.get(i).get(2), build_emp_database.get(i).get(3), build_emp_database.get(i).get(4));
}
Also, just for 2800 entries, it is taking more than a minute, is there any way to speed them up?? I have several small databases, and have loaded them successfully. This is the only big database and its creating an issue while inserting. Please help where am i going wrong.
Don't try to run the insert operation on the UI thread, for starters.
I suggest you investigate using a content provider as a wrapper around your database. The ContentResolver object provides methods for doing operations in batch, and is in general a more robust way of working with databases. Use an IntentService to run the insert operation in the background.
If you do a lot insert operation, you need use the ContentProviderOperation to optimize your db operation. like these:
ArrayList<ContentProviderOperation> ops = new ArrayList<ContentProviderOperation>();
for(int i = 0; i<build_emp_database.size();i++) {
ContentProviderOperation.Builder builder =
ContentProviderOperation.newInsert(yourUrl);
builder.withValue(VARIANT_ID, varid)
.withValue(VARIANT_MAKE_ID, makeid)
...
ops.add(builder.build());
}
yourContentResolver.applyBatch(yourauthority, ops);
I am trying to insert around 2800 records into the sqlite database, it is taking 150 sec, which is way too much! Could anyone please tell how to optimize this insertion.
public void createVariantEntry(ArrayList<ArrayList<String>> str) {
InsertHelper ih = new InsertHelper(Database, VARIANT_TABLE_NAME);
final int varid = ih.getColumnIndex(VARIANT_ID);
final int varmakeid = ih.getColumnIndex(VARIANT_MAKE_ID);
final int varmodid = ih.getColumnIndex(VARIANT_MODEL_ID);
final int varname = ih.getColumnIndex(VARIANT_NAME);
final int varposteddate = ih.getColumnIndex(VARIANT_POSTED_DATE);
for(int i=0;i<1253;i++)
{
ih.prepareForInsert();
ih.bind(varid, str.get(i).get(0));
ih.bind(varmakeid, str.get(i).get(1));
ih.bind(varmodid, str.get(i).get(2));
ih.bind(varname, str.get(i).get(3));
ih.bind(varposteddate, str.get(i).get(4));
ih.execute();
}
for(int i=1255;i<str.size();i++)
{
ih.prepareForInsert();
ih.bind(varid, str.get(i).get(0));
ih.bind(varmakeid, str.get(i).get(1));
ih.bind(varmodid, str.get(i).get(2));
ih.bind(varname, str.get(i).get(3));
ih.bind(varposteddate, str.get(i).get(4));
ih.execute();
}
ih.close();
}
a great boost in performance will be gained when using transactions.
try {
SQLiteDatabase db = MySQLiteOpenHelper.getWritableDatabse();
ContentValues values = new ContentValues();
db.beginTransaction();
while ( more_data_to_insert ) {
// put the data in 'values'
values.put("col_1", data_1);
values.put("col_2", data_2);
// ...
values.put("col_n", data_n);
// Insert the row into the database.
db.insert("table_name", null, values);
}
db.setTransactionSuccessful();
} catch ( SQLiteException e ) {
// handle your sqlite errors
} finally {
db.endTransaction();
}
and don't use InsertHelper. its deprecated now.
Here are some general tips that might help you:
You can bulkInsert or applyBatch using ContentProviders to do a bunch of operations in one go:
How to use bulkInsert() function in android?
You can use transactions to speed things up as well:
Android Database Transaction
In some cases DatabaseUtils.InsertHelper has been known to provide faster inserts than the normal sqlite insert:
http://www.outofwhatbox.com/blog/2010/12/android-using-databaseutils-inserthelper-for-faster-insertions-into-sqlite-database/
After this, You'll have to do some benchmarking and optimize for your specific situation analyzing performance vs data integrity tradeoffs etc. Good luck.