Part of my functionality requires updating a value in every row (only happens rarely, when a user selects a certain setting).
Trouble is, the query takes a good few minutes to perform (at best), and there's only 269 test records. Is there any way this could be optimized?
String allRecords = "SELECT id, weight FROM Workout_Entry";
Cursor cursor = db.rawQuery(allRecords, null);
int rows = cursor.getCount();
int id;
double weight;
try
{
if (cursor.moveToFirst())
{
do
{
for (int i = 0; i < rows; i++)
{
id = cursor.getInt(0);
weight = cursor.getInt(1) / 2.2;
String strFilter = "id = " + id;
ContentValues args = new ContentValues();
args.put("weight", weight);
db.update("Workout_Entry", args, strFilter, null);
}
} while (cursor.moveToNext());
}
} finally
{
cursor.close();
}
(db).close();
Thanks!
Just push the work to the database engine instead of pulling the data out one row at a time and firing up a new update query each time. Replace your code with something like:
db.execSQL("UPDATE Workout_Entry SET weight=weight/2.2");
Also, since this seems to be some kind of metric/imperial unit conversion, consider keeping the data in just one format in the database and convert/format to the appropriate unit for display purposes.
You should learn to use transactions - you can see example of how you use that in this presentation of mine.
Also showing the impact of not using the transaction.
Ok, I've realised my error. I had a do while loop for every record, and inside of that was a for loop which looped through x many times, where x was the amount of existing records.
So it was going through O(n)² times instead of O(n).
Thanks to those who replied! I found both of your comments useful.
Related
I am new to android and maybe its a silly question but i am not getting it. See i am designing a game in which we give scores to some persons. So i want to store the names of the persons in a database while installation and then their scores set to 0 initially which will be updated according to what the users select. Here i am not able to figure out that how should i enter the data as it will be around 100 names and their scores. Using INSERT INTO() statement will make it like 100 statements. So is there any short method like can we do it through strings or something. Just guessing though. Any help would be appreciated.
You don't hard-code names or scores into your SQL statements. Instead, you use parameters.
var command = new SQLiteCommand()
command.CommandText = "INSERT INTO Scores (name, score) VALUES(#name, #score)";
command.CommandType = CommandType.Text;
foreach (var item in data)
{
command.Parameters.Add(new SQLiteParameter("#name", item.Name));
command.Parameters.Add(new SQLiteParameter("#score", item.Score));
command.ExecuteNonQuery();
}
and then just loop through all of the names and scores.
I recommend you using a transaction.
You can archive this stating you want to use a transaction with beginTransaction(), do all the inserts on makeAllInserts() with a loop and if everything works then call setTransactionSuccessful() to do it in a batch operation. If something goes wrong, on the finally section you will call endTransaction() without setting the success, this will execute a rollback.
db.beginTransaction();
try {
makeAllInserts();
db.setTransactionSuccessful();
}catch {
//Error in between database transaction
}finally {
db.endTransaction();
}
For the makeAllInserts function, something like this could work out:
public void makeAllInserts() {
for(int i = 0; i < myData.size(); i++) {
myDataBase = openDatabase();
ContentValues values = new ContentValues();
values.put("name", myData.get(i).getName());
values.put("score", myData.get(i).getScore());
myDataBase.insert("MYTABLE", nullColumnHack, values);
}
}
If you also want to know about the nullColumnHack here you have a good link -> https://stackoverflow.com/a/2663620/709671
Hope it helps.
I have seen a lot of posts on optimizing SQLITE on android with bulk inserts
Currently its taking 90 seconds to do 900 inserts/updates. I added the Begin/End Transaction around them but only saw minor improvements.
So I want to add I believe SQLiteStatement
Here is my code currently
static ArrayList<ContentSystem> csList = new ArrayList<ContentSystem>();
..fill out csList..
_dh.BeginTransaction(Table);
for(int i = 0; i < csList.size(); ++i)
{
addItem(ma, csList.get(i), dh, Table, Key);
}
_dh.setTransactionSuccessful();
_dh.EndTransaction(Table);
public static void addItem(final MainActivity ma, ContentSystem cs,
final DatabaseHandler dh, String Table, String Key)
{
worked = dh.UpdateItem(cs.cv, Table, Key, cs.UUID);
if (worked == 0)
{
worked = dh.AddData(Table, cs.cv);
if (worked <= 0)
{
ErrorLogger.AddError("Error")
}
}
}
My problem is that if my csList contains ~1000 items and some are already in my list, some are not
so I am currently doing a update if the update returns 0 then I do an add
How could I get something like this to work in a bulk statement?
A bit more info
dh.Update
int wok = db.update(table, values, Key + " = ?", new String[] { Item });
dh.Add
int work = (int) db.insert(table, null, values);
ContentSystem is a list of ContentValues
Try INSERT OR REPLACE, instead of either just an update or a failed update followed by an insert.
Are you sure the instantiated SQLiteDatabase object in your DatabaseHandler class is the same as _dh?
You might have started a transaction for _dh, but that might not even be the instantiated object that is actually doing any work.
Hello I am pretty new with SQLite and I am trying to deal with some database manipulation in my project.
I have a table with almost 4000 rows and this is the format of every row:
problem_id (string)
problem_no (string)
problem_title (string)
dacu (int)
I need to query a bunch of problem_no based on the problem_id. The quantity of query is almost 1000 at a time. So I wrote a query code like this:
Set<Integer> getProblemsTitle(HashSet<String> problemsIDs) {
SQLiteDatabase db = this.getReadableDatabase();
HashSet<Integer> problemNo = new HashSet<Integer>();
Cursor cursor = null;
for (Iterator<String> iterator = problemsIDs.iterator(); iterator.hasNext();) {
cursor = db.query(CommonUtils.PROBLEM_TABLE, new String[] {
CommonUtils.KEY_PROBLEM_NO },
CommonUtils.KEY_PROBLEM_ID + "=?",
new String[] { iterator.next() }, null, null, null, null);
if (cursor != null && cursor.moveToFirst()) {
problemNo.add(cursor.getInt(0));
}
cursor.close();
}
db.close();
Set<Integer> set = new TreeSet<Integer>(problemNo);
return set;
}
I know this is not a optimized snippet. And I need to optimize it a lot to reduce the execution time of the query. I did it inside AsyncTask but it is taking too much time.
How can I do this efficiently with faster performance?
You might want to consider taking this out of the database. If you just grabbed all the problems, you could add them all in code. Running one SELECT with 4000 results is still going to be much faster than a thousand SELECT statements.
The approach would be to grab them all, but sorted(ORDER BY problem_id). You could then just check each item in problemIDs against it, and add when you get a match.
You could also use the IN operator as Mathew suggests, but I don't know how efficient that will be with 1000 items in the set.
Don't iterate over a collection of IDs, but use the IN operator in a WHERE condition.
SELECT * FROM Table WHERE problem_id IN (1,2,3,4,5)
This will return all the records in the set. Whereas you are querying them one at a time.
You could try compiling a query, and maybe you can try to load the database into memory before reading.
Create an index on the problem_id column.
I am testing the capabilities of the device -- to show the customer the size of data that can be stored inside the device, how fast it can be retrieved, how fast the search works, etc.
I am using my content provider to access the product database table with few columns. I have already moved the code to the content provider to avoid the extra communication when inserting the test records. The following code is called via menu from an activity to fill the table with the test content
Uri uri = Uri.parse(DemoContentProvider.PRODUCTS_CONTENT_URI + "/insertdemo");
getContentResolver().insert(uri, null);
The URI is recognized in the .insert method of the content provider and the following private method (of the same content provider) is called to fill the table (notice the 100 thousands of items):
private void insertDemoProducts() {
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
}
The inner insertProduct() looks like that:
private void insertProduct(String code, String name, String stock,
String price, String listprice) {
SQLiteDatabase sqlDB = database.getWritableDatabase();
ContentValues values = new ContentValues();
values.put(ProductTable.COLUMN_CODE, code);
values.put(ProductTable.COLUMN_NAME, name);
values.put(ProductTable.COLUMN_STOCK, stock);
values.put(ProductTable.COLUMN_PRICE, price);
values.put(ProductTable.COLUMN_LISTPRICE, listprice);
sqlDB.insert(ProductTable.TABLE_PRODUCT, null, values);
}
It works, but it takes "forever". How can I make it faster? What is the fastest method you know to fill the table?
Just some numbers to consider: 1000 items takes about 20 seconds to be created.
You need to use transactions when writing to a sqlite-database, otherwise it will persist the data for every insert i.e save it to sd which will take "forever".
for instance, make insertProduct take a list of products and save them in one transaction:
private void insertProducts(List<Product> products) {
try {
db.beginTransaction();
for(Product product : products) {
insertProduct(...);
}
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
}
This is how you can implement it in your existing code:
private void insertDemoProducts() {
SQLiteDatabase sqlDB = database.getWritableDatabase();
try {
sqlDB.beginTransaction();
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
sqlDB.setTransactionSuccessful();
} finally {
sqlDB.endTransaction();
}
}
Anyway, I am not completely satisfied with the accepted question because I do not understand the reason why adding the transaction makes it faster.
Looking at the Android sources, I have found that the sqlDB.insert(...) calls insertWithOnConflict(...) and that one construct the string for the SQL command using the StringBuilder class (with questionmarks as placeholders for the inserted values). Only then the string is passed to the SQLiteStatement constructor together with array of the inserted values. This means that string with the SQL command is being built again and again.
Further, a string representation of an SQL command template can be precompiled thus avoiding also the repeated compilation of the command. Then .bindXxx and .execute methods can be used for inserting the wanted records into the table. When put together, I did use the followig code (iside the outer transaction as Dean suggested):
StringBuilder sql = new StringBuilder();
sql.append("INSERT INTO ");
sql.append(ProductTable.TABLENAME);
sql.append("(");
sql.append(ProductTable.COLUMN_CODE);
sql.append(",");
sql.append(ProductTable.COLUMN_NAME);
sql.append(",");
sql.append(ProductTable.COLUMN_STOCK);
sql.append(",");
sql.append(ProductTable.COLUMN_PRICE);
sql.append(",");
sql.append(ProductTable.COLUMN_LISTPRICE);
sql.append(") VALUES (?, ?, 100, 75.50, 70.27)");
SQLiteStatement insert = sqldb.compileStatement(sql.toString());
for (int i = 1; i <= 300000; ++i) {
insert.bindLong(1, i);
insert.bindString(2, "Test product " + i);
insert.execute();
}
When compared with adding the transaction only, the result is about 3-times faster. The 300 thousands records were inserted in about 3 minutes and 15 seconds on Nexus 7.
So I'm still building a Database to support a project of mine. There are two different things to be saved: first, attribute values of some player objects and second, simple values stored in a java class.
ATM my problem lies in the process of loading values of a player object and writing it in the respective class.
Now let's see some code:
Following you see the method I want to use for saving the values in the database.
That works fine atm, but I just realized I'm still passing the contentValues object an extra value for the 'ID' , which I did set - and planned to keep that way - as autoincrement.
Any Idea how to work this in accordingly?
public void savePlayer(Player player[]) {
for (int i = 0; i <= 3; i++) {
playerValues.put("ID", i);
playerValues.put("Name", player[i].getName());
playerValues.put("HP", player[i].getHp());
playerValues.put("Satisfaction", player[i].getsatisfaction());
playerValues.put("Hygiene", player[i].isHygieneInt());
playerValues.put("IsAlive", player[i].isAliveInt());
}
db.insert("playertable", null, playerValues);
}
Okay, hold on to your hats because this might look a bit like spaghetti - the load-method:
public void loadPlayer() {
String[] namecolumn = { "Name" };
String[] intcolumn = { "ID, HP, Satisfaction, Hygiene, IsAlive" };
String[] namesToString = new String[4];
for (int j = 0; j <= 3; j++) {
Cursor playerCursorName = db.query("playertable", namecolumn, "ID="
+ j, null, null, null, null);
namesToString = cursorToString(playerCursorName);
Resource.playerArray[j].setName(namesToString[j]);
}
for (int i = 0; i < 3; i++) {
int[] restToInt;
Cursor playerCursorInt = db.query("playertable", intcolumn, "ID="
+ i, null, null, null, null);
restToInt = cursorToInt(playerCursorInt, 4);
Resource.playerArray[i].setHp(restToInt[i]);
Resource.playerArray[i].setsatisfaction(restToInt[i]);
Resource.playerArray[i].setHygieneInt(restToInt[i]);
Resource.playerArray[i].setAliveInt(restToInt[i]);
}
}
Yeah, I know this looks pretty ugly but let me explain it:
Because there are 4 player objects I planned on iterating through the database entries by using the ID as identifier to get exactly one row at a time and writing the name and the other values of this object in the java class where I want to manage them within my project.
Note: same problem with autoincrement here than in the save method
In addition, I get a CursorIndexOutOfBoundsException when calling loadPlayer because
Index -1 is being requested - isn't that the result of an operation on the database resulting in an error?
Yeah that's pretty much it, I'll provide you with additional code if requested, hope someone can help me
You are using Cursors in a slightly odd way here.
The point of a Cursor is to ask SQLite to do the hard work of fetching data for you, and your job is simply to use the cursor to iterate through the returned values.
Firstly, I would change the query here to ask for all values in the table (and perhaps put some condition to constrain what you get back), to make sure your cursor then contains all your values.
Then, I would loop through the cursor's values by using a while loop, (with cursor.moveToPosition(-1) before the loop) moving along the cursor by using cursor.moveToNext().
See the API for more information:
http://developer.android.com/reference/android/database/Cursor.html
With regard to the autoincrement problem, as far as I can remember you can leave out the ID and use db.insert() without that field and the database will provide an ID for you.
You shouldn't have the same issue in your load method because it doesn't make sense to autoincrement when loading, you just get back what's in the database.