Develop Reference

Android SQL query not recognizing first entry

Buckle up folks, this is a weird one. I'm currently working on an android app that involves storing and retrieving data in an sqlite database. I was going through the app and testing some of the basic features to make sure everything worked, and lo and behold I found a bug in retrieving data from my database. When a user inputs their very first entry to the app, everything works as expected, the values get processed and stored. However, when I go back and attempt to access that data using SELECT * FROM history; I get a cursor that returns true when I call data.moveToNext(), yet when I loop through it using while(data.moveToNext()) { //get values and add to a List<> } the while loop never gets executed.
I've looked at the contents of the database after moving the file to my computer and opening the database using this db browser and I can see my entry.
Here's the method that I call to get all the points from my database:
List<PointValue> getAllPoints() {
List<PointValue> points;
Cursor data = rawQuery("SELECT * FROM history");
if (data.moveToNext()) {
points = new ArrayList<>();
while (data.moveToNext()) {
System.out.println("Looped");
long timestamp = data.getLong(data.getColumnIndexOrThrow("timestamp"));
int level = data.getInt(data.getColumnIndexOrThrow("level"));
points.add(new PointValue(timestamp, level));
}
} else {
return null;
}
data.close();
if (points.size() == 0) {
return null;
}
return points;
}
The rawQuery method looks like this:
private Cursor rawQuery(String sql) {
SQLiteDatabase db = this.getReadableDatabase();
return db.rawQuery(sql, null);
}
When I tried debugging this on my own, the size of points is 0 even though I know that there's at least one point in the database. Thoughts? The class containing all of my sql related stuff extends SQLiteOpenHelper
EDIT:
Here's the solution suggested by #Isaac Payne (still doesn't work):
public List<PointValue> getAllPoints() {
List<PointValue> points = new ArrayList<>();
Cursor data = rawQuery("SELECT * FROM history");
while (data.moveToNext()) {
long timestamp = data.getLong(data.getColumnIndexOrThrow("timestamp"));
int level = data.getInt(data.getColumnIndexOrThrow("level"));
points.add(new PointValue(timestamp, level));
}
data.close();
if (points.size() == 0) {
return null;
}
return points;
}

The issue is that when you call data.moveToNext() in the if statement you are moving to the first entry, then you call moveToNext() again in your while loop moving to the second non-existent entry. Try removing the if statement

Add data.moveToFirst() before if loop.
Cursor data = rawQuery("SELECT * FROM history");
//add this line
data.moveToFirst();
if (data.moveToNext()) {

SQLite update query optimisation

Part of my functionality requires updating a value in every row (only happens rarely, when a user selects a certain setting).
Trouble is, the query takes a good few minutes to perform (at best), and there's only 269 test records. Is there any way this could be optimized?
String allRecords = "SELECT id, weight FROM Workout_Entry";
Cursor cursor = db.rawQuery(allRecords, null);
int rows = cursor.getCount();
int id;
double weight;
try
{
if (cursor.moveToFirst())
{
do
{
for (int i = 0; i < rows; i++)
{
id = cursor.getInt(0);
weight = cursor.getInt(1) / 2.2;
String strFilter = "id = " + id;
ContentValues args = new ContentValues();
args.put("weight", weight);
db.update("Workout_Entry", args, strFilter, null);
}
} while (cursor.moveToNext());
}
} finally
{
cursor.close();
}
(db).close();
Thanks!

Just push the work to the database engine instead of pulling the data out one row at a time and firing up a new update query each time. Replace your code with something like:
db.execSQL("UPDATE Workout_Entry SET weight=weight/2.2");
Also, since this seems to be some kind of metric/imperial unit conversion, consider keeping the data in just one format in the database and convert/format to the appropriate unit for display purposes.

You should learn to use transactions - you can see example of how you use that in this presentation of mine.
Also showing the impact of not using the transaction.

Ok, I've realised my error. I had a do while loop for every record, and inside of that was a for loop which looped through x many times, where x was the amount of existing records.
So it was going through O(n)² times instead of O(n).
Thanks to those who replied! I found both of your comments useful.

Android SQLite: How to generate big table for testing purpose?

I am testing the capabilities of the device -- to show the customer the size of data that can be stored inside the device, how fast it can be retrieved, how fast the search works, etc.
I am using my content provider to access the product database table with few columns. I have already moved the code to the content provider to avoid the extra communication when inserting the test records. The following code is called via menu from an activity to fill the table with the test content
Uri uri = Uri.parse(DemoContentProvider.PRODUCTS_CONTENT_URI + "/insertdemo");
getContentResolver().insert(uri, null);
The URI is recognized in the .insert method of the content provider and the following private method (of the same content provider) is called to fill the table (notice the 100 thousands of items):
private void insertDemoProducts() {
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
}
The inner insertProduct() looks like that:
private void insertProduct(String code, String name, String stock,
String price, String listprice) {
SQLiteDatabase sqlDB = database.getWritableDatabase();
ContentValues values = new ContentValues();
values.put(ProductTable.COLUMN_CODE, code);
values.put(ProductTable.COLUMN_NAME, name);
values.put(ProductTable.COLUMN_STOCK, stock);
values.put(ProductTable.COLUMN_PRICE, price);
values.put(ProductTable.COLUMN_LISTPRICE, listprice);
sqlDB.insert(ProductTable.TABLE_PRODUCT, null, values);
}
It works, but it takes "forever". How can I make it faster? What is the fastest method you know to fill the table?
Just some numbers to consider: 1000 items takes about 20 seconds to be created.

You need to use transactions when writing to a sqlite-database, otherwise it will persist the data for every insert i.e save it to sd which will take "forever".
for instance, make insertProduct take a list of products and save them in one transaction:
private void insertProducts(List<Product> products) {
try {
db.beginTransaction();
for(Product product : products) {
insertProduct(...);
}
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
}
This is how you can implement it in your existing code:
private void insertDemoProducts() {
SQLiteDatabase sqlDB = database.getWritableDatabase();
try {
sqlDB.beginTransaction();
for (int i = 1; i <= 100000; ++i) {
String id = Integer.toString(i);
insertProduct(id, "Test product " + id, "100", "75.50", "70.27");
}
sqlDB.setTransactionSuccessful();
} finally {
sqlDB.endTransaction();
}
}

Anyway, I am not completely satisfied with the accepted question because I do not understand the reason why adding the transaction makes it faster.
Looking at the Android sources, I have found that the sqlDB.insert(...) calls insertWithOnConflict(...) and that one construct the string for the SQL command using the StringBuilder class (with questionmarks as placeholders for the inserted values). Only then the string is passed to the SQLiteStatement constructor together with array of the inserted values. This means that string with the SQL command is being built again and again.
Further, a string representation of an SQL command template can be precompiled thus avoiding also the repeated compilation of the command. Then .bindXxx and .execute methods can be used for inserting the wanted records into the table. When put together, I did use the followig code (iside the outer transaction as Dean suggested):
StringBuilder sql = new StringBuilder();
sql.append("INSERT INTO ");
sql.append(ProductTable.TABLENAME);
sql.append("(");
sql.append(ProductTable.COLUMN_CODE);
sql.append(",");
sql.append(ProductTable.COLUMN_NAME);
sql.append(",");
sql.append(ProductTable.COLUMN_STOCK);
sql.append(",");
sql.append(ProductTable.COLUMN_PRICE);
sql.append(",");
sql.append(ProductTable.COLUMN_LISTPRICE);
sql.append(") VALUES (?, ?, 100, 75.50, 70.27)");
SQLiteStatement insert = sqldb.compileStatement(sql.toString());
for (int i = 1; i <= 300000; ++i) {
insert.bindLong(1, i);
insert.bindString(2, "Test product " + i);
insert.execute();
}
When compared with adding the transaction only, the result is about 3-times faster. The 300 thousands records were inserted in about 3 minutes and 15 seconds on Nexus 7.

Database insertion taking too much time- android sqlite

I am trying to insert around 2800 records into the sqlite database, it is taking 150 sec, which is way too much! Could anyone please tell how to optimize this insertion.
public void createVariantEntry(ArrayList<ArrayList<String>> str) {
InsertHelper ih = new InsertHelper(Database, VARIANT_TABLE_NAME);
final int varid = ih.getColumnIndex(VARIANT_ID);
final int varmakeid = ih.getColumnIndex(VARIANT_MAKE_ID);
final int varmodid = ih.getColumnIndex(VARIANT_MODEL_ID);
final int varname = ih.getColumnIndex(VARIANT_NAME);
final int varposteddate = ih.getColumnIndex(VARIANT_POSTED_DATE);
for(int i=0;i<1253;i++)
{
ih.prepareForInsert();
ih.bind(varid, str.get(i).get(0));
ih.bind(varmakeid, str.get(i).get(1));
ih.bind(varmodid, str.get(i).get(2));
ih.bind(varname, str.get(i).get(3));
ih.bind(varposteddate, str.get(i).get(4));
ih.execute();
}
for(int i=1255;i<str.size();i++)
{
ih.prepareForInsert();
ih.bind(varid, str.get(i).get(0));
ih.bind(varmakeid, str.get(i).get(1));
ih.bind(varmodid, str.get(i).get(2));
ih.bind(varname, str.get(i).get(3));
ih.bind(varposteddate, str.get(i).get(4));
ih.execute();
}
ih.close();
}

a great boost in performance will be gained when using transactions.
try {
SQLiteDatabase db = MySQLiteOpenHelper.getWritableDatabse();
ContentValues values = new ContentValues();
db.beginTransaction();
while ( more_data_to_insert ) {
// put the data in 'values'
values.put("col_1", data_1);
values.put("col_2", data_2);
// ...
values.put("col_n", data_n);
// Insert the row into the database.
db.insert("table_name", null, values);
}
db.setTransactionSuccessful();
} catch ( SQLiteException e ) {
// handle your sqlite errors
} finally {
db.endTransaction();
}
and don't use InsertHelper. its deprecated now.

Here are some general tips that might help you:
You can bulkInsert or applyBatch using ContentProviders to do a bunch of operations in one go:
How to use bulkInsert() function in android?
You can use transactions to speed things up as well:
Android Database Transaction
In some cases DatabaseUtils.InsertHelper has been known to provide faster inserts than the normal sqlite insert:
http://www.outofwhatbox.com/blog/2010/12/android-using-databaseutils-inserthelper-for-faster-insertions-into-sqlite-database/
After this, You'll have to do some benchmarking and optimize for your specific situation analyzing performance vs data integrity tradeoffs etc. Good luck.

Android: loading from SQLite Database

So I'm still building a Database to support a project of mine. There are two different things to be saved: first, attribute values of some player objects and second, simple values stored in a java class.
ATM my problem lies in the process of loading values of a player object and writing it in the respective class.
Now let's see some code:
Following you see the method I want to use for saving the values in the database.
That works fine atm, but I just realized I'm still passing the contentValues object an extra value for the 'ID' , which I did set - and planned to keep that way - as autoincrement.
Any Idea how to work this in accordingly?
public void savePlayer(Player player[]) {
for (int i = 0; i <= 3; i++) {
playerValues.put("ID", i);
playerValues.put("Name", player[i].getName());
playerValues.put("HP", player[i].getHp());
playerValues.put("Satisfaction", player[i].getsatisfaction());
playerValues.put("Hygiene", player[i].isHygieneInt());
playerValues.put("IsAlive", player[i].isAliveInt());
}
db.insert("playertable", null, playerValues);
}
Okay, hold on to your hats because this might look a bit like spaghetti - the load-method:
public void loadPlayer() {
String[] namecolumn = { "Name" };
String[] intcolumn = { "ID, HP, Satisfaction, Hygiene, IsAlive" };
String[] namesToString = new String[4];
for (int j = 0; j <= 3; j++) {
Cursor playerCursorName = db.query("playertable", namecolumn, "ID="
+ j, null, null, null, null);
namesToString = cursorToString(playerCursorName);
Resource.playerArray[j].setName(namesToString[j]);
}
for (int i = 0; i < 3; i++) {
int[] restToInt;
Cursor playerCursorInt = db.query("playertable", intcolumn, "ID="
+ i, null, null, null, null);
restToInt = cursorToInt(playerCursorInt, 4);
Resource.playerArray[i].setHp(restToInt[i]);
Resource.playerArray[i].setsatisfaction(restToInt[i]);
Resource.playerArray[i].setHygieneInt(restToInt[i]);
Resource.playerArray[i].setAliveInt(restToInt[i]);
}
}
Yeah, I know this looks pretty ugly but let me explain it:
Because there are 4 player objects I planned on iterating through the database entries by using the ID as identifier to get exactly one row at a time and writing the name and the other values of this object in the java class where I want to manage them within my project.
Note: same problem with autoincrement here than in the save method
In addition, I get a CursorIndexOutOfBoundsException when calling loadPlayer because
Index -1 is being requested - isn't that the result of an operation on the database resulting in an error?
Yeah that's pretty much it, I'll provide you with additional code if requested, hope someone can help me

You are using Cursors in a slightly odd way here.
The point of a Cursor is to ask SQLite to do the hard work of fetching data for you, and your job is simply to use the cursor to iterate through the returned values.
Firstly, I would change the query here to ask for all values in the table (and perhaps put some condition to constrain what you get back), to make sure your cursor then contains all your values.
Then, I would loop through the cursor's values by using a while loop, (with cursor.moveToPosition(-1) before the loop) moving along the cursor by using cursor.moveToNext().
See the API for more information:
http://developer.android.com/reference/android/database/Cursor.html
With regard to the autoincrement problem, as far as I can remember you can leave out the ID and use db.insert() without that field and the database will provide an ID for you.
You shouldn't have the same issue in your load method because it doesn't make sense to autoincrement when loading, you just get back what's in the database.