This is a general performance question, I have no error or something like this.
I am working on an app which uses a SQLite Database, and just want to know what is the best, fastest and most efficient way to query through a table to find a special value.
Example:
I have a table and I am searching for a special string.
I get all rows by:
Cursor cursor = db.rawQuery("SELECT * FROM " + TABLE_NAME, null);
and then iterate through the cursor like
String searchedString = "THIS_IS_JUST_AN_EXAMPLE_PLEASE_IGNORE_TYPING_ERROR";
boolean success = false;
int count = cursor.getCount();
for(int i=0;i<count;i++) {
String queryString = c.getString(1);
if(queryString.equals(searchedString) {
success=true;
break;
} else {
cursor.moveToNext();
}
}
Another possible way would be to use query():
Cursor cursor = db.query(
TABLE_NAME, new String[] {STRING_COL},
STRING_NAME + "=?",
new String[] {"THIS_IS_JUST_AN_EXAMPLE_PLEASE_IGNORE_TYPING_ERROR"},
null, null, null, null);
Is there a performance difference between these two methods?
A lot of tutorials out there are showing the iteration via loop, but the Docs recommend to use the query() method.
Is the query() method the same as iterating through a loop?
How exactly does it work? I can't find in any API.
Doing the search by hand requires more code (which increases the risk of errors).
The DB probably executes a similar loop, but doing the search in the DB does not require all of the data be moved from the DB to your application.
If there is not much data, there will not be any noticeable performance difference between the two algorithms. However, when the amount of data becomes larger, you can speed up the SQL search by simply creating an index.
The only difference I can spot is the WHERE part, which lacks in the first algorithm.
The first algorithm will benefit a lot, if you add a WHERE clause to the query. And then become identical to the second algorithm, in terms of performances.
Something like
Cursor cursor = db.rawQuery("SELECT * FROM " + TABLE_NAME + " WHERE STRING_COL = ?", new String[]{"THIS_IS_JUST_AN_EXAMPLE_PLEASE_IGNORE_TYPING_ERROR"});
As it is right now, the first algorithm is slower.
As noted bt #CL, both the algorithms can be drammatically improved by indexing the WHERE column.
Related
I want to store an image (size approx. 10MB) in the SQLite database. For that I created a DB helper, a Dao. Everything works fine, I can create several records and read them without a problem, I can even update the blob in the latest record without a problem.
But if I go back to an older record and update the blob, I cannot load this record with the blob any longer.
I have a list view where I show all the records, and for that I use a select that doesn't return the blob. This list works fine, but when I click on an item in the list, I try to load the record with the blob, the cursor returns 0 rows.
public void save(Bill aBill) {
ContentValues values = new ContentValues();
values.put(DatabaseHelper.BILL_NAME_COLUMN, aBill.getName());
values.put(DatabaseHelper.BILL_DUE_DATE_COLUMN, getContentValue(aBill.getDueDate()));
values.put(DatabaseHelper.BILL_IMAGE_COLUMN, aBill.getImage());
if (!aBill.isPersistent()) {
aBill.setId(database.insert(DatabaseHelper.BILL_TABLE, null, values));
aBill.setPersistent(true);
} else {
database.update(DatabaseHelper.BILL_TABLE, values, DatabaseHelper.BILL_ID_COLUMN + "=?", new String[]{String.valueOf(aBill.getId())});
}
}
// fails after updating the blob
public Bill get(long id) {
Cursor cursor = database.query(DatabaseHelper.BILL_TABLE,
new String[]{DatabaseHelper.BILL_ID_COLUMN, DatabaseHelper.BILL_NAME_COLUMN, DatabaseHelper.BILL_DUE_DATE_COLUMN, DatabaseHelper.BILL_IMAGE_COLUMN}, "id = ?", new String[] {String.valueOf(id)}, null,
null, DatabaseHelper.BILL_DUE_DATE_COLUMN);
Bill bill = null;
while (cursor.moveToNext()) {
bill = new Bill();
bill.setPersistent(true);
bill.setId(cursor.getLong(cursor.getColumnIndex(DatabaseHelper.BILL_ID_COLUMN)));
bill.setName(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_NAME_COLUMN)));
bill.setDueDate(getDate(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_DUE_DATE_COLUMN))));
bill.setImage(cursor.getBlob(cursor.getColumnIndex(DatabaseHelper.BILL_IMAGE_COLUMN)));
}
cursor.close();
return bill;
}
//works fine after updating the blob
public List findAll() {
List bills = new ArrayList();
Cursor cursor = database.query(DatabaseHelper.BILL_TABLE,
new String[]{DatabaseHelper.BILL_ID_COLUMN, DatabaseHelper.BILL_NAME_COLUMN, DatabaseHelper.BILL_DUE_DATE_COLUMN}, null, null, null,
null, DatabaseHelper.BILL_DUE_DATE_COLUMN);
while (cursor.moveToNext()) {
Bill bill = new Bill();
bill.setPersistent(true);
bill.setId(cursor.getLong(cursor.getColumnIndex(DatabaseHelper.BILL_ID_COLUMN)));
bill.setName(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_NAME_COLUMN)));
bill.setDueDate(getDate(cursor.getString(cursor.getColumnIndex(DatabaseHelper.BILL_DUE_DATE_COLUMN))));
bills.add(bill);
}
cursor.close();
return bills;
}
Here is the exception:
java.lang.IllegalStateException: Couldn't read row 0, col 0 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
at android.database.CursorWindow.nativeGetLong(Native Method)
at android.database.CursorWindow.getLong(CursorWindow.java:511)
at android.database.AbstractWindowedCursor.getLong(AbstractWindowedCursor.java:75)
at net.rka.android.billreminder.BillDao.get(BillDao.java:106)
I suspect that updating a blob in a row corrupts the database somehow.
Did anybody run into a similar problem? If so how did you solve it?
Your issue is very likely due to the size of the image(s) and a quirk, for want of a better term, that you can store large BLOB's without issue, but due to the size limitations of an Android's Cursor Window of 2m, that you may not be able to retrieve the BLOB. This sometimes compounded by some of the SQLiteDatabase/Cursor (The Cursor getBlob() or it's underlying methods in this case) methods that basically hide underlying failures, in order to provide what is often a simpler development experience.
If you used the SQLiteDatabase DatabaseUtils.dumpCursor this may highlight the issue(s) that may have been hidden by the SQLiteDatabase query convenience method. So adding :-
DatabaseUtils.dumpCursor(cursor); //<<<< ADDED
while (cursor.moveToNext()) { ........
May provide clues.
I can think of 3 options :-
Rather than store the files as BLOBS, store files as files on disk and store the path in the Database.
Significantly reduce the size of the images.
Look into using C++ and the native SQLIte3 libraries to retrieve the BLOBS into a suitably sized container.
Perhaps the may be some libraries that do this. However, I don't recall any being mentioned.
I am writing a dictionary application. I have a large static SQLite database with one table (100k+ rows) and an »_id«, »word« and »description« columns. Entries are lexicographically ordered by »word«. I use the code below and afterTextChanged(Editable searchText) to make instant searches in the database, loading a selection of rows into an ArrayList, and from there to a list adapter.
The code works, however, it is not very efficient, i.e.:
Searches for words beginning with »z« are much slower than those starting with an »a«.
After inputing »abc«, when user inputs »d«, search for »abcd« starts from the begining instead from the first occurence of »abc«.
Queries for nonexisting strings last very long, but we know that say, »abcdx« can only appear between »abcd« and »abce«.
Is there a way how I can optimize for those properties? Are there any ready-made solutions for dictionaries? Sorry, I am new to this.
I canot replace »LIKE« with »>=« for example, since words also consists of some non-ASCII characters and »>=« query doesn't work with them.
//---retrieves a database subset in the range (MATCH-RowsBefore, MATCH+RowsAfter)---
public Cursor getSearch(Editable typedKeys) throws SQLException
{
Cursor sCursor =
db.rawQuery("SELECT _id, word FROM dict WHERE word LIKE ? LIMIT 1",
new String[]{typedKeys + "%"});
if (sCursor.moveToFirst()) {
SearchedRow = Integer.parseInt(sCursor.getString(0));
int newstart = SearchedRow - RowsBefore;
String curlen = Integer.toString(RowsBefore + RowsAfter + 1);
sCursor = db.query(true, DB_TABLE, new String[] {_id,
word}, _id + ">=" + newstart, null, null, null, null, curlen);
}
return sCursor;
}
I have this function that is filling out a class based on data from several tables. I got the first cursor:
String query="SELECT * FROM SESSION where _id =" + mSessionID + ";";
Cursor c = dbAdapter.selectRecordsFromDB(query, null);
Session session=null;
c.moveToFirst();
This works great. Then a little lower I do this:
long galleryId = c.getInt(4);
long packageId = c.getInt(5);
long contractId = c.getInt(6);
String query2="SELECT * FROM PHOTOPACKAGES WHERE _id =" + packageId + ";";
Cursor p = dbAdapter.selectRecordsFromDB(query2, null);
and the p cursor always returns -1 for its count. I can go right into the sqlite in the adb and run the same query where packageId = 1 and it works great...so not sure why this is not working, i don't see any other errors...can you just not use two cursors on the same database? p.s. selectRecordsFromDB is a helper function:
public Cursor selectRecordsFromDB(String query, String[] selectionArgs) {
Cursor c = myDataBase.rawQuery(query, selectionArgs);
return myDataBase.rawQuery(query, selectionArgs);
}
To answer your actual question: Yes you can target the same DB with multiple cursors. I believe there is something else wrong with your code.
Also as Philip pointed out, creating Cursors is very costly and you do not want to make extras just because, and always close them when finished with them.
Your selectRecordsFromDB function looks pretty darned weird, but it will probably work after a fashion, because the first cursor that you create goes out of focus straight away. Leaking open cursors like that is not a good idea though.
I query and get a result set back, but I need to do some calculations that are impossible in the SQLite WHERE clause in order to determine what shows up in the ListView. How can I remove certain rows from the cursor? I know it is the same question as this Filter rows from Cursor so they don't show up in ListView but that answer does not help. Can an example be provided if there isn't a simpler way to do this?
It might work to simply retain all the rows in the Cursor, but then use a custom adapter to hide the unwanted rows at display time. For example, if you extend CursorAdapter, then you might have something like this in your bindView implementation:
View v = view.findViewById(R.id.my_list_entry);
boolean keepThisRow = .......; // do my calculations
v.setVisibility(keepThisRow ? View.VISIBLE : View.GONE);
There should be a better way to do this, but what I ended up doing is storing the ID of each row I wanted in a string ArrayList, and then requerying where _id IN arraListOfIds.toString(), replacing the square brackets with parentheses to fit SQL syntax.
// Get all of the rows from the database
mTasksCursor = mDbHelper.fetchAllTasks();
ArrayList<String> activeTaskIDs = new ArrayList<String>();
// calculate which ones belong
// .....
if (!hasCompleted)
activeTaskIDs.add(mTasksCursor.getString(TaskerDBadapter.INDEX_ID));
// requery on my list of IDs
mTasksCursor = mDbHelper.fetchActiveTasks(activeTaskIDs);
public Cursor fetchActiveTasks(ArrayList<String> activeTaskIDs)
{
String inClause = activeTaskIDs.toString();
inClause = inClause.replace('[', '(');
inClause = inClause.replace(']', ')');
Cursor mCursor = mDb.query(true, DATABASE_TABLE, columnStringArray(),
KEY_ROWID + " IN " + inClause,
null, null, null, null, null);
if (mCursor != null) { mCursor.moveToFirst(); }
return mCursor;
}
ContentResolver cr = getContentResolver();
Cursor groupCur = cr.query(
Groups.CONTENT_URI, // what table/content
new String [] {Groups._ID, Groups.NAME}, // what columns
"Groups.NAME NOT LIKE + 'System Group:%'", // where clause(s)
null, // ???
Groups.NAME + " ASC" // sort order
);
The "What Columns" piece above is where you can tell the cursor which rows to return. Using "null" returns them all.
I need to do some calculations that
are impossible in the SQLite WHERE
clause
I find this very hard to believe; my experience has been that SQL will let you query for just about anything you'd ever need (with the exception of heirarchical or recursive queries in SQLite's case). If there's some function you need that isn't supported, you can add it easily with sqlite_create_function() and use it in your app. Or perhaps a creative use of the SELECT clause can do what you are looking for.
Can you explain what these impossible calculations are?
EDIT: Nevermind, checking out this webpage reveals that the sqlite_create_function() adapter is all closed up by the Android SQLite wrapper. That's annoying.
I have a query that selects rows in a ListView without having a limit. But now that I have implemented a SharedPreferences that the user can select how much rows will be displayed in the ListView, my SQLite query doesn't work. I'm passing the argument this way:
return wDb.query(TABELANOME, new String[] {IDTIT, TAREFATIT, SUMARIOTIT}, CONCLUIDOTIT + "=1", null, null, null, null, "LIMIT='" + limite + "'");
The equals (=) operator is not used with the LIMIT clause. Remove it.
Here's an example LIMIT query:
SELECT column FROM table ORDER BY somethingelse LIMIT 5, 10
Or:
SELECT column FROM table ORDER BY somethingelse LIMIT 10
In your case, the correct statement would be:
return wDb.query(TABELANOME, new String[] {IDTIT, TAREFATIT, SUMARIOTIT}, CONCLUIDOTIT + "=1", null, null, null, null, String.valueOf(limite));
Take a look here at the SQLite select syntax: http://www.sqlite.org/syntaxdiagrams.html#select-stmt
This image is rather useful: http://www.sqlite.org/images/syntax/select-stmt.gif
For anyone stumbling across this answer looking for a way to use a LIMIT clause with an OFFSET, I found out from this bug that Android uses the following regex to parse the limit clause of a query:
From <framework/base/core/java/android/database/sqlite/SQLiteQueryBuilder.java>
LIMIT clause is checked with following sLimitPattern.
private static final Pattern sLimitPattern = Pattern.compile("\\s*\\d+\\s*(,\\s*\\d+\\s*)?");
Note that the regex does accept the format offsetNumber,limitNumber even though it doesn't accept the OFFSET statement directly.
Due to this bug which also doesn't allow for negative limits
8,-1
I had to use this workaround
SQLiteQueryBuilder builder = new SQLiteQueryBuilder();
builder.setTables(table);
String query = builder.buildQuery(projection, selection, null, null, null, sortOrder, null);
query+=" LIMIT 8,-1";