I'm writing on a small Android App with SQLite 3 support. ATM I'm implementing some DB-Functions and asking myself if it is better have one big joined query or if it is OK to use multiple queries in respect to App performance.
Let's say I have a 3 Tables "drivers", "driven_runs", "race_disciplines". Every driven_run has a driver and a race_discipline. Let's assume I want to get all race disciplines by a certain driver in a certain discipline.
Solution 1
I already coded a function getDriver(driverID) with a Driver-Object in return and a function getRaceDiscipline(disciplineID) with a Race Discipline in return. So I would just create a function
public ArrayList<DrivenRun> getDrivenRunsOnDiscipline(short driverID, short disciplineID) {
ArrayList<DrivenRun> drivenRuns = new ArrayList<DrivenRun>();
String sql = "SELECT * FROM " + DBHelper.TABLE_DRIVEN_RUNS + " WHERE "
+ DBHelper.DRIVEN_RUNS_COLUMN_DRIVER_ID + "=" + driverID + " AND "
+ DBHelper.DRIVEN_RUNS_COLUMN_RACE_DISCIPLINE_ID + "=" + disciplineID + ";";
Cursor result = rawQuery(sql);
if (result.moveToFirst()) {
do {
Driver driver = getDriver(driverID);
RaceDiscipline discipline = getRaceDiscipline(disciplineID);
DrivenRun run = new DrivenRun();
run.setDriver(driver);
run.setDiscipline(discipline);
run.setResult("WHATEVER");
drivenRuns.add(run);
} while(result.moveToNext());
}
return drivenRuns;
}
in this case there would be 3 queries executed on after another but the coding is much more simple.
Solution 2
I would create one big joined query like
String sql = "SELECT * FROM driven_runs CROSS JOIN drivers CROSS_JOIN race_disciplines WHERE driven_runs.driver_id=drivers.id AND driven_runs.race_discipline_id=race_disciplines.id"
Cursor result = rawQuery(sql);
and would manually create the Driver and DrivenRun Object.
This solution needs much more writing but only one query is executed (or does the DB executes 3 queries as well when joining 3 tables?)
Long story short, is it OK to go with solution 1 because in regards to performance there isn't much of a difference?
In general, go for the simpler code until there's a good performance reason not to. Given that this is SQLite anyway, I don't think there's likely to be much performance difference, since the overhead for queries is pretty low.
Premature optimization is the root of all evil.
You should use proper join syntax, then your query won't look so cumbersome:
SELECT THECOLUMNSYOUREALLYNEED
FROM driven_runs JOIN
drivers
on driven_runs.driver_id=drivers.id join
race_disciplines
on driven_runs.race_discipline_id=race_disciplines.id
where driver.id = YOURDRIVEIDHERE and race_discipline = YOURDISCIPLINEIDHERE
Also, only return the columns that you need. Second, insert the appropriate ids in the where clause. Your version is returning everything, which is totally unnecessary.
This is a pretty simple query and it does what SQL databases do best -- joining large tables together. Even with SQLite, you are probably better off letting the database do the work. It is, at the very least, going to save some round trip communication from the database layer back to the application layer.
In a more sophisticated environment, the database will take advantage of multiple processors, multiple disks, and intelligently cache results to further optimize query response.
Related
I saw in a tutorial a code updating data from a SQlite database using execSQL:
String update = "UPDATE FRUIT SET COLOR=? WHERE ID=?";
myDatabase.execSQL( update, new Object[] {"RED", 7});
Cursor cursor = myDatabase.rawQuery("SELECT * FROM FRUIT;", null);
if (cursor.moveToFirst()) {
do {
String name = cursor.getString(cursor.getColumnIndex("NAME"));
String color = cursor.getString(cursor.getColumnIndex("COLOR"));
Log.i(TAG, "onCreate: Name: " + name + ", color: " + color);
} while (cursor.moveToNext());
}
but, I read this in the oficial documentation of Android:
The code using execSQL worked but it's better to use update or I can still use execSQL since it worked? What's better for good practice? Since this tutorial is from a trustworthy source, why are they using execSQL?
The issue/warning regarding using execSQL may be for a few reasons, one of them being that you do no get anything returned when using execSQL, whilst the convenience methods return potentially useful values.
insert will return a long containing the id of the inserted row else -1.
update and delete return the number of affected rows.
Using the convenience methods also reduces the chance of making typing errors by build the underlying SQL, adding the keywords and enclosing strings and for some importantly offering a level of protection against SQL injection (execSQL's
bindArgs also offers protection against SQL Injection, likewise with rawQuery).
However, there are limitations and sometimes the use of execSQL/rawQuery becomes necessary for some more complex situations.
I am building an Android app that uses a SQLite database.
For this one task I have to run a query that looks like this:
SELECT item.id, item.price, t1.quantity
FROM item, (SELECT id, price
FROM list
WHERE list.state = 'sold') t1
WHERE item.id = t1.id
So far, I have tried:
Cursor c = resolver.query(uriRawQuery, null, selection, null, null)
where uriRawQuery is used to tell the ContentProvider that it should perform a db.rawQuery(selection, null) and selection is a string similar to the query above.
The problem is no data is returned into the Cursor. When I call c.moveToFirst() I get false.
The weird thing is that if I open the database file in SQLite Manager and run the exact same query I get results.
I know I can modify the query to make a join between the original list and item tables but I find it to be less efficient that way.
Any ideas would be very appreciated as I have spent too man hours on this already.
EDIT
I know what a join is, what I said is that it is a lot more efficient if I do it like this instead of using the entire list table.
I forgot a very important aspect
The WHERE clause looks like
" WHERE list.state = 'sold' and list.name like '" + arg + "%'"
where arg is a string.
I managed to solve the problem, I still don't know why this was happening but at least I got the Cursor to actually select the rows.
After many trials I thought about ditching the syntax above and write this instead:
" WHERE list.state = 'sold' and list.name like ? "
and move the argument in
selectionArgs = new String[]{arg + "%"}
I am going to wait a while before accepting the answer, in case someone provides an explanation as to why even though both queries look exactly the same they get different results.
I need to improve the speed of this query,it takes too long(16992ms) and the SQLite Admin makes it much faster
public Cursor getContacts(String search)
{
Cursor c;
String[]columns = {Constants.USER_NAME,Constants.PHONE_NUMBER};
String Limit = "0,20";
String query = search != null ? Constants.USER_NAME + " LIKE '" + search + "%' " : "";
c = myDataBase.query(Constants.TABLE_NAME,columns, query, null, null, null, null, Limit);
return c;
}
Depending on the size of your Database, a LIKE-statement takes some time.
Form your method-name I guess you're implementing a search for your application? There is a nice tutorial on how to do that: Link. This also covers how you can speed up the search.
LIKE statements are very intensive in SQLite in large databases, especially with large strings.
You can try a quick trick and turn pragma off to see if it speeds anything up. Do not do this if your database is constantly updating, as it may interfere with atomicity. If its a static DB, this is excellent. Speeds up my mass queries and inserts by at least double, but they weren't using like. I'd be interested to hear how this effects your speed.
rawQuery('PRAGMA synchronous = OFF');
I have a database with five tables in an Android application. I have been surfing around looking for a way to put conditions in the query (WHERE, &&, OR).
My queries are the form:
public Cursor getAlternative(int questionid) {
Cursor cursor = mDb.query(DBTABLE_ALTERNATIVE, new String[] { KEY_ALT }, KEY_QID + "=" + questionid, null, null, null, null, null);
return cursor;
}
But I find that many people write their queries with regular SQL, for ex:
Cursor c = myDB.query("SELECT FirstName,Age" +
" FROM " + MY_DATABASE_TABLE
+ " WHERE Age > 10 LIMIT 7;",
null);
What is the most efficient way? To me it seems easier form regular SQL statements, but after reading the tutorials on the Android Dev site I started forming the queries like above.
Question 2: if I use the first way, how can I use two conditions? Say I have two parameters, questionid and categoryid, how do I put the next KEY_CID + "=" + categoryid in there?
I have tried with && and AND but none seem to work. Thanks.
What is the most efficient way?
It depends... but generally speaking the second way will be faster. Why? because it won't need to build the query string using the parameters that the query method takes.
However, I'd rather use the first way since it's less error prone.
Question 2. If I use the first way, how can I use two conditions?
It should work this way:
KEY_QID + "=" + questionid + " AND " +KEY_CATID + "=" + categoryid
I am not sure if the simple implications about String and StringBuffers (StringBuilder would even better) hold, as the SQL engine also needs to parse that query string again.
The db.query() way may have the advantage that some parts of the query can be stored in a pre-parsed way (think "PreparedStatement"). Especially if the parameters are not put in the string, but as placeholders
E.g. where KEY_QID=? AND KEY_CATID=?
Here the basic query "stays constant" and the system can optimize.
We have about 7-8 tables in our Android application each having about 8 columns on an average. Both read and write operations are performed on the database and I am experimenting and trying to find ways to enhance the performance of the DataAccess layer. So, far I have tried the following:
Use positional arguments in where clauses (Reason: so that sqlite makes use of the same execution plan)
Enclose inserts and update with transactions(Reason: every db operation is enclosed within a transaction by default. Doing this will remove that overhead)
Indexing: I have not created any explicit index other than those created by default on the primary key and unique keys columns.(Reason: indexing will improve seek time)
I have mentioned my assumptions in paranthesis; please correct me if I am wrong.
Questions:
Can I add anything else to this list? I read somewhere that avoiding the use of db-journal can improve performance of updates? Is this a myth or fact? How can this be done, if recomended?
Are nested transactions allowed in SQLite3? How do they affect performance?
The thing is I have a function which runs an update in a loop, so, i have enclosed the loop within a transaction block. Sometimes this function is called from another loop inside some other function. The calling function also encloses the loop within a transaction block. How does such a nesting of transactions affect performance?
The where clauses on my queries use more than one columns to build the predicate. These columns might not necessarily by a primary key or unique columns. Should I create indices on these columns too? Is it a good idea to create multiple indices for such a table?
Pin down exactly which queries you need to optimize. Grab a copy of a typical database and use the REPL to time queries. Use this to benchmark any gains as you optimize.
Use ANALYZE to allow SQLite's query planner to work more efficiently.
For SELECTs and UPDATEs, indexes can things up, but only if the indexes you create can actually be used by the queries that you need speeding up. Use EXPLAIN QUERY PLAN on your queries to see which index would be used or if the query requires a full table scan. For large tables, a full table scan is bad and you probably want an index. Only one index will be used on any given query. If you have multiple predicates, then the index that will be used is the one that is expected to reduce the result set the most (based on ANALYZE). You can have indexes that contain multiple columns (to assist queries with multiple predicates). If you have indexes with multiple columns, they are usable only if the predicates fit the index from left to right with no gaps (but unused columns at the end are fine). If you use an ordering predicate (<, <=, > etc) then that needs to be in the last used column of the index. Using both WHERE predicates and ORDER BY both require an index and SQLite can only use one, so that can be a point where performance suffers. The more indexes you have, the slower your INSERTs will be, so you will have to work out the best trade-off for your situation.
If you have more complex queries that can't make use of any indexes that you might create, you can de-normalize your schema, structuring your data in such a way that the queries are simpler and can be answered using indexes.
If you are doing a large number of INSERTs, try dropping indexes and recreating them at the end. You will need to benchmark this.
SQLite does support nested transactions using savepoints, but I'm not sure that you'll gain anything there performance-wise.
You can gain lots of speed by compromising on data integrity. If you can recover from database corruption yourself, then this might work for you. You could perhaps only do this when you're doing intensive operations that you can recover from manually.
I'm not sure how much of this you can get to from an Android application. There is a more detailed guide for optimizing SQLite in general in the SQLite documentation.
Here's a bit of code to get EXPLAIN QUERY PLAN results into Android logcat from a running Android app. I'm starting with an SQLiteOpenHelper dbHelper and an SQLiteQueryBuilder qb.
String sql = qb.buildQuery(projection,selection,selectionArgs,groupBy,having,sortOrder,limit);
android.util.Log.d("EXPLAIN",sql + "; " + java.util.Arrays.toString(selectionArgs));
Cursor c = dbHelper.getReadableDatabase().rawQuery("EXPLAIN QUERY PLAN " + sql,selectionArgs);
if(c.moveToFirst()) {
do {
StringBuilder sb = new StringBuilder();
for(int i = 0; i < c.getColumnCount(); i++) {
sb.append(c.getColumnName(i)).append(":").append(c.getString(i)).append(", ");
}
android.util.Log.d("EXPLAIN",sb.toString());
} while(c.moveToNext());
}
c.close();
I dropped this into my ContentProvider.query() and now I can see exactly how all the queries are getting performed. (In my case it looks like the problem is too many queries rather than poor use of indexing; but maybe this will help someone else...)
I would add these :
Using of rawQuery() instead of building using ContentValues will fasten up in certain cases. off course it is a little tedious to write raw query.
If you have a lot of string / text type data, consider creating Virtual tables using full text search (FTS3), which can run faster query. you can search in google for the exact speed improvements.
A minor point to add to Robie's otherwise comprehensive answer: the VFS in SQLite (which is mostly concerned with locking) can be swapped out for alternatives. You may find one of the alternatives like unix-excl or unix-none to be faster but heed the warnings on the SQLite VFS page!
Normalization (of table structures) is also worth considering (if you haven't already) simply because it tends to provide the smallest representation of the data in the database; this is a trade-off, less I/O for more CPU, and one that is usually worthwhile in medium-scale enterprise databases (the sort I'm most familiar with), but I'm afraid I've no idea whether the trade-off works well on small-scale platforms like Android.