SQLite Indexing? - android

My app is using an external SQLite database. The database is created using DB Browser for SQLite software. I am using the following method to query my table with the column ENGLISH (same as en_word). However, problem is the query is slow when my database become large.
public static final String ENGLISH = "en_word";
public static final String TABLE_NAME = "words";
String sql = "SELECT * FROM " + TABLE_NAME +
" WHERE " + ENGLISH + " LIKE ? ORDER BY LENGTH(" + ENGLISH + ") LIMIT 100";
SQLiteDatabase db = initializer.getReadableDatabase();
Cursor cursor = null;
try {
cursor = db.rawQuery(sql, new String[]{"%" + englishWord.trim() + "%"});
List<Bean> wordList = new ArrayList<Bean>();
while(cursor.moveToNext()) {
String english = cursor.getString(1);
String mal = cursor.getString(2);
wordList.add(new Bean(english, mal));
}
return wordList;
} catch (SQLiteException exception) {
exception.printStackTrace();
return null;
} finally {
if (cursor != null)
cursor.close();
}
I tried to create index using DB Browser for SQLite.
CREATE INDEX `kindx` ON `words` ( `en_word` )
However, do I need to modify my code so that my app will query the database using this index? If so, how to do that?

The problem is that SQLite, like most relational databases, can use an index when the parameter to a 'like' clause ends with a wildcard, it cannot use an index when the parameter begins with a wildcard.
So, for this type of query, the index will not be used, and you wind up with a full table scan. This is why it is slower with a large number of rows.
You are actually attempting to do what is known as "full text search", which is not really possible to do efficiently without database features to support it directly.
I have not tried it, but I see that SQLite does have full-text search capabilities, and that it is supported on Android. See Full text search example in Android for an example.

Related

Sql Query to retrieve a particular data from particular column and row in android?

I want to fetch phone number linked to particular email in the database. I am not able to find the query for it or how
public String getContactNumber(String email){
SQLiteDatabase db = this.getReadableDatabase();
String query = "SELECT " + COLUMN_USER_MOBILE_NUMBER + " FROM " + TABLE_USER + " WHERE " + email + " = " + COLUMN_USER_EMAIL;
Cursor cursor = db.rawQuery(query,null);
//What to put here to extract the data.
String contact = cursor.getString(get);
cursor.close();
return contact;
}
to extract the data. Completely a beginner
Try this ..
public List<String> getMyItemsD(String emailData) {
List<String> stringList = new ArrayList<>();
SQLiteDatabase db = this.getReadableDatabase();
String selectQuery = "SELECT COLUMN_USER_MOBILE_NUMBER FROM " + USER_TABLE_NAME + " WHERE email= " + emailData;
Cursor c = db.rawQuery(selectQuery, null);
if (c != null) {
c.moveToFirst();
while (c.isAfterLast() == false) {
String name = (c.getString(c.getColumnIndex("Item_Name")));
stringList.add(name);
c.moveToNext();
}
}
return stringList;
}
public String getContactNumber(String email){
String contact = "";
SQLiteDatabase db = this.getReadableDatabase();
String query = "SELECT " + COLUMN_USER_MOBILE_NUMBER + " FROM " + TABLE_USER + " WHERE " + email + " = " + COLUMN_USER_EMAIL;
Cursor cursor = db.rawQuery(query,null);
if(cursor.getCount()>0) {
cursor.moveToNext();
contact = cursor.getString(cursor.getColumnIndex(COLUMN_USER_MOBILE_NUMBER));
}
//What to put here to extract the data.
cursor.close();
return contact;
}
From this method you get phone number value of that email which you pass any other method easily.
I'd suggest the following :-
public String getContactNumber(String email){
String contact = "NO CONTACT FOUND"; //<<<<<<<<<< Default in case no row is found.
SQLiteDatabase db = this.getWritableDatabase(); //<<<<<<<<<< Generally getReadable gets a writable database
String[] columns_to_get = new String[]{COLUMN_USER_MOBILE_NUMBER};
String whereclause = COLUMN_USER_EMAIL + "=?";
String[] whereargs = new String[]{email};
Cursor cursor = db.query(TABLE_USER,columns_to_get,whereclause,whereargs,null,null,null);
//What to put here to extract the data.
if (cursor.moveToFirst()) {
contact = csr.getString(csr.getColumnIndex(COLUMN_USER_MOBILE_NUMBER));
}
cursor.close();
return contact;
}
The above does assumes that there will only be 1 row per email (which is most likely).
Explanations
A default value is set so that you can easily tell if an invalid/non-existent email is passed (you'd check the return value if need be (might be easier to simply have "" and check the length as a check)).
getReadableDatabase has been replaced with getWritableDatabase as unless there are issues with the database a writable database will be returned, as per :-
Create and/or open a database. This will be the same object returned
by getWritableDatabase() unless some problem, such as a full disk,
requires the database to be opened read-only. In that case, a
read-only database object will be returned. If the problem is fixed, a
future call to getWritableDatabase() may succeed, in which case the
read-only database object will be closed and the read/write object
will be returned in the future.
getReadableDatabase
Note no real problem either way;
The recommended query method has been used instead of the rawQuery method. This has distinct advantages, it builds the underlying SQL and also offers protection against SQL injection (just in case the email passed is input by a user).
this version of the method takes 7 parameters :-
The table name as a string
The columns to be extracted as an array of Strings (aka String array). null can be all columns.
The where clause less the WHERE keyword with ?'s to represent arguments (see next). null if no WHERE clause.
The arguments to be applied (replace ?'s 1 for 1) as a String array. null if none or no WHERE clause.
The GROUP BY clause, less the GROUP BY keywords. null if no GROUP BY clause.
The HAVING clause, less the HAVING keyword. null if no HAVING clause.
The ORDER BY clause, less the ORDER BY keywords. null if no ORDER BY clause.
SQLiteDatabase - query
- Note there are 4 query methods (see link for the subtle difference, I believe this is the most commonly used)
The data extraction is the new code. When a Cursor is returned it is at a position BEFORE THE FIRST ROW, so you need to move to a valid row. So the moveToFirst* method is suitable (note that if a move cannot be made by a move method that it will return false, hence how you can say if (cursor.moveToFirst())). The data is then extracted from the appropriate column use the **getString method, which takes an int as an argumnet for the column offset (0 in this case). However, using hard coded values can lead to issues so the getColumnIndex method is used to get the offset according to the column name (-1 is returned if the named column is not in the Cursor).

View/edit SQLite database on Android emulator

I am developing an app (that uses SQLite) in Android Studio 2.3 on Mac OS and would like to view the SQLite database, but can't seem to find the sqlite file. Where are the AVM files and specifically the sqlite file?
I found the AVD folder
/Users/<my name>/.android/avd/Pixel_XL_API_26.avd but can't find the user generated data. Any suggestions?
You could always include methods to find out information about the database.
For example you can query the sqlite_master table e.g. :-
Cursor csr = db.query("sqlite_master",null,null,null,null,null,null);
which equates to :-
SELECT * FROM sqlite_master
The are PRAGMA statments that allow you to interrogate/change internals PRAGMA Statements.
Note use rawQuery to obtain information into a cursor but use execSQL to make changes e.g. to set and get user_version (Database Version) respectively:-
db.execSQL("PRAGMA user_version=" + Integer.toString(version));
Cursor csr = db.rawQuery("PRAGMA user_version",null);
You could also look at the data, output to the log, in the tables with something based upon (where tablename would be changed to the appropriate tablename) :-
Cursor csr = db.query("tablename",null,null,null,null,null,null);
String logdata;
while (csr.moveToNext()) {
Log.d("DBINFO","Row " + csr.getPosition());
logdata = "";
for (int i =0; i < csr.getColumnCount(); i++) {
String val;
try {
val = csr.getString(i);
}
catch (Exception e) {
val = "unobtainable";
}
logdata = logdata + "\t\nColumn Name=" +
csr.getColumnName(i) +
" Value=" + val;
}
Log.d("DBINFO",logdata);
}
csr.close();
Note! this only uses getString so will not properly represent some values e.g. double's and floats, if a BLOB then the value will be unobtainable.

How to efficiently query sqlite database multiple times on Android

For my application, I need to query a sqlite database around 40-50 times. I am sure that the code I wrote is very inefficient. Unfortunately, I cannot find many examples online that involves querying the database many times.
String[] entryValArray = new String[indicesList.size()];
DBHelper dbHelper = new DBHelper(MainActivity.context);
SQLiteDatabase db = dbHelper.getReadableDatabase();
for (int i = 0; i < indicesList.size(); i++) {
int moddedIndex = Integer.parseInt(indicesList.get(i), 16) % DBHelper.numEntries;
String queryStr = "select * from " + DBHelper.TBL_NAME + " where " + DBHelper.IDStr +
" = " + Integer.toString(moddedIndex);
Cursor cursor = db.rawQuery(queryStr, null);
if (cursor.moveToFirst())
entryValArray[i] = cursor.getString(1);
cursor.close();
}
Basically, I am taking a list of strings, converting them to hex values, and then modding the value to get an index into a sqlite database. This is for a password generator application.
Is there a better way to do this, especially regarding creating a cursor and then closing it in every iteration.
First of all you have to change your query string as you need only one column value but you are using
Select *
instead of
Select yourColumn
. Secondly if your indices list size is not very large you can use
IN(values ) function of db instead of
" where " + DBHelper.IDStr +" = " + Integer.toString(moddedIndex);
this will return the result in only one query you don't have to run a whole loop.

Fastest way to search through strings stored in sqlite database

I have large number of strings, approximately 15,000 that I stored in a SQLite database using the following code:
void addKey(String key, String value, String table) {
SQLiteDatabase db = this.getWritableDatabase();
ContentValues values = new ContentValues();
values.put(KEY_KEY, key); // Contact Name
values.put(KEY_VALUE, value); // Contact Phone
// Inserting Row
db.insert(table, null, values);
db.close(); // Closing database connection
}
And then i search through that database using the following method in order to pick out any strings that match the key im looking for:
public String searchKeyString(String key, String table){
String rtn = "";
Log.d("searchKeyString",table);
// Select All Query
String selectQuery = "SELECT * FROM " + table;
SQLiteDatabase db = this.getWritableDatabase();
Cursor cursor = db.rawQuery(selectQuery, null);
// looping through all rows and adding to list
if (cursor.moveToFirst()) {
do {
Log.d("searchKeyString","searching");
if(cursor.getString(1).equals(key))
rtn = rtn + "," + cursor.getString(2);
} while (cursor.moveToNext());
}
cursor.close();
db.close();
Log.d("searchKeyString","finish search");
return rtn;
}
The goal is to do this in real time as the user is typing on the keep board so response time is key and the way it stands now it takes over a second to run through the search.
I considered reading all of the items into an array list initially and sorting through that which might be faster, but i thought an array list of that size might cause memory issues. What is the best way to search through these entries in my database?
A couple of things you can do...
Change the return to a StringBuilder until the end.
Only use a readable version of the database (that's probably not making much difference though)
Do not get a new instance of the database every time, keep it opened until you don't need it anymore
Query for only what you need with the "WHERE" argument in the SQL query.
See the code below with some changes:
// move this somewhere else in your Activity or such
SQLiteDatabase db = this.getReadableDatabase();
public String searchKeyString(String key, String table){
StringBuilder rtn = new StringBuilder();
Log.d("searchKeyString",table);
// Select All Query
String selectQuery = "SELECT * FROM " + table + " WHERE KEY_KEY=?";
Cursor cursor = db.rawQuery(selectQuery, new String[] {key});
// you can change it to
// db.rawQuery("SELECT * FROM "+table+" WHERE KEY_KEY LIKE ?", new String[] {key+"%"});
// if you want to get everything starting with that key value
// looping through all rows and adding to list
if (cursor.moveToFirst()) {
do {
Log.d("searchKeyString","searching");
rtn.append(",").append(cursor.getString(2));
} while (cursor.moveToNext());
}
cursor.close();
Log.d("searchKeyString","finish search");
return rtn.toString();
}
Note even if you want this to happen in "real-time" for the user, you will still need to move this to a separate Thread or ASyncTask or you are going to run into problems....
You should consider using SELECT * FROM your-table LIMIT 50, for example. And you can put two buttons "Back", "Next" on your view. If every page has max 50 items, the user is at page 1, and he taps "Next", then you can use this query:
SELECT * FROM your-table LIMIT 50 OFFSET 50
If your table contains most of text-data, and you want to integrate search deeply into your app, consider using virtual table with FTS.
Let sqlite do the hard lifting.
First off, add an index to the field you're searching for, if you don't have one already. Secondly, don't do a SELECT all with manual table scan, but rather use a query in the form
SELECT column_value
FROM my_table
WHERE column_key LIKE "ABC%"
This returns the least amount of data, and the sql engine uses the index.
i dunno about better but maybe it'd be faster to make queries for the selected strings one by one.
public String searchKeyString(String key, String table){
String rtn = "";
Log.d("searchKeyString",table);
// Select All Query
String selectQuery = "SELECT * FROM " + table + "WHERE column_1 = " + key;
SQLiteDatabase db = this.getWritableDatabase();
Cursor cursor = db.rawQuery(selectQuery, null);
// looping through all rows and adding to list
if (cursor.moveToFirst()) {
rtn = rtn + "," + cursor.getString(2);
}
cursor.close();
db.close();
Log.d("searchKeyString","finish search");
return rtn;
}
EDIT:
Well i dunno how those custom keyboard apps do it, but those AutoCompleteTextViews are hooked up to adapters. you could just as easily make a cursorAdapter and hook your auto-complete view to it.
http://www.outofwhatbox.com/blog/2010/11/android-autocompletetextview-sqlite-and-dependent-fields/
http://www.opgenorth.net/blog/2011/09/06/using-autocompletetextview-and-simplecursoradapter-2/

Android database strangeness listing columns

I am getting inconsistent results between two methods of reading the columns in an Android SQLite database.
First, this is part of a database upgrade routine as per the accepted answer here: Upgrade SQLite database from one version to another?
The technique involves moving the current table away with a temporary name, creating a new table with the new schema, and then copying relevant data from the old table into the new one before deleting the old temporary table.
The particular problem I have is when I remove a column from the schema. So, a particular column exists in the old version of the table, but not the new one.
That answer suggests using a method like this to list the columns in the table:
/**
* Returns a list of the table's column names.
*/
private List<String> getColumns(SQLiteDatabase db, final String tableName) {
List<String> ar = null;
Cursor c = null;
try {
c = db.rawQuery("SELECT * FROM " + tableName + " LIMIT 1", null);
if (c != null) {
ar = new ArrayList<String>(Arrays.asList(c.getColumnNames()));
}
} finally {
if (c != null)
c.close();
}
return ar;
}
That works fine on the old table, before I move it away with a temporary name and replace it. When I run the same query again later, on the newly-created empty table, it still lists the old table schema with the name of the column which no longer exists. It looks as if it's reusing stale cached results for that query.
If I read the columns a different way, using this instead, then it returns the new column list as expected:
private void listColumns(SQLiteDatabase db, final String tableName) {
final String query = "PRAGMA table_info(" + tableName + ");";
Cursor c = db.rawQuery(query, null);
while (c.moveToNext()) {
Log.v("MyApp", "Column: " + c.getString(1));
}
c.close();
}
The complete sequence is:
final String tempTableName = "temp_" + tableName;
table.addToDb(db); // ensure it exists to start with
// get column names of existing table
final List<String> columns = getColumns(db, tableName);
// backup table
db.execSQL("ALTER TABLE " + tableName + " RENAME TO " + tempTableName);
// create new table
table.addToDb(db);
// delete old columns which aren't in the new schema
columns.retainAll(getColumns(db, tableName));
// restore data from old into new table
String columnList = TextUtils.join(",", columns);
db.execSQL(String.format("INSERT INTO %s (%s) SELECT %s from %s", tableName, columnList, columnList,
tempTableName));
// remove backup
db.execSQL(DROP_TABLE + tempTableName);
What's the reason for the different results?
I assume you have done something similar to this:
ALTER TABLE "main"."mytable" RENAME TO "newtable";
CREATE TABLE "main"."mytable" ("key1" text PRIMARY KEY,"key2" text,"key3" text);
INSERT INTO "main"."mytable" SELECT "key1","key2","key3" FROM "main"."newtable";
DROP TABLE "main"."newtable";
If you have, please share the equivalent code, just to rule out any errors with this part.
I never got to the bottom of this. I just ended up using the second method I mentioned, which doesn't exhibit the problem.

Categories

Resources