SQLite Fts select query - android

I am making a dictionary of over 20,000 words in it. So, to make it work faster when search data, i am using fts3 table to do it.
my select query:
Cursor c=db.rawQuery("Select * from data where Word MATCH '"+word+"*'", null);
Using this query, it will show all the word that contain 'word' , but what i want is to get only the word that contain the beginning of the searching word.
Mean that i want it work like this query:
Cursor c=db.rawQuery("Select * from data where Word like '"+word+"%'", null);
Ex: I have : apple, app, and, book, bad, cat, car.
when I type 'a': i want it to show only: apple, app, and
What can i solve with this?


table(_id primary key not null autoincrement, word text)
FTS table does not use the above attributes. It ignores data type. It does not auto increment columns other than the hidden rowid column. "_id" will not act as a primary key here. Please verify that you are implementing an FTS table
https://www.sqlite.org/fts3.html
a datatype name may be optionally specified for each column. This is
pure syntactic sugar, the supplied typenames are not used by FTS or
the SQLite core for any purpose. The same applies to any constraints
specified along with an FTS column name - they are parsed but not used
or recorded by the system in any way.
As for your original question, match "abc*" already searches from the beginning of the word. For instance match "man*" will not match "woman".


FTS supports searching for the beginning of a string with ^:
SELECT * FROM FtsTable WHERE Word MATCH '^word*'
However, the full-text search index is designed to find words inside larger texts.
If your Word column contains only a single word, your query is more efficient if you use LIKE 'a%' and rely on a normal index.
To allow an index to be used with LIKE, the table column must have TEXT affinity, and the index must be declared as COLLATE NOCASE (because LIKE is not case sensitive):
CREATE TABLE data (
...
Word TEXT,
...
);
CREATE INDEX data_Word_index ON data(Word COLLATE NOCASE);
If you were to use GLOB instead, the index would have to be case sensitive (the default).
You can use EXPLAIN QUERY PLAN to check whether the query uses the index:
sqlite> EXPLAIN QUERY PLAN SELECT * FROM data WHERE Word LIKE 'a%';
0|0|0|SEARCH TABLE data USING INDEX data_Word_index (Word>? AND Word<?)

Related

Using FTS3/4 Table based on Select Statements from Normal Table

In my app, I'm developing in Android, I have a Sqlite Table called Transactions, with these fields:
_Id | Date | Value | Notes
I already have a ListView showing results filtering by Date (for example):
Select * FROM Transacions WHERE Date BETWEEN '2016-04-25' AND '2016-05-14'
It works fine, but I want to implement a SearchView to search transactions between a custom date have in field Notes, some text typed in SearchView.
I read about adding a SearchView, and the best way to implement to search is using a FTS3 or FTS4 Table, allowing the user, for example, type "SUPERMARKET" and find a Transaction where the Notes have this text.
The problem appears because a FTS table is slow to perform WHERE conditions (like the above, to filter date)...
How can I implement both Filtering date, using WHERE date BETWEEN ... and ..., and filtering text Notes with the performance of a FTS Table?
If it is not possible to do so, is it a good idea to have a query like this:
Select * FROM Transacions WHERE (Date BETWEEN '2016-04-25' AND '2016-05-14) AND Notes LIKE '%text%''?

Do not think of an FTS table as a table, but as an index.
With the notes indexed like this:
CREATE VIRTUAL TABLE Transactions_FTS USING FTS4(Notes);
you would have to ensure that the IDs of both tables match, and could then combine the tables like this:
SELECT *
FROM Transactions
WHERE Date BETWEEN '2016-04-25' AND '2016-05-14'
AND _Id IN (SELECT docid
FROM Transactions_FTS
WHERE Notes MATCH 'supermarket');
or this:
SELECT *
FROM Transactions
JOIN Transactions_FTS ON Transactions._Id = Transactions_FTS.docid
WHERE Date BETWEEN '2016-04-25' AND '2016-05-14'
AND Transactions_FTS.Notes MATCH 'supermarket';
(If you care about the amount of storage used, consider an external content FTS table.)

Android Sqlite FTS3 how to select words that starts with?

For example, if i have these records
word
AAA
AAB
AAC
BAA AA
With a normal table i would use sql like
select * from table where word like 'AA%'order by H collate nocase asc
How do i select with FTS3 table instead?
Also i would like to know if FTS3 will still have better performance than normal table with this kind of query?

How do i select with FTS3 table instead?
Quoting the documentation:
An FTS table may be queried for all documents that contain a specified term (the simple case described above), or for all documents that contain a term with a specified prefix. As we have seen, the query expression for a specific term is simply the term itself. The query expression used to search for a term prefix is the prefix itself with a '*' character appended to it.
The documentation also gives a sample:
-- Query for all documents containing a term with the prefix "lin". This will match
-- all documents that contain "linux", but also those that contain terms "linear",
--"linker", "linguistic" and so on.
SELECT * FROM docs WHERE docs MATCH 'lin*';

How to write contains query in SQLite fts3 fulltext search

I want to make fulltext search in my index table which is sqlite fts3.
For example;
the data set is { "David Luiz", "David Villa", "Diego Costa", "Diego Ribas", "Diego Milito","Gabriel Milito", }
When I type "vid i" I want to get {"David Luiz", "David Villa"}
In documentation of SQLite I found this
http://www.sqlite.org/fts3.html#section_3
but it contains just startswith query.
my query is:
SELECT *FROM Table WHERE Table MATCH "*vid* *i*"
I dont know it is possible or not. If it is possible to make search in sqlite fts3, any help will be appreciated

The FTS index is optimized for word searches, and supports word prefix searches.
There is no index that can help with searches inside words.
You have to use LIKE '%vid%' (which scans the entire table).

Change your query from
SELECT * FROM Table WHERE Table MATCH "*vid* *i*"
To
SELECT * FROM SOME_TABLE WHERE some_column LIKE '%vid%'

How to do a word search on a large text database

I have a large database in my app. One column is made of text strings that are about a sentence to a paragraph long. I would like to make this column searchable by word(s) that the user inputs.
How would I make a quick search? I've heard of making an index but I don't know how to do that for a text search.

SQLite has a mechanism for storing a lot of text in a database, it's called FTS (short for full text search).
Android supports all SQLite commands, so you can easily just use FTS3.
How is explained in the documentation linked above.
Example for creating a table:
CREATE VIRTUAL TABLE enrondata1 USING fts3(content TEXT); /* FTS3 table */
CREATE TABLE enrondata2(content TEXT); /* Ordinary table */
Query:
SELECT count(*) FROM enrondata1 WHERE content MATCH 'linux'; /* 0.03 seconds */
SELECT count(*) FROM enrondata2 WHERE content LIKE '%linux%'; /* 22.5 seconds */

Determine data type of a column in SQLite

I'm working on an Android App where the user has different options for sorting the displayed data that comes from the database. Currently my orderBy string that I pass to Androids query() method looks like this:
"LOWER("+columnName+") ASC"
The problem with this is that if the data type in the column specified by columnName is integer, calling LOWER() on it will cause it to be sorted alphabetically, i.e. based only on the leftmost digit, which of course doesn't make any sense for numeric data. Hence I only want to apply LOWER() if the data type of the column is not integer. What I have in mind is a statement like this:
"CASE WHEN [data type of columnName is integer] THEN "+columnName+" ASC ELSE LOWER("+columName+") ASC END"
The part in the brackets is what I don't know how to do. Does SQLite provide a function to determine a column's data type?

Do you really want the type of the column, or the type of the value? (SQLite is dynamically-typed, so the distinction is important.)
If you want the latter, you can use typeof(columnName).

Use:
PRAGMA table_info(table-name);
to get table info.

Taken directly from SQLite docs about datatypes for SQLite Version 3:
Most SQL database engines (every SQL database engine other than SQLite, as far as we know) uses static, rigid typing. With static typing, the datatype of a value is determined by its container - the particular column in which the value is stored.
SQLite uses a more general dynamic type system. In SQLite, the datatype of a value is associated with the value itself, not with its container. The dynamic type system of SQLite is backwards compatible with the more common static type systems of other database engines in the sense that SQL statements that work on statically typed databases should work the same way in SQLite. However, the dynamic typing in SQLite allows it to do things which are not possible in traditional rigidly typed databases.
Column affinity: use PRAGMA table_info(table-name);. PRAGMA table_info() gives a table with columns cid, name, type, notnull, dflt_value, and pk.
Columns in the result set include the column name, data type, whether or not the column can be NULL, and the default value for the column. The "pk" column in the result set is zero for columns that are not part of the primary key, and is the index of the column in the primary key for columns that are part of the primary key.
Datatype of value: Use typeof(column) to see how values are actually stored by SQLite.
Example adapted from section 3.4:
CREATE TABLE t1(
t TEXT, -- text affinity by rule 2
nu NUMERIC, -- numeric affinity by rule 5
i INTEGER, -- integer affinity by rule 1
r REAL, -- real affinity by rule 4
no BLOB -- no affinity by rule 3
);
-- Values stored as TEXT, INTEGER, INTEGER, REAL, TEXT.
INSERT INTO t1 VALUES('500.0', '500.0', '500.0', '500.0', '500.0');
-- Values stored as TEXT, INTEGER, INTEGER, REAL, REAL.
INSERT INTO t1 VALUES(500.0, 500.0, 500.0, 500.0, 500.0);
-- Values stored as TEXT, INTEGER, INTEGER, REAL, INTEGER.
INSERT INTO t1 VALUES(500, 500, 500, 500, 500);
-- BLOBs are always stored as BLOBs regardless of column affinity.
INSERT INTO t1 VALUES(x'0500', x'0500', x'0500', x'0500', x'0500');
-- NULLs are also unaffected by affinity
INSERT INTO t1 VALUES(NULL,NULL,NULL,NULL,NULL);
Output of PRAGMA table_info(t1);:
0|t|TEXT|0||0
1|nu|NUMERIC|0||0
2|i|INTEGER|0||0
3|r|REAL|0||0
4|no|BLOB|0||0
Output of SELECT typeof(t), typeof(nu), typeof(i), typeof(r), typeof(no) FROM t1; (notice each value in a column has its own datatype):
text|integer|integer|real|text
text|integer|integer|real|real
text|integer|integer|real|integer
blob|blob|blob|blob|blob
null|null|null|null|null

Did you declare the column as an integer when setting up the table? Otherwise sqlite will store it as text and the sorts will act as you've described.
create table if not exists exampletable (columnName integer);

To get information of Table use
PRAGMA table_info(table-name);
If you want the latter, you can use
typeof(columnName)

Categories

Resources