Develop Reference

Using Like in Sqlite with android for searching in encrypted data

I have a table (Table1) in SQLite with one column (Col1), this table has 100,000 rows that all values in Col1 are encrypted with special algortith.
I've used select sql ... like command in Android, Like this:
Select Col1
from Table1
where Col1 like 'A%';
I want to return all rows that started with 'A' letter.
But actually Cal1 is encrypted!! even if I use this:
"Select Col1 from Table1 where Col1 like '"+my_method_encryption("A")+"%';" .. it will be wrong, becuase may the Encrypted values of 'A' letter in Col1 has different value with return value of my_method_encryption("A").
What should I do?
Actually There is another way to solve it, if I select all 100,000 rows and after that I will decrypt all 100,000 rows and then search. But this way will be so slow becuase maybe I will need to use this select ... like more than 10 times.
Thanks

Encrypt the database file containing the plain-text column and table key. Make and link by key a separate db file for non-encrypted data.
Decrypt the file on open, making searches possible and join the other data based on the key.

where Col1 like '"+my_method_encryption("A")+"%';"
That won't work, it's in the purpose of encryption that you cannot tell from the encrypted text what the first character(s) of the plaintext are, otherwise encryption would be meaningless.
What you could do is move your decryption function to SQLite as an user function (https://www.sqlite.org/appfunc.html)
The use something like:
where my_decrypt(Col1,'key') like 'A%'"
But this would be identical to what you wrote:
Actually There is another way to solve it, if I select all 100,000 rows and after that I will decrypt all 100,000 rows and then search.
This is what SQLite will do, just that internally.
However, what you are trying to achieve seems to be conceptually wrong; namely:
your encryption scope is a row:column
you want to query the encrypted data based on what usually requires an external aggregate structure (an index). It's the only way to get a sub-linear search performance.
You should consider expanding your encryption scope to the whole table; for instance SQLite provides https://www.sqlite.org/see/doc/trunk/www/readme.wiki which blocklevel-encrypts the whole database (database scope, block unit). You can also logically join an unencrypted db with an encrypted db; using ATTACH you bring both dbs into the same scope then use a normal JOIN, maybe even in a view, to bring the data together.
I'm not familiar with the Android ecosystem but a simple search for "android sqlite encryption extension" reveals that there is no shortage of alternatives for DB-level encryption.

Best way to enter complex table to sqlite database

I have some kind of this table.
The question is what is the best way to create this kind of table?
Should I create for each item one table is it possible to create only one table??

Updated: See comments under #Emil.
You should have 1 tables as #Emil has suggested.
This should look like, soemthing like
_id, sort, grade, diameter, length, price1_dol, price1_euros, price2_dol, price2_euros, final,
Note: I have split up prices columns up - so you have price1_dol, price1_euros, price2_dol, price2_euros.

It is indeed possible to make this data into just one table. The columns sort and grade seem to uniquely identify one row so together they might make up a candidate key. If so you could use those as your primary key, or create a new integer column that you use as the primary key.
You should definitely not create one table per item. The database schema should never change with normal use. Only when you add, remove or change the type of data you have in your database should you consider changing the schema. Otherwise you should design and normalize your database in such a way that it's possible to grow the data only by inserting new rows, not new tables.

Changing a Column type in my Android App's Database

I have an app published in the play store.
The app uses a database which holds a table which has a column of type int.
I'm doing a new change where I need to change the column type to long.
How do I go about handling it in the DatabaseHandler I'v created.
I want to preserve the data stored in the older apps database, so what should ideally be the code in the onUpgrade() function???

You don't need to change the database column type. An INTEGER column will happily contain all the bits needed to represent a Java long.
In fact, there's no long column type in sqlite.

I think using SQLite, the best way is to create a temporary table, copy all your table content, drop the old table and recreate the table with the right type on your column, then you can just copy the content from the temporary table and drop it...
I know this don't fell like the best approach, but I don't think SQLite have some alter table function.

As far I know you can t do this . But You can drop your table if it exists and create it again . Maybe you can find out some useful information here SQLite Modify Column or here Modify a Column's Type in sqlite3

What is the advantage of FTS over custom solution?

I have a biggish database ~32mb which has lots of text in 4 languages. Including Arabic and Urdu. I need to search this text in the most efficient way (speed & size).
I am considering FTS, and trying to find out how to implement it. Right now I am reading http://www.sqlite.org/fts3.html#section_1_2 about it.
It seems to me, an FTS table is just like a normal table used to index all the different words. So my questions are:
1) If to populate FTS I have to do all the inserts myself, then why not make my own indexed word table, what is the difference?
Answer : Yes there are many advantages, many built in functions that help. For example with ranking etc, searching of stems and the transparent nature of how it all works in android makes the FTS approach more appealing.
2) On the google docs I read its a virtual in memory table, now this would be massive right... but it doesnt mention this on the SQLite website. So which is it?
3) Is there an easy way to generate all the different words from my columns?
4) Will the FTS handle arabic words properly?

FTS allows for fast searching of words; normal indexes only allow to search for entire values or for the beginning of the value.
If you table has only one word in each field, using FTS does not make sense.
FTS is a virtual table, but not an in-memory table.
You can get individual terms from the full-text index with the fts4aux table.
The default tokenizer works only with ASCII text.
You have to test whether the ICU or UNICODE61 tokenizers work with your data.

1) If to populate FTS I have to do all the inserts myself, then why
not make my own indexed word table, what is the difference?
Using your own indexed word table, you would have parse words in sentences. You would then need a table for sentences and another to words. And you should do this efficiently.
2) On the google docs I read its a virtual in memory table, now this
would be massive right... but it doesnt mention this on the SQLite
website. So which is it?
Don't understand your question. Data is handled via virtual table extension, however back storage is done in database (FTS4 creates 5 tables for each virtual table). Check this:
sqlite> CREATE VIRTUAL TABLE docs USING fts4();
sqlite> .schema
CREATE VIRTUAL TABLE docs USING fts4();
CREATE TABLE 'docs_content'(docid INTEGER PRIMARY KEY, 'content');
CREATE TABLE 'docs_segments'(blockid INTEGER PRIMARY KEY, block BLOB);
CREATE TABLE 'docs_segdir'(level INTEGER,idx INTEGER,start_block INTEGER,leaves_
end_block INTEGER,end_block INTEGER,root BLOB,PRIMARY KEY(level, idx));
CREATE TABLE 'docs_docsize'(docid INTEGER PRIMARY KEY, size BLOB);
CREATE TABLE 'docs_stat'(id INTEGER PRIMARY KEY, value BLOB);
sqlite>
3) Is there an easy way to generate all the different words from my
columns?
For sure. But that's not easy. That's what FTS does.
4) Will the FTS handle arabic words properly?
I'm not sure. Does arabic languages uses ICU word boundaries? From Tokenizer:
The ICU tokenizer implementation is very simple. It splits the input
text according to the ICU rules for finding word boundaries and
discards any tokens that consist entirely of white-space. This may be
suitable for some applications in some locales, but not all. If more
complex processing is required, for example to implement stemming or
discard punctuation, this can be done by creating a tokenizer
implementation that uses the ICU tokenizer as part of its
implementation.

Need a good method to change (SQLite) column data type

I am having a table with 'int' column. During software upgrade, I want to change it to 'long' data type. It seems SQLite does not provide an option (in alter statement) to change/modify column properties. Since I want to do an upgrade, the table might contain data which the user should not lose. So please suggest me a good method to change the column data type property without data loss.
One way suggested in the search links is to create a temporary table, copy the records from the existing table, delete the existing table, and rename the temporary table. I doubt that is efficient.
Your help appreciated!
Regards
Vivek Ragunathan

I used the follow statements to change the type of the column.
CREATE TABLE IF NOT EXISTS **TEMP_TABLE** (id integer primary key autoincrement, **col2change integer not null**, ...)
INSERT INTO TEMP_TABLE SELECT * FROM EXISTING_TABLE
DROP TABLE EXISTING_TABLE
ALTER TABLE TEMP_TABLE RENAME TO EXISTING_TABLE
I changed the int type column in the existing table to integer type. For a few hundred rows, it was reasonably fast.

SQLite3 columns do not have data types, only affinities -- there is no benefit in changing the column type from int to long.
If a wrapper program is truncating values before giving them to SQLite3, there is a way to fix this by editing the schema. It is a dangerous operation, so do it only after backing up your database. The schema is stored in the table sqlite_master; normally it is read-only, but you can modify it if you enable it with the writable_schema pragma. Be careful to only make changes that do not invalidate your data; you may change int to long int since they both have INTEGER affinity.

From SQLite documentation
It is not possible to rename a column, remove a column, or add or
remove constraints from a table.
Check this link
Please do remember that column data types are not rigid in SQLite. Check this link
Edit:
Following your comments on another answer, I guess the option you mentioned - working through the temp table - is the only one, which is not efficient off course.

you could add a new colum, copy the values form the old to the new column, delete the old column and then rename the new column to the old name

AFAIK there is no way in Android to change column data types once a table is created and used. The practiced way is to make a new table and copy the data which you read about