In the SQLite database, I have stored all values in uppercase.
How can I select the specified value in the database using lower case?
For anyone else that arrived here from Google looking how to select as upper case in SQLite, as expected, this will work:
sqlite> SELECT UPPER("Hello, WORLD!");
HELLO, WORLD!
https://sqlite.org/lang_corefunc.html#upper
SQLite has a LOWER function for this:
sqlite> SELECT LOWER("Hello, WORLD!");
hello, world!
The lower(X) function returns a copy of string X with all ASCII characters converted to lower case. The default built-in lower() function works for ASCII characters only. To do case conversions on non-ASCII characters, load the ICU extension.
Related
I am coding a dictionary project. Its can translate english to arabic or arabic to english. Words are stored in sqlite fts4 database.
Arabic letter in database stored html encoded like
غير
When i use fts4 query syntax in english to arabic for example => stor
SELECT * FROM fts_dic WHERE english MATCH '"^stor*"';
Returned results are good for me like
store
stored
storage
But when i search arabic to english for => غير
SELECT * FROM fts_dic WHERE english MATCH '"^غير*"';
Returned results
ظغير
֎غيرظ
But i want to see result only start with my searched html encoded text like
غيرخ
غيرٗ
I use "^" at the begining of the word to get this result as you see. In english to arabic works fine but arabic to english not working properly.
The FTS documentation says:
A term is a contiguous sequence of eligible characters, where eligible characters are all alphanumeric characters and all characters with Unicode codepoint values greater than or equal to 128. All other characters are discarded when splitting a document into terms. Their only contribution is to separate adjacent terms.
In other words, punctuation characters like &#; are completely ignored; what FTS sees are the three words 1594, 1610, and 1585.
In the FTS table, you should not HTML-encode anything; just use the plain Unicode characters.
Furthermore, ^ works only in FTS4 tables (which may not be available in all Android versions).
I'm trying to filter out accented words if user searches for them in local database. But I have problems, namely with slavic letters ČŠŽ. In my SQLite database I have a field "title" with value: "Želodček"
If I try to select LOWER(title) I always get back the same value "Želodček" whilst other words are correctly lower cased. Only if the word begins with ČŽŠ then it doesn't get lower cased. This only persists with words which have leading accented letters.
Database records
Stomach
Želodček
Uppercase with UPPER()
STOMACH
ŽELODčEK
Lowercase with LOWER()
stomach
Želodček
I've already tried setting localization with setLocale() with no luck. I also tried different collation like NOCASE, UNICODE, LOCALIZED but nothing worked. I'm wondering why when lower cased the first letter is not lower cased and when upper cased other accented words are lowercase.
I've solved the problem with LIKE searches where I replace accented words with their lower cased counterpart. But I have problem with full text(FTS3) searching because I can't use the same trick with MATCH.
-- works but it's a hack
SELECT title FROM articles WHERE REPLACE(LOWER(title),'Ž','ž') LIKE '%želodček%'
-- can't seem to get it work
SELECT title FROM articles WHERE title MATCH 'želodček' COLLATE NOCASE
Is there any solution to this or is there a bigger problem?
Update:
No optimal solution yet.
Un-optimal solution 1:
I decided to deal with the problem directly by changing data in the select query. While this doesn't work for all cases (and I would have to cover all accents) it suits my case for now. So I'm posting it:
-- LIKE query
SELECT title FROM articles WHERE (REPLACE(REPLACE(REPLACE(LOWER(title),'Č','č'),'Š','š'),'Ž','ž') LIKE ? COLLATE NOCASE))
-- MATCH query (FTS)
-- In this case I programmatically replace searched word with 2 word variation (one that starts with lowercase and one that starts with uppercase) ie: title='želodček OR Želodček'
SELECT title FROM articles WHERE title MATCH ? COLLATE UNICODE
Un-optimal solution 2:
As suggested by user CL. to insert in normalized form (didn't work for me because normalized form was basically the original unicode form). I took it futher and insert title stripped of of accents (basically ASCII form). This is maybe better than solution one in ways of general solution. Since I only cover some accents in the first.
But there are downsides:
data doubles (one unicode title and one ASCII title). Which can be a problem if you have a lot of data.
some characters are not supported (like chinese characters will be gone after normalization and stripping)
ambiguity which you get by stripping accents (ie. two words "zelo" and "želo" have different meanings but will both turn up when searching).
Here's the Java code for it:
// Gets you the ASCII version of unicode title which you insert into different column
String titleAsciiName = Normalizer.normalize(title, Normalizer.Form.NFD)
.replaceAll("[^\\p{ASCII}]", "");
LIKE never uses a custom collation.
FTS can use a custom tokenizer, but you have to check whether unicode61 is available in all Android versions you want to support.
The Android database API does not allow to create custom implementations of LIKE or of a FTS tokenizer.
You might want to store a normalized version of your strings in the database.
Actually question was asked several times, but I didn't manage to find answer.
There's set of SQLite table(s) which are read-only - I can't change their structure or redefine collation rules. Tables consisting some international characters (Russian/Chinese, etc).
I would like to get some case-insensitive selection like:
select name from names_table where upper(name) glob "*"+constraint.toUpperCase()+"*"
It works only when name is latin/ASCII charset, for international chars it doesn't work.
SQLite's manual reads:
The upper(X) function returns a copy of input string X in which all
lower-case ASCII characters are converted to their upper-case
equivalent.
So the question is: how to resolve this issue and make international chars in upper/lower case?
This is known problem in sqlite. You can redefine built-in functions via Android NDK. This is not a simple way. Look at this question
Notice that indexes of your tables will not work (for UDF) and query can be very slow.
Instead of it you can store your data (which you look for) in other column in ascii format.
For example:
"insert into names_table (name, name_ind) values ('"+name+"',"+"'"+toAsciiEquivalent(name)+"')"
name name_ind
----------------
Имя imya
Name name
ыыы yyy
and search string by column name_ind
select name from names_table where name_ind glob "*"+toAsciiEquivalent(constraint)+"*"
This solution requires more space for data, but it is simple and fast.
Instead of providing full Unicode case support by default, SQLite provides the ability to link against external Unicode comparison and conversion routines. The application can overload the built-in NOCASE collating sequence (using sqlite3_create_collation()) and the built-in like(), upper(), and lower() functions (using sqlite3_create_function()). The SQLite source code includes an "ICU" extension that does these overloads. Or, developers can write their own overloads based on their own Unicode-aware comparison routines already contained within their project.
Reference: http://www.sqlite.org/faq.html
How I should write my sql query in android that special characters work correct. For example when I filter String with like which has % symbol, it doesn't work correct.
For LIKE, you choose in the query what to escape them with;
SELECT * FROM Olle WHERE name LIKE 'Many ^% have fallen' ESCAPE '^';
...will only match the actual character % since it's been escaped with the ^ that is give in the ESCAPE clause.
SQLfiddle here.
I am trying to create a database for an android app including, in part, non-English words which require underlines and accents for proper spelling. I set my encoding for this package to utf-8, which allowed the accented characters to store and display properly. However, I cannot seem to get a single character underlined. It displays an empty box for an unrecognized character.
An example of my database helper to create the sqlite is as follows:
cv.put(ENGLISH, "to be alive");
cv.put(NATIVE, "okch_á_a or okchaha");
cv.put(PART_OF_SPEECH, "verb");
cv.put(AUDIO, "alive");
cv.put(VIDEO, "none");
cv.put(IMAGE_DEFAULT, "none");
cv.put(IMAGE_OPTIONAL, "none");
cv.put(IMAGE_TO_USE, "none");
db.insert("words", ENGLISH, cv);
That
_ a _
is the best I can come up with so far, but the a should actually be an underlined character.
I tried html tags like u and /u:
<u>a</u>
since that works with string arrays, but it displays as:
<u>a</u>
(the html is never interpreted).
I tried using:
"\u0332"
as explained at http://www.fileformat.info/info/unicode/char/332/index.htm , but that, too, is never interpreted, so it displays as:
a\u0332
I also tried:
& # 818 ;
and:
& # x332 ;
in a similar manner, with similar lack of results.
Any ideas?
You can store your string in Html format and call .setText(Html.fromHtml(somestring)) from the textview were you want to display it.