Develop Reference

Android 9(only observed) using Room(for non-images): Row too big to fit into CursorWindow requiredPos=0, totalRows=1

I've only observed this on Android 9 and possibly only on Samsung devices. I'm storing multiple JSON responses to multiple serialized strings into my DB later to be typeConverted using Moshi again into a model.
The query that causes this error is:
#Query(“SELECT * FROM tasks”)
public abstract Flowable<List<TaskEntity>> getAll();
The last instance had a total of about 392,000 characters TOTAL in the table. These are actually split up into about 5500 character size entries within the table.
Why would the cursor have a problem with ~11k byte sized entries? Does the fact that I'm selecting * mean the cursor is grabbing the whole table into memory and not a single row at a time?
Why only Android 9?
Thanks.

Does the fact that I'm selecting * mean the cursor is grabbing the whole table into memory and not a single row at a time?
SELECT * means you are retrieving all columns. A SELECT without a WHERE clause (or other types of constraints) means that you are retrieving all rows. So, SELECT * FROM tasks will attempt to retrieve the entire table contents into memory.
You could add #Transaction to this function, as that may help get past this error. Quoting the documentation:
When used on a Query method that has a SELECT statement, the generated code for the Query will be run in a transaction. There are 2 main cases where you may want to do that:
If the result of the query is fairly big, it is better to run it inside a transaction to receive a consistent result. Otherwise, if the query result does not fit into a single CursorWindow, the query result may be corrupted due to changes in the database in between cursor window swaps.
If the result of the query is a POJO with Relation fields, these fields are queried separately. To receive consistent results between these queries, you also want to run them in a single transaction.
Even better would be to not load the entire table's content's into memory (and then convert the entire table's rows into entity objects). Heap space is limited.
Why only Android 9?
No clue. I wouldn't worry about that — if you focus on retrieving less data, that will have benefits for all your users.

How can I select the last index of a column split with bigquery

There are a lot of questions about splitting a BigQuery, MySQL column, but I can't find one that fits my situation.
I am processing a large dataset (3rd party) that includes a freeform location field to normalize it for my Android app. When I run a select I'd like to split the column data by commas, take only the last segment and trim it of whitespace.
So far I've come up with the following by Googling documentation:
SELECT RTRIM(LOWER(SPLIT(location, ',')[OFFSET(-1)])) FROM `users` WHERE location <> ''
But the -1 trick to split at last element does not work (with either offset or ordinal). I can't use ARRAY_LENGTH with the same array inline and I'm not exactly sure how to structure a nested query and know the last column index of the row.
I might be approaching this from the wrong angle, I work with Android and NoSQL now so I haven't used MySQL in a long time
How do I structure this query correctly?

I'd like to split the column data by commas, take only the last segment ...
You can use below approach (BigQuery Standard SQL)
SELECT ARRAY_REVERSE(SPLIT(location))[SAFE_OFFSET(0)]
Below is an example illustrating it:
#standardSQL
WITH `project.dataset.table` AS (
SELECT '1,2,3,4,5' location UNION ALL
SELECT '6,7,8'
)
SELECT location, ARRAY_REVERSE(SPLIT(location))[SAFE_OFFSET(0)] last_segment
FROM `project.dataset.table`
with result
Row location last_segment
1 1,2,3,4,5 5
2 6,7,8 8
For trimming - you can use LTRIM(RTRIM()) - like in
SELECT LTRIM(RTRIM(ARRAY_REVERSE(SPLIT(location))[SAFE_OFFSET(0)]))

To get the last part of the split string, I use the len(string) - len(replace(string,delimeter,'')) trick to count the number of delimiters:
split(<string>,'-')[OFFSET(length(<string>)-length(replace(<string>,'-',''))]

Natural sorting of alphanumeric values in sqlite using android

I have a list of names of starts with characters and end with numbers like: -
ka1, ka10, ka 2, ka, sa2, sa1, sa10, p1a10, 1kb, p1a2, p1a11, p1a.
I want to sort it in natural order, that is: -
1kb, ka, ka1, ka 2, ka10, p1a, p1a2, p1a10, p1a11, sa1, sa2, sa10.
The main problem I am seeing here is no delimiter between text and numeric part, there also a chance of without numeric part also.
I am using sqlite in android, I can do sorting using java after fetching points by cacheing cursor data, but I am using(recommended to use) cursor adapter.
Please suggest a query for sorting or is there any way to apply sorting in cursor?

I tried below query for Natural sorting:
SELECT
item_no
FROM
items
ORDER BY
LENGTH(item_no), item_no;
It worked for me in Sqlite db too. Please see this link, for more details.

I can propose using regex replacement adding zeros, creating temporary table of original and corresponding values, then follow this link for sorting it: http://www.saltycrane.com/blog/2007/12/how-to-sort-table-by-columns-in-python/
tip for regex add as many zeros after last letter, but limit the number of total digits for predicted maximum number of digits. If You need help with regex as well, provide exact info of valid and invalid values, so can halp with that too.
PS if want to be sure that zeros goes before last digits search for char from the end

Updated
You can use different ways - Some of are mentioned below:
BIN Way
SELECT
tbl_column,
BIN(tbl_column) AS binray_not_needed_column
FROM db_table
ORDER BY binray_not_needed_column ASC , tbl_column ASC
Cast Way
SELECT
tbl_column,
CAST(tbl_column as SIGNED) AS casted_column
FROM db_table
ORDER BY casted_column ASC , tbl_column ASC
or try the solution:
There are a whole lot of solutions out there if you hit up Google, and
you can, of course, just use the natsort() function in PHP, but it's
simple enough to accomplish natural sorting in MySQL: sort by length
first, then the column value.
Query: SELECT alphanumeric, integer FROM sorting_test ORDER BY LENGTH(alphanumeric), alphanumeric from here

Declaring SQLite table for Android word game with 700 000 words

For an Android word game (with minSdkLevel=9 meaning SQLite version 3.6.22) -
I would like to deliver the dictionary as a prefilled SQLite table within the APK file (with the help of SQLiteAssetHelper).
In the SQLite database there will be just 1 table:
create table dict ( /* contains 700 000 unique words */
word text not null
);
My question please:
How to declare the table for the best performance and which kind of SQL-query to use?
(When checking if a word entered by player is present in the dict table or not - that will be the main usage of the SQLite database in the app).
Should I create index (is it possible to have index for text columns at all)?
Or should I declare the word column as primary key?
Also, some SQLite for Android guides suggest to have an _id column in each table (probably to enable fetching the last inserted record? - which I don't really need here). Should I maybe use
create table dict (
_id integer primary key,
word text unique not null
);
create index word_index on dict(word);
or will that be a waste of 4 x 700 000 bytes? (Or is it added as _rowid_ anyway?)

Quick answer: yes, you can create index on text column.
However for best performance, this may not be the best option.
Because the index created by SQLite should be simply a b-tree (binary tree), which speed up the search by binary search. i.e. with 700k words, the binary search has to run about 20 intervals. But this could be fast enough, you need to test it to actually know the performance.
Some alternative methods would be to create multiple tables (buckets), e.g. create table as wordA, wordB, wordC etc.
And use the first character to determine which table the word is put.
This drops the size of each table to contains about 27k records. (of course each bucket is not of equal size)
By doing this, it reduces the interval used performing the binary search.
And actually you should use hash function to determine the bucket, which makes the size of each buckets more balanced and you can freely control the number of buckets.
And you have to actually fine tune to know what is the optimal bucket size.

Android Cursor maximum size

It appears that Cursors in android can only hold up to 1 MB of data.
What would be the most efficient way to pull the maximum number of rows from a table in a SQLite database that stays under the 1 MB limit?

I don't think there's a hard and fast way to determine the right limit, but I looked in the CursorWindow documentation and found that copyStringToBuffer (int row, int column, CharArrayBuffer buffer) seems promising as CharArrayBuffer has a integer field called sizeCopied and the method copyStringToBuffer takes the text at the specified row and column. Maybe you can get the size from the buffer and add for each row and column you have? If you're using SQLiteCursor, you can use setWindow(CursorWindow window) to set your own window.

You can put LIMIT clause, so that rows are fetched in parts.