I have an Sqlite database insertion function
ContentValues values=new ContentValues();
values.put(CardTable.KEY_USERID, card_id);
values.put("id", some id);
values.put("namr", some name);
......
getContentResolver().insert(CardManagingProvider.CONTENT_URI_DETAIL, values);
My insert query is not working. Is it because one of the values in values is a long string (Base64-encoded form of an image)? Can any one please give me any suggestions?
You can see the length limits of SQLite here.
The maximum number of bytes in a string or BLOB in SQLite is defined by the preprocessor macro SQLITE_MAX_LENGTH. The default value of this macro is 1 billion (1 thousand million or 1,000,000,000). You can raise or lower this value at compile-time using a command-line option like this:
-DSQLITE_MAX_LENGTH=123456789
The current implementation will only support a string or BLOB length up to 231-1 or 2147483647. And some built-in functions such as hex() might fail well before that point. In security-sensitive applications it is best not to try to increase the maximum string and blob length. In fact, you might do well to lower the maximum string and blob length to something more in the range of a few million if that is possible.
You can determine if your Base64 exceeds those limits, however unlikely it may be.
I would recommend, however, not storing your Base64 string in the database. The larger your database is, the longer it takes for SQL statements to process. It's considered good practice to only store the URI of your image in the database, then use that URI to locate/load your image from disk/web.
Related
Interesting issue while using SQLite in Android. I am seeing an inconsistency in the string length and quoting of a string between what is stored in the database and the materialized value seen in Java.
We are using an ORM called SugarORM to query the DB, but I've traced the offending code to the internal android.database.sqlite.SQLiteCursor class used within SugarORM, specifically the cursor.getString(columnIndex) method.
I have a string in the database that is an ISO data string 2019-03-25T19:19:39.664Z and is stored in a VARCHAR column . I have confirmed using DB Browser for SQLite that the length of the string as its stored in the database is indeed 24 characters. SELECT LENGTH(MyStringColumn) FROM MyTable WHERE ...
When I get the value of this string via cursor.getString(columnIndex), it is returning the string "2019-03-25T19:19:39.664Z". Notice the leading and trailing quotes. Java reports to me that the string is 26 characters long.
Any value that I store in this column that is not an ISO data does not have this behavior. I tried tracing the SQLiteCursor source back, but ultimately it ends up being a Native method and that's where my skill set stops.
Can anyone explain what might be going on here? I am probably just going to write a wrapper around my queries to get rid of the quotes, but its all very perplexing. The date string is being fed to a JavaScript interpreter and causing it to fail when creating a JavaScript Date object.
If it helps, I have replicated the behavior on both my S7 physical device and a Pixel 6 emulator.
As a quick get around you could use :-
SELECT length(replace(mystringcolumn,'"','')) FROM mytable;
or before using the original SELECT use :-
UPDATE mytable SET mystringcolumn = replace(mystringcolumn,'"','');
If this doesn't fix the issue, then for some reason it is the code that retrieves the data that is at fault.
e.g. consider :-
DROP TABLE IF EXISTS mytable;
CREATE TABLE IF NOT EXISTS mytable (mystringcolumn VARCHAR);
INSERT INTO mytable VALUES('2019-03-25T19:19:39.664Z'),('"2019-03-25T19:19:39.664Z"');
SELECT length(mystringcolumn), length(replace(mystringcolumn,'"','')) FROM mytable;
which results in :-
i.e. The 2nd row, 2nd column retrieves the appropriate value by using the replace function to strip of the quotes, if they exist.
As to why the quotes exist could depend upon either the way that the data is inserted (perhaps you have inadvertenly coded the quotes but the db being looked at isn't the actual database as copied from the App) or the way in which the data is being retrieved that for some reason adds them.
I don't believe it likely that the Cursor getString method has a bug in which the quotes are added, otherwise such an issue would likely be a recurring issue.
I want to save image using my developed app into mobile what is the best way to it.
I was trying to save via sqlite but is there any other options?
The recommended way is to store the image as a file not in the database and then store the path or enough of the path to uniquely identify the image in the database.
To store an image you have to store it as a BLOB based upon a byte[] (byte array);
However, using the Android SDK, there is a limitation that only images less than 2MB can be retrieved even though they can be saved.
Technically you could store the image as a hex, binary, octal string but that would be more complex, less efficient and even more restrictive.. So really you have to store it as a BLOB is not completely factual.
It's also actually pretty useless storing images (or any large files) with SQLite or most structured databases as there is little that you can do with the data query wise.
You actually save a BLOB via SQL using something like :-
INSERT INTO your_table VALUES(x'00FF01EF');
i.e. 00 (0) is the first byte, FF (255) the 2nd, 01 (01) the 3rd .........
note the above would only work for a table with a single column
However, the SQLiteDatabase insert convenience method does the conversion from the byte[] to the correct SQL (hex string) on your behalf.
So you'd typically use :-
ContentValues cv = new ContentValues();
cv.put("your_column",your_image_as_a_byte_array);
your_sqlitedatabase_object.insert("your_table",null,cv);
You retrieve the byte array from a Cursor (result of a query) using something along the lines of
your_retrieved_image_byte_array = your_cursor.getBlob(your_column_offset_as_an_int); //<<<<<<<<< CursorWindow full error if the row cannot fit into the CursorWindow which has 2MB limit
You may wish to have a look at How can I insert image in a sqlite database, as this goes into more detail and has code that stores both images and paths (either one is stored based upon the image size, less then 100k, then the image is stored, else the path is stored) and additionally retrieves the images into a ListView. Additionally, although not recommended this is a way of getting around the 2Mb limit How to use images in Android SQLite that are larger than the limitations of a CursorWindow?
I have a sqlite BD with a table that contains coordinates of a large polyline (about 2000 characters). The structure of BD is:
CREATE TABLE "province" (
`_id` INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
`_line` TEXT,
);
The format of values of _line is:
lat1,lon1;lat2,lon2;lat3,lon3;lat4,lon4 ...
A little example:
28.164033,-15.709076;28.151925,-15.699463;28.134972,-15.703583;28.121650,-15.707703;28.107115,-15.704956;28.079250,-15.713196;28.067133,-15.735168
Right now, field _line is TEXT type on SQLite Android database. In my class DAO I parse this string to an ArrayList, thats is a list of points.
My question is, it will be recomended change of _line datatype from TEXT to BLOB? Will be improve the performance?
In the database itself, TEXT and BLOB values are stored in exactly the same way. The only difference is the type reported to the application, or the behaviour of the built-in string functions.
However, the Android framework will automatically convert TEXT values between the database encoding (UTF-8) and the Java string encoding (UTF-16). This does not happen for BLOB values.
If you store your values as blobs, you can read them out of the database in exactly the same format in which you stored them there, but you risk that that format is UTF-16, which would waste lots of space.
Is there any limit on how big the selection statement can be?
for example suppose I have 100 failed student as selection, will my below code work?
ArrayList<Long> ids_toupdate = getFailedStudents();// has size 100.
String selection = String.format(Locale.US, STUDENT._ID + " IN (%s)",TextUtils.join(", ", ids_toupdate));
ContentValues cv = new ContentValues();
cv.put(RawContacts.FAILED, 1);
getContentResolver().update(STUDENT.CONTENT_URI,cv, selection, null);
Roughly speaking the default limit is 1,000,000 bytes or 1,000,000 characters, so unless your 'students' are over 100,000 characters each your statement should be fine.
The following is taken from http://www.sqlite.org/limits.html
Maximum Length Of An SQL Statement
The maximum number of bytes in the text of an SQL statement is limited to SQLITE_MAX_SQL_LENGTH which defaults to 1000000. You can redefine this limit to be as large as the smaller of SQLITE_MAX_LENGTH and 1073741824.
If an SQL statement is limited to be a million bytes in length, then obviously you will not be able to insert multi-million byte strings by embedding them as literals inside of INSERT statements. But you should not do that anyway. Use host parameters for your data. Prepare short SQL statements like this:
INSERT INTO tab1 VALUES(?,?,?);
Then use the sqlite3_bind_XXXX() functions to bind your large string values to the SQL statement. The use of binding obviates the need to escape quote characters in the string, reducing the risk of SQL injection attacks. It is also runs faster since the large string does not need to be parsed or copied as much.
The maximum length of an SQL statement can be lowered at run-time using the sqlite3_limit(db,SQLITE_LIMIT_SQL_LENGTH,size) interface.
EDIT: 26/10/2022 As Tobias explains in his comment, the linked article now depicts the default max length as follows:
Maximum Length Of An SQL Statement
The maximum number of bytes in the text of an SQL statement is limited to SQLITE_MAX_SQL_LENGTH which defaults to 1,000,000,000.
There is also SQLITE_MAX_VARIABLE_NUMBER which limits the number of variables in a query. The default value is 999 but some distributions are compiled with higher settings.
I have two columns in my SQLite Database, name and score. I need to display the out put of all table records by score descending from highest value. I have this working at the moment but due to score being a String, of course it only goes by the first number in the string, so 30 is above 200 etc...
Here is my SQL code:
private static final String fields[] = { "name", "score", BaseColumns._ID };
Cursor data = database.query("scores", fields, null, null, null, null, "score DESC");
I have no idea how I can continue to use the above code as well as converting all of the score values to integers in order to sort by highest score first. I started to do this by converting each score value into an integer and store it in an array in order to sort them, but then I only have half of my information, so this was a bad idea, I had not thought through.
I spent about an hour reading the SQLite documentation in order to seek some way of doing this more efficiently as well as scour Stackoverflow, but to no avail. Can anyone provide advice on how I should proceed to do this?
This answer solves the problem by casting the strings as integers in the query. This is better than doing it in the program because the database is built for storing and sequencing large amounts of data, so it is much more efficient to change the query than the code.
I think the right answer is to store them as their correct type, since you only do that once when you INSERT. Why reformat every time you query and display? Makes no sense to me.