I have a ContentProvider that uses a custom CursorFacory in debug to print out the SQL queries (for debugging).
A certain query was returning 0 rows, while I knew there were rows that should have been included. So I copied the query from my logs, replaced the bind values and ran it in sqlite3 shell on the device and got the correct result.
The Query Code
cr.query (contentUri,
Projection.columns,
FeedColumns.FEED_TYPE + "=? AND " +
FeedColumns.SUB_TYPE + "=? AND " +
ProfileUpdateFeedItem.UPDATED_FIELD + "=? AND " +
FeedColumns.IS_NOTIFIED + "=?",
new String[] {FeedType.USER, // 2
WallPostData.WallPostType.PROFILE_UPDATE, // 1
ProfileUpdateData.ProfileField.STATUS, // 0
SQLBoolean.FALSE // 0
},
FeedColumns.CREATED + " ASC");
From the logs:
07-04 12:48:51.339 4067-4314/com.redacted.android D/DATABASE﹕ QUERY: SQLiteQuery: SELECT DISTINCT id, sender, data_1, data_2, photo, feed_type, sub_type, created, expiry, updated, comment_count, comment_unread, reaction_count, reaction_unread, sender_name, sender_photo, _id FROM wall WHERE feed_type=? AND sub_type=? AND data_1=? AND is_notified=? ORDER BY created ASC LIMIT 100
On device:
Enter SQL statements terminated with a ";"
sqlite> SELECT DISTINCT id, sender, data_1, data_2, photo, feed_type, sub_type, created, expiry, updated, comment_count, comment_unread, reaction_count, reaction_unread, sender_name, sender_photo, _id FROM wall WHERE feed_type=2 AND sub_type=1 AND data_1=0 AND is_notified=0 ORDER BY created ASC LIMIT 100;
53b702b827d7482062f52b03|a7e759d78abe4bfa97045ce49a24ab57|0|Educ||2|1|1404502712279|1404761912325|1404502712279|||||Luke Skywalker|pr/e5c2c0398b267f93683c80dc5009722e|49
The ContentProvider, however, doesn't agree and cursor.getCount() returns 0.
Any ideas why this is happening?
feed_type, sub_type, and is_notified are INTEGER columns.
data_1 is a BLOB that is storing an integer for any row that would qualify for this query, but stores strings for other types of data that could go in this table.
When you run in the shell i'm surprised you get any rows. The blob data type may not convert the keyed value properly for you. Typically the database API requires a special function to set the blob value as well as retrieve it.
So the problem here was the BLOB column. It was being evaluated properly in queries (The data in the table is used in a ListView and is displayed differently depending on the contents of the data_1 and data_2 columns).
Everything in the feed category gets parsed into a member of a class hierarchy rooted at an AnstractFeedObject.
Most fields that use both data_1 and data_2 store text in both, but some fields (those who correspond to a subset of the mentioned class hierarchy) use data_1 as a type enumeration that the UI uses to interpret the value stored in data_2. For example, a 0 type means that data_2 is a picture id (construct the url and download), while type 1 means it's actual text content.
What I ended up doing was that I replaced data_1 with an integer column called type_enumeration and renamed data_2 to data_1. Now that I know BLOB can cause those kinds of issues, I'll be changin data_2 also to a TEXT column.
If at some point in the future I need to store binary data in the DB, I'll add a bin_data to the column.
Now usually in a proper normalized schema you'd use linked tables to represent such hierarchy, but in a mobile environment, you want to minimize joins so a few extra columns are cheaper in terms of performance (at least that's been my experience).
Related
I have a value from the epoch time like this 1549251913000, I save this value to SQLite. I create the table like the following:
CREATE TABLE TABLE_BOOKMARK (COLUMN_ID INTEGER PRIMARY KEY AUTOINCREMENT, COLUMN_TITLE TEXT, COLUMN_SOURCE TEXT, COLUMN_DATEANDTIME INTEGER, COLUMN_GUID TEXT);
that value is COLUMN_DATEANDTIME but with INTEGER type. but when I take the value, it doesn't match what I expected. it becomes like this -1231280856.
Please give me some advice, thanks
I have tried this solution but still mismatch when I get it from SQLite (seems it didn't work with my problem)
Create a column in as INTEGER datatype put LONG value into INTEGER in column
Make sure when you retrieve the value from cursor as LONG
Cursor cursor = db.rawQuery("SELECT * FROM " + TABLE_NAME, null);
long value = cursor.getLong(0);
For more : SQLite DataType Doc
Just to clarify the column type is, with one exception, largely irrelevant as any type of data can be stored in any type of column.
The exception is the rowid column or an alias of the rowid column, such a column MUST store an integer. By integer it is a 64bit signed integer and thus encompasses a java long.
The column type itself is also flexible for example CREATE TABLE mytable (mycolumn a_pretty_weird_column_type) is valid (as would be LONG). Such types are converted according to 5 rules to the column affinity.
Putting the above together, using :-
CREATE TABLE IF NOT EXISTS mytable (mycolumn a_pretty_weird_column_type);
DELETE FROM mytable;
INSERT INTO mytable VALUES
(1549251913000),
(999999999999999),('Fred'),(x'010203040506070809'),(0.234567),(null),
('999999999999999'), -- Note although specified as TEXT as mycolumn is effectively NUMERIC stored as INTEGER
('0.234567') -- As above but stored as REAL
;
SELECT
*,
typeof(mycolumn) AS coltype, -- The column type (note as per value not column definition)
hex(mycolumn) AS as_hex, -- Convert column to a hex representation of the data
CAST(mycolumn AS TEXT) AS as_text, -- follow rules
CAST(mycolumn AS INTEGER) AS as_integer,
CAST(mycolumn AS REAL) AS as_real,
CAST(mycolumn AS NUMERIC) AS as_numeric,
CAST(mycolumn AS BLOB) AS as_blob
FROM mytable;
results in :-
The bible as such is Datatypes In SQLite Version 3.
but when I take the value, it doesn't match what I expected. it
becomes like this -1231280856
The Issue
As such your issue is nothing to do with the column type, not SQLite as such, rather it's due to using the Cursor getInt method, instead of the getLong method.
One answer says,
Its always better to store date and time in form of Text in sqlite.
This is incorrect, from a space point of view and therefore the underlying efficiency the use of a numeric representation, i.e. 64bit signed integer as per SQL As Understood By SQLite - Date And Time Functions - Time Strings (maximum of 8 bytes), of the date and time will be more efficient than storing the 19 bytes (for an accuracy down to a second).
As per the docs sqlite does not have default storage class for date and time. Its always better to store date and time in form of Text in sqlite. You can then parse them into Date runtime whenever you want to use them
How can I find the tables having column Blob type in Sqlite. I need to get the table names from which I get the column blob type and then want to see the total no. of records where the blob is not empty.
If you wanted tables that have a column defined as a blob then you could use
SELECT * FROM sqlite_master WHERE sql LIKE '%blob%';
as the basis for determining the tables. e.g. this could return results such as :-
However, this does not necessarily find all values that are stored as blobs. This is because with the exception of the rowid column or an alias thereof, any type of value (blob included) can be stored in any column.
e.g. consider the following :-
DROP TABLE IF EXISTS not_a_blob_table;
CREATE TABLE IF NOT EXISTS not_a_blob_table (col1 TEXT, col2 INTEGER, col3 REAL, col4 something_or_other);
INSERT INTO not_a_blob_table VALUES
('test text',123,123.4567,'anything'), -- Insert using types as defined
(x'00',x'12',x'34',x'1234567890abcdefff00') -- Insert with all columns as blobs
;
SELECT typeof(col1),typeof(col2),typeof(col3),typeof(col4) FROM not_a_blob_table;
This results in :-
If you want to find all blobs then you would need to process all columns from all rows of all tables based upon a check for the column type. This could perhaps be based upon :-
SELECT typeof(col1),typeof(col2),typeof(col3),typeof(col4),* FROM not_a_blob_table
WHERE typeof(col1) = 'blob' OR typeof(col2) = 'blob' OR typeof(col3) = 'blob' OR typeof(col4) = 'blob';
Using the table above this would result (only the 2nd row has blobs) in :-
A further complication is what you mean by not empty, null obviously. However what about x'00'? or if you used a default of zeroblob(0) ?.
zeroblob(N)
The zeroblob(N) function returns a BLOB consisting of N bytes of 0x00. SQLite manages these zeroblobs very efficiently. Zeroblobs can
be used to reserve space for a BLOB that is later written using
incremental BLOB I/O. This SQL function is implemented using the
sqlite3_result_zeroblob() routine from the C/C++ interface.
If null though then this wouldn't have a type of blob, instead it's type would be null, which could complicate matters if checking for all values stored as blobs.
You may wish to consider having a look at the code from Are there any methods that assist with resolving common SQLite issues?
as this could well be the basis for what you want.
You also wish to have a look at typeof(X) and zeroblob(N).
Conclusion: Android's database APIs work but the documentation is horribly incomplete.
I have recently run into a brain wrecking situation due to the flexibility Sqlite provides by not forcing you to specify the data type when creating the table. The problem was my mindset that assumed that every data type would be a general character sequence if not specified and therefore the way to talk to database is through java.lang.String.
But you can't blame me either when you see methods like the below:
int delete (String table,
String whereClause,
String[] whereArgs)
in the SqlDatabase class from Android docs.
I have a table consisting of Phone No(that I stored as java.lang.String) and Timestamp as a long field. When I tried deleting a record using this method, it just never got deleted despite countless debugging.
I checked everything and query was alright and table is existent and all the checklist until by chance, I discovered that removing the '' around the timestamp while querying in a raw manner instead of using the above method yields a successful deletion, something like this:
DELETE FROM messages_records_table WHERE messageTimestamp = 1508494606000;
instead of the following:
DELETE FROM messages_records_table WHERE messageTimestamp = '1508494606000';
or,
DELETE FROM messages_records_table WHERE messageTimestamp = "1508494606000";
Phone No isn't a problem; it's the timestamp that was creating the problem in INSERTION/DELETION
So, I tried running a raw deletion query with quotes removed(that are required with a string/varchar type) and it yielded successful deletion. I used the following method for this:
db.execSQL(String sql, Object[] whereArgs)
The key thing to notice here is that Object[] is different from String[] when compared to delete(). I passed a Long to Object to make it work but passing a Long.toString() in delete() seems to be useless.
So my question is, Is my analysis correct and delete() API is basically useless or have I missed some bigger picture..after all, it's provided by Android team carefully?
SQLite supports multiple data types; and while column types are not strictly enforced, values might be automatically converted in some cases (this is called affinity).
When your values are stored as numbers, you should access them as numbers, not as strings.
The Android database API does not allow you to use parameter types other than strings in most functions. This is a horrible design bug.
To search for a number, either use execSQL(), which allows you to use number parameters, or convert the string value back into a number:
db.delete(..., "timestamp = CAST(? AS NUMBER)",
new String[]{ String.valueOf(ts) });
The problem was my mindset that assumed that every data type would be
a general character sequence if not specified and therefore the way to
talk to database is through java.lang.String.
I think that's the real issue.
If you specify no type e.g.
CREATE TABLE mytable (col1,col2,col3)
Then according to Determination of Column Affinity(3.1) rule 3:-
3) If the declared type for a column contains the string "BLOB" or if no
type is specified then the column has affinity BLOB.
And then according to Section 3
A column with affinity BLOB does not prefer one storage class over
another and no attempt is made to coerce data from one storage class
into another.
I've personally never had an issue with delete. However I do have a tendency to always delete according to rowid.
Here's a working example usage that shows that delete isn't useless and is deleting according to a long. However the columns are all of type INTEGER :-
int pudeletes;
int sldeletes;
int rdeletes;
int pdeletes;
if(doesProductExist(productid)) {
// if not in a transaction then begin a transaction
if(!intransaction) {
db.beginTransaction();
}
String whereargs[] = { Long.toString(productid)};
// Delete ProductUsage rows that use this product
pudeletes = db.delete(
DBProductusageTableConstants.PRODUCTUSAGE_TABLE,
DBProductusageTableConstants.PRODUCTUSAGE_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete ShopList rows that use this product
sldeletes = db.delete(
DBShopListTableConstants.SHOPLIST_TABLE,
DBShopListTableConstants.SHOPLIST_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete Rules rows that use this product
rdeletes = db.delete(
DBRulesTableConstants.RULES_TABLE,
DBRulesTableConstants.RULES_PRODUCTREF_COL +
" = ?",
whereargs
);
// Delete the Product
pdeletes = db.delete(
DBProductsTableConstants.PRODUCTS_TABLE,
DBProductsTableConstants.PRODUCTS_ID_COL +
" = ?",
whereargs
);
// if originally not in a transaction then as one was started
// complete and end the transaction
if(!intransaction) {
db.setTransactionSuccessful();
db.endTransaction();
}
}
In android, SQLiteDatabase has a update function
update(String table, ContentValues values, String whereClause, String[] whereArgs)
new values in put in values
If I want to update a column A by adding one to it, how should I prepare the ContentValues values variable? I don't think the following would work.
cv.put("A", "A" + 1);
I can sure run execSQL with raw sql, but it does not return num of row updated
If you'd execute a raw query, something like this should work to increment the current value in the column:
UPDATE table_name SET column_a = column_a + 1 WHERE _id = 1
(where 1 is just an example to illustrate how to apply it to a specific row)
The same probably wouldn't work with ContentValues, since (as the name indicates) it takes the values to set the column to. That means it needs to have been evaluated before building the ContentValues, whereas with a raw query the value isn't evaluated until the query actually runs on the database.
You can of course retrieve the current value first and then increment that accordingly when issuing an update; that requires a select query first. Quite commonly though, you're working with objects in Java, where the column value for a row is bound up to a member field of the object. If you've got a setup like that, then you probably already have the current value at the moment you want to run an update query.
As such, it would just look somewhat like:
SomeObject object = ...;
cv.put("column_a", object.getSomeValue() + 1);
(where I'm assuming object.getSomeValue() will return an int)
// edit: here's some more examples for the raw query approach:
SQLite - increase value by a certain number
// edit2: You've edited your original question and added:
I can sure run execSQL with raw sql, but it does not return num of
row updated
If knowing how many rows the query changed is a must, then you can potentially leverage the changes() function. It still means you're going to have to run a second query though.
SELECT changes() FROM table_name
The docs say:
The changes() function returns the number of database rows that were
changed or inserted or deleted by the most recently completed INSERT,
DELETE, or UPDATE statement, exclusive of statements in lower-level
triggers. The changes() SQL function is a wrapper around the
sqlite3_changes() C/C++ function and hence follows the same rules for
counting changes.
Alternatively, you could look into the rawQuery() method that takes an SQL statement and returns the result as a Cursor. Not sure if it that even works for an update query, or whether the result would be anything sensible, but if you're really lucky, you may find that Cursor.getCount() gives you the number of affected rows.
To expand upon #MH's solution, there actually is a way to do a raw update AND get back the number of rows updated (because I'm doing the same thing in one of my projects). You have to use a compiled SQLiteStatement and then call the method executeUpdateDelete(). From the documentation:
public int executeUpdateDelete ()
Execute this SQL statement, if the the number of rows affected by execution of this SQL statement is of any importance to the caller - for example, UPDATE / DELETE SQL statements.
Returns
the number of rows affected by this SQL statement execution.
See the following sample code where I add a new column to my table and then update each column similarly to how you were asking:
db.beginTransaction();
try {
db.execSQL("ALTER TABLE " + TABLE_NAME + " ADD COLUMN "
+ COLUMN_NAME_LOCALTIME + " INTEGER");
String stmtString = "UPDATE " + TABLE_NAME + " SET "
+ COLUMN_NAME_LOCALTIME + "="
+ COLUMN_NAME_TIME + "+ (" + tzOffset + ")";
SQLiteStatement sqlStmt = db.compileStatement(stmtString);
int rows = sqlStmt.executeUpdateDelete();
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
I'm using a transaction here because in case I add the new column but CANNOT update the values, I want everything to rollback so I can attempt something else.
I have a table (rowID, name, familyName) with at least 100.000 rows (SQLite). I am using this code to query my table when the text of an EditText changes (onTextChanged):
mySelectSql = "Select rowID from gamtable WHERE name GLOB ' " + searchEditText.getText().toString() + "*';";
Cursor c1 = db.rawQuery(mySelectSql, null);
rowID is primary key, and name is indexed! This code runs slowly (~6000ms) just on the first key that entered in the EditText, and works fast and very fast on second, third, etc.
I need to improve my code to result in first entered key as fast as second and third one...
Any help on this issue?
Assuming it's actually using an index (sounds like you've checked?), I would simply limit the number of results. Using the maximum number of displayable results in the UI preview.
SELECT rowID FROM gamtable WHERE name GLOB 'foo*' LIMIT 15;