I have an SQLite database in my Android app that I would like to update. The update requires a change of datatype from INT to FLOAT. I understand the way to achieve this in SQLite is to create a new table, copy the data, and then replace the old table with the new one. Copying would be done as follows:
INSERT INTO newTable SELECT * FROM oldTable
where integer values would be conveniently converted into floating point values.
Now, there is the problem that another, previously optional column is now ´NOT NULL´ but has a default value. I would like to have all NULL instances replaced by the default as part of the above process using the following statement:
INSERT INTO newTable SELECT * FROM oldTable ON CONFLICT REPLACE
or, for completeness, in Android/Kotlin:
db.execSQL(“INSERT INTO newTable SELECT * FROM oldTable ON CONFLICT REPLACE”)
This gives me a syntax error, and I could not find a suitable example elsewhere. What would be the correct syntax to achieve this - and is what I'm trying to do even possible?
If you look at the documentation for INSERT, you'll see it should be INSERT OR REPLACE ..., and, yes, it'll work to replace null values in the not-null column with the default value.
Replace conflict resolution documentation:
When a UNIQUE or PRIMARY KEY constraint violation occurs, the REPLACE algorithm deletes pre-existing rows that are causing the constraint violation prior to inserting or updating the current row and the command continues executing normally. If a NOT NULL constraint violation occurs, the REPLACE conflict resolution replaces the NULL value with the default value for that column, or if the column has no default value, then the ABORT algorithm is used. If a CHECK constraint or foreign key constraint violation occurs, the REPLACE conflict resolution algorithm works like ABORT.
Example:
sqlite> CREATE TABLE foo(x, y NOT NULL DEFAULT 1);
sqlite> INSERT INTO foo(x,y) VALUES ('a', null);
Error: NOT NULL constraint failed: foo.y
sqlite> INSERT OR REPLACE INTO foo(x,y) VALUES ('a', null);
sqlite> SELECT * FROM foo;
x y
---------- ----------
a 1
Related
Is marking a primary key with #PrimaryKey(autoGenerate = true) exactly the same as if you had used PRIMARY KEY AUTOINCREMENT in an SQL statement?
Intuition tells me yes, but documentation seems to suggest no.
Room javadoc states:
Set to true to let SQLite generate the unique id.
as if setting it false will prevent SQLite from generating the key.
But SQLite documentation for AUTOINCREMENT states that SQLite always generates a currently-unique key if none is given when doing an INSERT, and that AUTOINCREMENT merely adds the additional behavior that SQLite will never allow an automatically generated key to overlap with any previously deleted row.
The SQLite documentation also recommends not using AUTOINCREMENT if it isn't needed (for performance reasons), and states that it is usually not needed. From the description, that seems to match my case. My table will be fine if a previously deleted row ID gets reused.
Is marking a primary key with #PrimaryKey(autoGenerate = true) exactly the same as if you had used PRIMARY KEY AUTOINCREMENT in an SQL statement?
Yes, as using autoGenerate=true adds the AUTOINCREMENT keyword.
But
as if setting it false will prevent SQLite from generating the key.
Is false.
If a class is:-
annotated with #Entity, and
the column/variable/member is annotated with #PrimaryKey, and
if the type resolves to an integer type
(byte .... double, primitive or Object (e.g. Double))
then the value can be generated (it is INTEGER PRIMARY KEY that makes the column a special column that can be generated as that column is then an alias of the rowid (a normally hidden column)).
AUTOINCREMENT is only applicable to aliases of the rowid (i.e. INTEGER PRIMARY KEY). It does not determine whether the value can be generated (in the absence of a value for the column or when the value is null).
What AUTOINCREMENT does is add an additional rule when generating the value. That rule being that the value MUST be higher than any ever used for that table.
There are subtle differences.
Without AUTOINCREMENT
deleting the row with the highest value, frees that value for subsequent use (and would be used to generate the value still higher than any other value that exists at that time), and
should the highest value (9223372036854775807) be reached SQLite will try to find a free lower value, and
lastly it is possible to double the range of values by using negative values.
With AUTOINCREMENT
deleting the row with the highest value does not free that value for subsequent use
should the highest value (9223372036854775807) be reached then subsequent attempts to insert with a generated value will fail with an SQLITE FULL error.
If you insert 1 row with a value of 9223372036854775807 then that's the only row that can be inserted.
negative values cannot be generated (can still be used)
an additional table is required (sqlite_sequence), which is automatically created by SQLite, that will have a row per table with AUTOINCREMENT. The highest used value is stored in the row. So whenever inserting when the value is to be generated requires the respective row to be retrieved and the value obtained, after insertion the value has to be updated. As such there are overheads associated with using AUTOINCREMENT.
Note the above is assuming that methods to circumvent SQLite's in-built handling are not circumvented (such as updating values in the sqlite_sequence table).
I would always advocate using (not using autoGenerate=true) e.g.
#PrimaryKey
Long id_column=null;
or
#PrimaryKey
var id_column: Long?=null
thus an #Insert (convenience insert) will autogenerate if no value is given for the id_column.
Demo
Consider the following two #Entity annotated classes (with and without autoGenerate=true) :-
AutoInc:-
#Entity
data class AutoInc(
#PrimaryKey(autoGenerate = true)
val id: Long?=null,
val other: String
)
NoAutoInc:-
#Entity
data class NoAutoInc(
#PrimaryKey
var id: Long?=null,
var other:String
)
Room (after compiling and looking at the generated java in the class that is the same name as the #Database annotated class) has the following in the createAllTables method/function:-
_db.execSQL("CREATE TABLE IF NOT EXISTS `AutoInc` (`id` INTEGER PRIMARY KEY AUTOINCREMENT, `other` TEXT NOT NULL)");
_db.execSQL("CREATE TABLE IF NOT EXISTS `NoAutoInc` (`id` INTEGER, `other` TEXT NOT NULL, PRIMARY KEY(`id`))");
i.e. the only difference is the AUTOINCREMENT keyword.
Then consider the following code :-
/* Typical where the id will be generated */
dao.insert(AutoInc(other = "A"))
dao.insert(AutoInc(null,other = "B"))
dao.insert(NoAutoInc(other ="A"))
dao.insert(NoAutoInc(null, other = "B"))
/* Beware */
/* Room interprets types different ways
here 0 is taken to be 0 as id is an Object
if long (Java) then 0 will be generated id
getters/setters are taken in to consideration when determining type
* */
dao.insert(AutoInc(0,other = "W"))
dao.insert(NoAutoInc(0,other ="W"))
/* Unusual */
dao.insert(AutoInc(-100,"X"))
dao.insert(NoAutoInc(-100,other ="X"))
dao.insert(AutoInc(9223372036854775807,"Y")) /* The maximum value for an id */
dao.insert(NoAutoInc(9223372036854775807,"Y")) /* The maximum value for an id */
When run then the tables (via Android Studio's App Inspection) are:-
AutInc:-
Note the Z row has not been added due to :-
E/SQLiteLog: (13) statement aborts at 4: [INSERT OR ABORT INTO `AutoInc` (`id`,`other`) VALUES (?,?)] database or disk is full
However, the disk isn't full as Disk Explorer shows:-
It's by no means full as Disk Explorer shows (and of course the subsequent step works inserting a row into the database):-
and
NoAutInc
Here the Z row has been added with a generated id based upon SQLite finding an unused value due to the highest allowable value for an id having been reached as opposed to the failure due to the disk/table full.
I have a table in SQLite where I would like to set the value of 3rd column based on the input value of first two columns.
Does SQLite even support an expression in the default value of a column?
create table contract(
id primary key autoincrement,
insurance_pct integer not null default 0,
historical_yield integer not null default 0,
guaranteed_yield integer not null default (insurance_pct/100 * historical_yield)
)
When I run the above statement I see following error.
Query execution failed
Reason:
SQL Error [1]: [SQLITE_ERROR] SQL error or missing database (default value of column [GUARANTEED_YIELD] is not constant)
As the error message "default value of column [GUARANTEED_YIELD] is not constant" clearly points out, you cannot use variables in the default expression.
One way to achieve what you want is to make an after insert trigger, that updates the column, when it was inserted as null. That however requires, that the column is not declared not null as otherwise the INSERT will fail. So you'd have to check that too in a before update trigger.
CREATE TABLE contract
(id integer PRIMARY KEY
AUTOINCREMENT,
insurance_pct integer
NOT NULL
DEFAULT 0,
historical_yield integer
NOT NULL
DEFAULT 0,
guaranteed_yield integer
NULL
DEFAULT NULL);
CREATE TRIGGER contract_ai
AFTER INSERT
ON contract
FOR EACH ROW
WHEN (new.guaranteed_yield IS NULL)
BEGIN
UPDATE contract
SET guaranteed_yield = insurance_pct / 100 * historical_yield
WHERE id = new.id;
END;
CREATE TRIGGER contract_bu
BEFORE UPDATE
ON contract
FOR EACH ROW
WHEN (new.guaranteed_yield IS NULL)
BEGIN
SELECT raise(FAIL, 'NOT NULL constraint failed: contract.guaranteed_yield');
END;
One other thing I noticed: guaranteed_yield is an integer but your default expression pretty likely produces non integer values. You might lose something due to the required rounding. I'm not sure whether this is intentional.
Addendum:
Looking at the comments to your question I'm not sure whether you merely want a default -- i.e. the value of guaranteed_yield should have the value of the expression, if no other value is explicitly given at INSERT but it is possible for it to have other (non null) values either from an INSERT or or a subsequent UPDATE -- or if you intend this to be a calculated column, that always has the value the expression gives. In the latter case: I second the other commenters. This is a potentially dangerous thing regarding inconsistencies. Preferably use the expression in your queries or create a view.
This is not supported in SQLite.
The CREATE TABLE documentation states :
An explicit DEFAULT clause may specify that the default value is NULL, a string constant, a blob constant, a signed-number, or any constant expression enclosed in parentheses.
Further, the document defines the concept of constant expression :
For the purposes of the DEFAULT clause, an expression is considered constant if it contains no sub-queries, column or table references, bound parameters, or string literals enclosed in double-quotes instead of single-quotes.
This namely excludes references to other columns.
Other possible approaches in your use case :
compute the value dynamically in your DML queries (UPDATE and INSERT)
use a trigger to provide a dynamic default on DML operations
My teammate made an SQLite database for our android app using SQLite Manager for Firefox but I noticed that most of the columns marked with an asterisk in the ERD do not have the constraint not null in the database he made. I asked him about it and he said he wasn't sure what to put for the default value. I've tried adding a column with a constraint myself. If I select not null but leave the default value blank I get this
SQLiteManager: ALTER TABLE "main"."productorder" ADD COLUMN "cart_id" INTEGER NOT NULL [ Cannot add a NOT NULL column with default value NULL ]
I prefer that there isn't a default value for most of the columns but that the programmer is just prevented from making a row without giving a value.
We can give all mandatory columns default values if we have to. I found this question and it said that '' is acceptable as the default value for VARCHAR columns
Laravel migration with sqllite 'Cannot add a NOT NULL column with default value NULL'
However I don't know what the default value should be for foreign keys (like cart_id), dates, zipcodes, etc.
I'm developing an Android 2.2+ app with SQLite db support. I'm trying to define referential integrity constraints while creating the tables. I've also enabled the foreign key support by executing db.execSQL( "PRAGMA foreign_keys=ON;" ); in onCreate and onOpen methods of SQLiteOpenHelper. Afteer setting up my tables and proper foreign key references, it still allows me to insert rows in tables where reference records are missing. For e.g. I've following structure
CREATE TABLE Questions(_id integer primary key,question text not null);
CREATE TABLE Ques_Ans(_id integer primary key autoincrement,qid integer not null,
aid integer not null,is_correct integer default 0,
FOREIGN KEY (qid) REFERENCES Questions(_id));
and following my data in the table
INSERT INTO Questions VALUES(1, 'Some text');
INSERT INTO Ques_Ans(qid, aid, is_correct) VALUES(20, 1, 0);
If foreign key is properly set on Ques_Ans table, 2nd insert should have failed as there is no record in Questions table with id 20, but somehow my app does not thrown any error and inserts the 2nd insert statement. Can anybody tell me whats wrong over here or am I missing any configuration over here?
Update [03-Mar-2012]: After discussion thread with #Catcall
Using sqlite3 tool, switching on PRAGMA foreign_keys=ON; foreign key works as expected
Same thing if used via app on Emulator or on Phone does not work
Insert statements executed using insert() or execSQL(). None of them throw foreign key constraint failed error
PRAGMA integrity_check returns ok. So database is not corrupt.
Your foreign key is on the column "qid", not on the column "_id".
This INSERT statement
INSERT INTO Ques_Ans VALUES(20, 1, 0);
should have thrown the error
Error: table Ques_Ans has 4 columns but 3 values were supplied
This INSERT should succeed.
INSERT INTO Ques_Ans VALUES(20, 1, 0, 0);
This one should fail.
INSERT INTO Ques_Ans VALUES(21, 2, 0, 0);
Error: foreign key constraint failed
Later . . .
Since it works in sqlite, but not in your emulator, the problem is probably in either the emulator or your code.
This is the idiom for database transactions.
db.beginTransaction();
try {
...
db.setTransactionSuccessful();
} finally {
db.endTransaction();
}
Android docs suggest using insert() instead of execSQL(). Make sure you're trapping, catching, or checking for errors in every function that can return an error. So, for example, if you switch to insert(), check its return value for -1.
If that doesn't help, you might try tagging this question with "Java" (for example), or asking a new question that focuses on your code and on catching errors.
IN SQLite Foreign key constraints are disabled by default (for backwards compatibility). You have to enable it explicitly using
PRAGMA foreign_keys = 1
after you establishing your connection with the database. Here's the link to the official docs that explains it in more depth. http://sqlite.org/foreignkeys.html Please navigate to enabling foreign key support in the above link.
How does Android's SQLite library resolve conflicts if I insert duplicate rows? I am building my app for API level 7, which unfortunately does not have the SQLiteDatabase.insertOnConflict() method that was included starting in API level 8.
You can specify a UNIQUE index in the table definition which will allow rows to be REPLACED:
CREATE TABLE mytable (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
UNIQUE (id) ON CONFLICT REPLACE
)
If a row an INSERT or UPDATE statement tries to add a row with an id which already exists, the existing row is replaced with the new one.
There's a ON CONFLICT clause in SQLite that you can say INSERT INTO ... ON CONFLICT....
Read the documentation please. http://www.sqlite.org/lang_conflict.html
By default i you have unique index exception will be thrown on duplicated index insert. Common solution is check record existence before execute insert - pseudo-code:
select * from people where name = 'Smith';
//if resultset is empty then perform insert
insert into people values 'Smith'