Develop Reference

SQLite - integer PRIMARY INDEX constraint failing?

In my Android app, I create a FULLTEXT table like this:
CREATE VIRTUAL TABLE products USING fts3 (
_id integer PRIMARY KEY,
product_name text NOT NULL,
...
)
And I add this index:
CREATE INDEX product_name_index ON products (product_name)
The app populates the table with various products, each with a unique _id value.
However, when I then try to insert an already-existing product ID (using an _id value that is already in the table, but with a different product_name value) like this:
long rowId = db.insertOrThrow("products", null, contentValues);
a new row is added to the table (with a brand new rowId value returned)!
I expected the insertOrThrow command to fail, so where am I going wrong? Is it something to do with the fact that it's a FULLTEXT table or could the index I specified on the product_name column be messing things up somehow?
I read this section about INTEGER PRIMARY KEY, but unfortunately I'm none the wiser.
Update
When I try to perform the same operation on a standard (non-FULLTEXT) table, then the insertOrThrow command results in the expected SQLiteConstraintException.

I think the issue might be that an FTS table has the concept of a docid and a rowid column and specifying null for the docid results in that being given a value.
as per :-
There is one other subtle difference between "docid" and the normal
SQLite aliases for the rowid column.
Normally, if an INSERT or UPDATE
statement assigns discrete values to two or more aliases of the rowid
column, SQLite writes the rightmost of such values specified in the
INSERT or UPDATE statement to the database.
However, assigning a
non-NULL value to both the "docid" and one or more of the SQLite rowid
aliases when inserting or updating an FTS table is considered an
error. See below for an example.
1.3. Populating FTS Tables

How to set a 'DEFAULT' field when using fts3 Sqlite table?

I'm trying to set a default field in my table to current time. When I use a fts3 virtual table, inserting a row doesn't fill the default field to what it should be. Instead, it inserts null.
If I create the same table as normal table, the exact same query works and the field is populated.
Here are the 2 different table structures I'm using:
Normal table that default value does work
CREATE TABLE Emlak_test2 (_id INTEGER PRIMARY KEY,emlak_id TEXT,created_at TEXT DEFAULT (datetime('now', 'localtime')),emlak_sellorrent TEXT,emlak_cat TEXT,emlak_altcat TEXT,emlak_desc TEXT,emlak_living_rooms INTEGER,emlak_rooms INTEGER,emlak_sellprice INTEGER,emlak_temp TEXT,emlak_city TEXT,emlak_state TEXT,emlak_address TEXT,img_p1 TEXT,img_p2 TEXT,img_p3 TEXT,img_p4 TEXT,img_p5 TEXT,musteri_id TEXT);
FTS3 table that the default value does not work
CREATE VIRTUAL TABLE Emlak_test USING fts3 (_id INTEGER PRIMARY KEY,emlak_id TEXT,created_at TEXT DEFAULT (datetime('now', 'localtime')),emlak_sellorrent TEXT,emlak_cat TEXT,emlak_altcat TEXT,emlak_desc TEXT,emlak_living_rooms INTEGER,emlak_rooms INTEGER,emlak_sellprice INTEGER,emlak_temp TEXT,emlak_city TEXT,emlak_state TEXT,emlak_address TEXT,img_p1 TEXT,img_p2 TEXT,img_p3 TEXT,img_p4 TEXT,img_p5 TEXT,musteri_id TEXT);
Now, if I use this query;
insert into table_name default values;
on the first table, I can see that created_at field is populated. On the second table, the field is empty.
I hope you can help me with this.
Thank you!

The documentation says:
If column names are explicitly provided for the FTS table as part of the CREATE VIRTUAL TABLE statement, then a datatype name may be optionally specified for each column. This is pure syntactic sugar, the supplied typenames are not used by FTS or the SQLite core for any purpose. The same applies to any constraints specified along with an FTS column name – they are parsed but not used or recorded by the system in any way.
So it is not possible to have default values.
And,
it is not possible to create indices or triggers attached to FTS tables.
So it is not possible to work around this.

How to query an external content FTS4 table but return additional columns from the original content table

I am creating an FTS4 external content table in SQLite like this:
CREATE TABLE t2(id INTEGER PRIMARY KEY, col_a, col_b, col_text);
CREATE VIRTUAL TABLE fts_table USING fts4(content="t2", col_text);
I'm using an external content table so that I don't need to store duplicate values of col_text in fts_table. I'm only indexing col_text because col_a and col_b don't need to be indexed.
However, when I do a query of fts_table like this
SELECT * FROM fts_table WHERE fts_table MATCH 'something';
I don't have access to col_a and col_b from the content table t2. How do return all these columns (col_a, col_b, col_text) from a single FTS query?
Update
I tried using the notindexed=column_name option as in
CREATE VIRTUAL TABLE fts_table USING fts4(content="t2", col_a, col_b, col_text, notindexed=col_a, notindexed=col_b);
This should work for some people, but I am using it in Android and the notindexed option isn't supported until SQLite 3.8, which Android doesn't support until Android version 5.x. And I need to support android 4.x. I am updating this question to include the Android tag.

FTS tables have an internal INTEGER PRIMARY KEY column called docid or rowid.
When inserting a row in the FTS table, set that column to the primary key of the row in the original table.
Then you can easily look up the corresponding row, either with a separate query, or with a join like this:
SELECT *
FROM t2
WHERE id IN (SELECT docid
FROM fts_table
WHERE col_text MATCH 'something')

Change column name in sqlite when other tables have foreign keys to it

I have a table named groups, and I want to rename one of its columns. Is was ok, so far. I know sqlite doesn't support renaming columns, so I did:
ALTER TABLE groups RENAME to tmp_groups;
CREATE TABLE groups(
_ID integer primary key autoincrement,
new_column_name integer
);
INSERT INTO groups(_ID, new_column_name) SELECT _ID, old_column_name FROM tmp_groups;
DROP TABLE tmp_groups;
But, when I drop the table tmp_groups, the table members, that had a foreign key with ON DELETE CASCADE has its records deleted as well, so I had to do the following:
Create a table tmp_members with the same columns as members, and without the foreign key;
Insert the records from members in tmp_members;
Drop the table members;
Run the code from the first part (with the groups table);
Re-create the table members with its foreign key;
Insert in members data from tmp_members;
Man, that was tiring! Its too much code to simply rename a column;
Is there any simpler way to handle this constraint problem, or is this the "sqlite way"?

For historical reasons, SQLite allows to disable foreign key constraints (and this is even the default currently).
Just run PRAGMA foreign_keys = off before doing the groups table stuff.
It would also be possible to rename a column by using PRAGMA writable_schema, but you should do this only if you know what you're doing.

Automatically Create Insert Statements to Fill Junction Table in a Pre-Created DB

For a simple android app I'm creating as a teaching tool for myself (for using relational dbs/SQL among other things - pardon the simplicity of the question if you will). I'm pre-creating a sqlite db to ship with the application. I'm doing this based on the following SO question.
I've got two tables with a many to many relationship and a junction table to define those relationships as follows:
CREATE TABLE Names (_id INTEGER PRIMARY KEY,
name TEXT
);
CREATE TABLE Categories (_id INTEGER PRIMARY KEY,
category TEXT
);
CREATE TABLE Name_Category (name_id INTEGER,
category_id INTEGER,
PRIMARY KEY (name_id, category_id),
foreign key (name_id) references Names(_id),
foreign key (category_id) references Categories(_id)
);
I've got sets of insert statements to fill the Names and Categories tables. I'm now faced with the task of filling the junction table. I'm sure that I could create the insert statements by hand by looking up the ids of the names and categories that I want to match, but that seems a bit silly.
In order to automatically create the insert statements for the junction table, I imagine that I could create a script based on a set of name and category pairs that will search for the appropriate ids and dump an insert statement. (I came up with this as I was asking the question and will research it. Don't you love it when that happens?)
Does anybody have any suggestions for ways to do this?
EDIT I added the foreign keys because, as pointed out below, they'll help maintain integrity between the tables.
EDIT #2 To solve this, I created a simple Perl script that would take a text file with name - category pairs and dump them out into another file with the appropriate SQL statements.
The name - category text file has a format as follows:
'Name' 'Category'
The Perl script looks like this:
use strict;
use warnings;
open (my $name_category_pair_file, "<", "name_category.txt") or die "Can't open name_category.txt: $!";
open (my $output_sql_file, ">", "load_name_category_junction_table.sqlite") or die "Can't open load_name_category_junction_table.sqlite: $!";
while (<$name_category_pair_file>) {
if (/('[a-zA-Z ]*') ('[a-zA-Z ]*')/) {
my $sql_statement = "INSERT INTO Name_Category VALUES (
(SELECT _id FROM Names WHERE name = $1),
(SELECT _id FROM Categories WHERE category = $2))\;\n\n";
print $output_sql_file $sql_statement;
}
}
close $name_category_pair_file or die "$name_category_pair_file: $!";
close $output_sql_file or die "$output_sql_file: $!";

You can use this insert in your script or code (replacing the strings or using ?):
insert into Name_Category values(
(select _id from Categories where category='CAT1'),
(select _id from Names where name='NAME1'));
Also, you can alter the Name_Category table to constraint on the values that can be inserted and/or deleted:
CREATE TABLE Name_Category ( name_id INTEGER NOT NULL,
category_id INTEGER NOT NULL,
PRIMARY KEY (name_id, category_id),
foreign key (name_id) references Names(_id),
foreign key (category_id) references Categories(_id));

create two main tables first and then create a junction table in which primary key of both main tables will be available as foreign key.. Primary key of junction table will be union
of primary key of first and second main table.
Create trigger now to automatically insert into junction table...
Also don't forget to create table with cascade deletion and cascade updatation so that any value updated or deleted in main tables will be automatically reflected in junction table