Develop Reference

How to insert only 3 columns data in room database table if we have more columns?

I have a project that I wrote in kotlin. I want to insert data in different columns of the same table on different pages. I have specified these columns in the dataclass, but it gives a null data error.
In order to make this insert process more healthy, should I divide the table into two separate tables or send static 'null' data and update these fields?

In a database, such as SQLite (which Room is a wrapper around), the unit of insertion is a row.
A row will consist of the same number of columns. You cannot insert a column on it's own, other than if you ALTER the table to add or remove a column, when the change is reflected throughout the entire table.
if adding a column then a DEFAULT VALUE must be provided, this could be the default/implicit value of null or another specific value.
Room with Kotlin will apply a constraint (rule) of NOT NULL unless nulls are specifically allowed using for example ?
var theColumn: Int has the implicit NOT NULL
var theColumn: Int? does not have the implicit NOT NULL and nulls can be stored
var theColumn: Int=-1 will apply a default value of -1 in the absence of the field not being supplied a value when instantiating the object.
var theColumn: Int?=null will apply null in the absence of the field not being supplied a value when instantiating the object.
obviously fields may be altered before inserting the object, if var rather than val is used.
The data stored in the column can be interpreted to represent whatever you wish, often NULL will be used to represent a special situation such as no value.
If using an #Insert annotated function, then ALL columns are applied the values as obtained from the object or objects passed to the function. In Kotlin whether or not NULLs can be used is dependent upon the field definition or in some cases the #NonNull annotation.
#Insert indicates what is termed as a convenience method, it actually builds the underlying SQL along with binding the values using the SQLite API.
However, if you want flexibility, then an #Query annotation with suitable INSERT SQL statement can be used.
e.g. you could perhaps have a table that has 4 columns COL1, COL2, COL3 and COL4 and only apply some of the columns (the DEFAULT VALUE will be applied to the other column if specified, if not the NULL but if there is a NOT NULL constraint then a conflict would be raised).
So to insert when only two of the columns (COL2 and COL4) then you could use:-
#Query("INSERT INTO theTable (COL2,COL4) VALUES(:valueForCol2,:valueForCol4)")
fun insertCol2AndCol4Only(valueForCol2: Int, valueForCol4: Int?)
Note that valueForCol4 could be NULL. However, whether or not a NULL will result in a conflict depends upon how the field is defined in the #Entity annotated class.
Conflicts (breaking a rule) can be handled by SQLite, depending upon the type of the conflict. UNIQUE, PRIMARY KEY (which is really a UNIQUE conflict), CHECK (Room doesn't cater for CHECK constraints) and NOT NULL constraints can be handled in various ways at the SQLite level.
A common use of conflict handling is to IGNORE the conflict, in which case the action (INSERT or UPDATE) is ignored. In the case of INSERT the row is not inserted but SQLite ignores the conflict and doesn't issue an error.
So if for example COL4's field was var COL4: Int and not var COL4: Int? then the insert would fail and an SQlite Exception would occurr.
However if instead
#Query("INSERT OR IGNORE INTO theTable (COL2,COL4) VALUES(:valueForCol2,:valueForCol4)")
were used and the COL4 field were defined as var COL4: Int (implied NOT NULL constraint) then the conflict if NULL was passed as valueForCol4 then the row would not be inserted but no failure would occur as the NOT NULL conflict would be ignored.
With the #Insert annotation you can defined this conflict handling via the onConflictStrategy parameter e.g. #Insert(onConflictStrategy=OnConflict.IGNORE)
You may wish to consider reading the following:-
The On Conflict Clause
INSERT
In order to make this insert process more healthy, should I divide the table into two separate tables or send static 'null' data and update these fields?
Note the above is only a summary, INTEGER PRIMARY KEY aka #PrimaryKey var id: Long?=null or variations such as #PrimaryKey(autoGenerate=true) etc has specifically not been discussed.
The design of the database could be handled either way, from the very limited description of the scenario, a most likely suitable scenario cannot really be advised, although either could probably be an approach.
Additional
Based upon the comment:-
For example, I'm going to add the features of a car to the database, but it could be a different type at a time. So on the first page, the type of car will be chosen, like off road, sedan, 4x4, hatchback.
The perhaps consider having a feature table and a mapping table for a many-many relationship between car and it's features as per my response:-
I would suggest that features be a table and with a many-many relationship with the car. That is a car could have a 0-n features and a feature could be used by 0-n cars. The many-many relationship would require a third table known by many terms such as an associative table/reference table/ mapping table. Such a table has 2 core columns a column to map to the car and a column to map to the feature, the primary key being a composite of both these columns.
Here's a basic example of how this could work from an SQLite basis:-
DROP INDEX IF EXISTS carFeatureMap_idxon_feature;
DROP TABLE IF EXISTS carFeatureMap;
DROP TABLE IF EXISTS car;
DROP TABLE IF EXISTS feature;
CREATE TABLE IF NOT EXISTS car (
carId INTEGER PRIMARY KEY,
carname TEXT /* and other columns */
);
CREATE TABLE IF NOT EXISTS feature (
featureId INTEGER PRIMARY KEY,
featureDescription TEXT
);
CREATE TABLE IF NOT EXISTS carFeatureMap (
carIdMap INTEGER REFERENCES car(carId) ON DELETE CASCADE ON UPDATE CASCADE,
featureIdMap INTEGER REFERENCES feature(featureId) ON DELETE CASCADE ON UPDATE CASCADE,
PRIMARY KEY(carIdMap, featureIdMap)
);
/* Should improve efficiency of mapping from a feature */
CREATE INDEX IF NOT EXISTS carFeatureMap_idxon_feature ON carFeatureMap(featureIdMap);
/* Add some features */
INSERT OR IGNORE INTO feature VALUES(100,'4x4'),(101,'Sedan'),(106,'Convertable'),(null /*<<<< featureId generated by SQLite*/ ,'Hatchback');
/*Report1 Output the features */
SELECT * FROM feature;
/* Add some cars */
INSERT OR IGNORE INTO car VALUES(10,'Car1'),(20,'Car2'),(30,'Car3');
/*Report2 Output the cars */
SELECT * FROM car;
/* add the mappings/relationships/associations between cars and features */
INSERT OR IGNORE INTO carFeatureMap VALUES (10,101) /* Car 1 has 4x4*/,(10,106) /* Car 1 has Sedan */,(20,100);
/*Report3 Get the Cars with features cartesian product */
SELECT
car.carName,
featureDescription
FROM car
JOIN carFeatureMap ON car.carId=carFeatureMap.carIdMap
JOIN feature ON featureIdMap=featureId
;
/*Report4 Get the Cars with features with all the features concatendated, i.e. single output per car with features */
SELECT
car.carName,
group_concat(featureDescription,': ') AS allFeatures
FROM car
JOIN carFeatureMap ON car.carId=carFeatureMap.carIdMap
JOIN feature ON featureIdMap=featureId GROUP BY (carId)
;
/*Report5 Similar to the previous BUT if no features then output none so ALL cars are output */
SELECT
carName,
coalesce(
(
SELECT
group_concat(featureDescription)
FROM feature
JOIN carFeatureMap ON carFeatureMap.featureIdMap=featureId AND carFeatureMap.carIdMap=carId
),
'none'
) AS features
FROM car
;
/* Clean Up After Demo*/
DROP INDEX IF EXISTS carFeatureMap_idxon_feature;
DROP TABLE IF EXISTS carFeatureMap;
DROP TABLE IF EXISTS car;
DROP TABLE IF EXISTS feature;
Results from the demo code above
Report1 - The features
Report2 - The cars
**Report3 ** Cars and features
Report 4 Cars and features 2
Report 5 Cars and features 3

Is PrimaryKey's autoGenerate exactly equivalent to SQLite's AUTOINCREMENT?

Is marking a primary key with #PrimaryKey(autoGenerate = true) exactly the same as if you had used PRIMARY KEY AUTOINCREMENT in an SQL statement?
Intuition tells me yes, but documentation seems to suggest no.
Room javadoc states:
Set to true to let SQLite generate the unique id.
as if setting it false will prevent SQLite from generating the key.
But SQLite documentation for AUTOINCREMENT states that SQLite always generates a currently-unique key if none is given when doing an INSERT, and that AUTOINCREMENT merely adds the additional behavior that SQLite will never allow an automatically generated key to overlap with any previously deleted row.
The SQLite documentation also recommends not using AUTOINCREMENT if it isn't needed (for performance reasons), and states that it is usually not needed. From the description, that seems to match my case. My table will be fine if a previously deleted row ID gets reused.

Is marking a primary key with #PrimaryKey(autoGenerate = true) exactly the same as if you had used PRIMARY KEY AUTOINCREMENT in an SQL statement?
Yes, as using autoGenerate=true adds the AUTOINCREMENT keyword.
But
as if setting it false will prevent SQLite from generating the key.
Is false.
If a class is:-
annotated with #Entity, and
the column/variable/member is annotated with #PrimaryKey, and
if the type resolves to an integer type
(byte .... double, primitive or Object (e.g. Double))
then the value can be generated (it is INTEGER PRIMARY KEY that makes the column a special column that can be generated as that column is then an alias of the rowid (a normally hidden column)).
AUTOINCREMENT is only applicable to aliases of the rowid (i.e. INTEGER PRIMARY KEY). It does not determine whether the value can be generated (in the absence of a value for the column or when the value is null).
What AUTOINCREMENT does is add an additional rule when generating the value. That rule being that the value MUST be higher than any ever used for that table.
There are subtle differences.
Without AUTOINCREMENT
deleting the row with the highest value, frees that value for subsequent use (and would be used to generate the value still higher than any other value that exists at that time), and
should the highest value (9223372036854775807) be reached SQLite will try to find a free lower value, and
lastly it is possible to double the range of values by using negative values.
With AUTOINCREMENT
deleting the row with the highest value does not free that value for subsequent use
should the highest value (9223372036854775807) be reached then subsequent attempts to insert with a generated value will fail with an SQLITE FULL error.
If you insert 1 row with a value of 9223372036854775807 then that's the only row that can be inserted.
negative values cannot be generated (can still be used)
an additional table is required (sqlite_sequence), which is automatically created by SQLite, that will have a row per table with AUTOINCREMENT. The highest used value is stored in the row. So whenever inserting when the value is to be generated requires the respective row to be retrieved and the value obtained, after insertion the value has to be updated. As such there are overheads associated with using AUTOINCREMENT.
Note the above is assuming that methods to circumvent SQLite's in-built handling are not circumvented (such as updating values in the sqlite_sequence table).
I would always advocate using (not using autoGenerate=true) e.g.
#PrimaryKey
Long id_column=null;
or
#PrimaryKey
var id_column: Long?=null
thus an #Insert (convenience insert) will autogenerate if no value is given for the id_column.
Demo
Consider the following two #Entity annotated classes (with and without autoGenerate=true) :-
AutoInc:-
#Entity
data class AutoInc(
#PrimaryKey(autoGenerate = true)
val id: Long?=null,
val other: String
)
NoAutoInc:-
#Entity
data class NoAutoInc(
#PrimaryKey
var id: Long?=null,
var other:String
)
Room (after compiling and looking at the generated java in the class that is the same name as the #Database annotated class) has the following in the createAllTables method/function:-
_db.execSQL("CREATE TABLE IF NOT EXISTS `AutoInc` (`id` INTEGER PRIMARY KEY AUTOINCREMENT, `other` TEXT NOT NULL)");
_db.execSQL("CREATE TABLE IF NOT EXISTS `NoAutoInc` (`id` INTEGER, `other` TEXT NOT NULL, PRIMARY KEY(`id`))");
i.e. the only difference is the AUTOINCREMENT keyword.
Then consider the following code :-
/* Typical where the id will be generated */
dao.insert(AutoInc(other = "A"))
dao.insert(AutoInc(null,other = "B"))
dao.insert(NoAutoInc(other ="A"))
dao.insert(NoAutoInc(null, other = "B"))
/* Beware */
/* Room interprets types different ways
here 0 is taken to be 0 as id is an Object
if long (Java) then 0 will be generated id
getters/setters are taken in to consideration when determining type
* */
dao.insert(AutoInc(0,other = "W"))
dao.insert(NoAutoInc(0,other ="W"))
/* Unusual */
dao.insert(AutoInc(-100,"X"))
dao.insert(NoAutoInc(-100,other ="X"))
dao.insert(AutoInc(9223372036854775807,"Y")) /* The maximum value for an id */
dao.insert(NoAutoInc(9223372036854775807,"Y")) /* The maximum value for an id */
When run then the tables (via Android Studio's App Inspection) are:-
AutInc:-
Note the Z row has not been added due to :-
E/SQLiteLog: (13) statement aborts at 4: [INSERT OR ABORT INTO `AutoInc` (`id`,`other`) VALUES (?,?)] database or disk is full
However, the disk isn't full as Disk Explorer shows:-
It's by no means full as Disk Explorer shows (and of course the subsequent step works inserting a row into the database):-
and
NoAutInc
Here the Z row has been added with a generated id based upon SQLite finding an unused value due to the highest allowable value for an id having been reached as opposed to the failure due to the disk/table full.

SQLite - integer PRIMARY INDEX constraint failing?

In my Android app, I create a FULLTEXT table like this:
CREATE VIRTUAL TABLE products USING fts3 (
_id integer PRIMARY KEY,
product_name text NOT NULL,
...
)
And I add this index:
CREATE INDEX product_name_index ON products (product_name)
The app populates the table with various products, each with a unique _id value.
However, when I then try to insert an already-existing product ID (using an _id value that is already in the table, but with a different product_name value) like this:
long rowId = db.insertOrThrow("products", null, contentValues);
a new row is added to the table (with a brand new rowId value returned)!
I expected the insertOrThrow command to fail, so where am I going wrong? Is it something to do with the fact that it's a FULLTEXT table or could the index I specified on the product_name column be messing things up somehow?
I read this section about INTEGER PRIMARY KEY, but unfortunately I'm none the wiser.
Update
When I try to perform the same operation on a standard (non-FULLTEXT) table, then the insertOrThrow command results in the expected SQLiteConstraintException.

I think the issue might be that an FTS table has the concept of a docid and a rowid column and specifying null for the docid results in that being given a value.
as per :-
There is one other subtle difference between "docid" and the normal
SQLite aliases for the rowid column.
Normally, if an INSERT or UPDATE
statement assigns discrete values to two or more aliases of the rowid
column, SQLite writes the rightmost of such values specified in the
INSERT or UPDATE statement to the database.
However, assigning a
non-NULL value to both the "docid" and one or more of the SQLite rowid
aliases when inserting or updating an FTS table is considered an
error. See below for an example.
1.3. Populating FTS Tables

How can i delete every value row in my SQLite Table?

I want to delete all my values in my sqlite Table.
When I click my button it deletes everything besides the id it keeps counting
so for example:
ID, FIRSTNAME, LASTNAME
1, Jack Sparrow
2, Johhny Dep
if I press delete and add new values, it shows this
ID, FIRSTNAME, LASTNAME
2, Obama Barack
3, Mike Tyson
this is my method
private void DeleteEverything()
{
SQLiteDatabase db = mDbHelper.getWritableDatabase();
db.execSQL("delete from "+NamesContract.NameEntry.TABLE_NAME);
}

You should consider to drop and recreate the table.
You can find an example here

If you specify ?? INTEGER PRIMARY KEY or ?? INTEGER PRIMARY KEY AUTOINCREMENT then SQlite treats this in a specific way. That is ?? (by the looks of it ID in your case) is an alias for SQLite's rowid.
rowid if not specified when inserting a row will automatically be generated and is in general guaranteed to be unique and is normally incremented by one. However, it is not guaranteed to be incremented nor incremented by one.
If you specify AUTOINCREMENT (i.e. the latter of the two above) then the guarantee, is that a new rowid will be greater but not necessarily by 1.
There is a limit of 9223372036854775807 on the value of rowid. If AUTOINCREMENT is not used and this limit has been reached then an attempt will made to utilise free numbers (e.g. the rowid of rows that have been deleted). If AUTOINCREMENT is specified and the limit has been reached then an insert will fail with an SQLITE_FULL error.
As such, in your case the freed ID's from deleting rows will not be reused.
In short you should never rely upon the rowid (or an alias of it) column being a specific value, rather you should rely upon it just being a unique value purely for the purpose of uniquely identifying a row (and perhaps the fastest way of accessing a row).
You can, albeit it inadvisable, set rowid either by say INSERT INTO mytable (rowid, myothercolumn) VALUES(1, 'other data') or if ID has been used as an alias then INSERT INTO mytable (ID, myothercolumn)VALUES(1, 'other data').
If you really need the first row to be 1 and then 2 and so on then you could DROP the table and then recreate it rather then deleting all rows. e.g. DROP TABLE mytable, followed by CREATE TABLE mytable (ID INTEGER PRIMARY KEY, FIRSTNAME TEXT, LASTNAME TEXT). However, I'd suggest you will just end up with issues which will subsequently be far more complicated to determine and correct.
If you think about it, really what does Johnny Depp being 2 signify rather than Barack Obama being 2? If it is, for example, a popularity rating the surely you'd have some means of determining the popularity and input that directly rather than input it somewhere else to then be used to determine the insertion order and then do the inserts.
Much of the above is basically a summary of SQLite Autoincrement
As an example of unpredictability, a table was created with:-
CREATE TABLE mytable (ID INTEGER PRIMARY KEY, FIRSTNAME TEXT, LASTNAME TEXT)
A row was inserted using INSERT INTO mytable (ID, FIRSTNAME, LASTNAME) VALUES(9223372036854776000,'FRED','BLOGGS'). Note the use of 9223372036854776000 forces above the limit processing.
This was then followed by a number of INSERT INTO mytable(FIRSTNAME, LASTNAME) VALUES('????','????') inserts. Note! without ID, so using SQLITE's unique identifer determination (above the limit processinmg). The resultant table :-
MARY QUITECONTRARY was the 2nd row inserted,TOM SMITH the 3rd. The two RICHARD BURTON rows are an example of where the unique id could be essential for determining a specific row, both were also inserted without specifying the insertion order.
Note! if the above were tried but with AUTOINCREMENT specified, then the second insert, and any subsequent inserts, would fail with an SQLITE_FULL error.

Designing date orientated database sqlite

I'm new in sqlite. I've built database but based on query I was trying to solve for it (which was over-complicated), I was suggested to look into normalising database, which I did, but can't seem to find examples on database that would be orientated around dates like a diary, with lots of daily entries. I'm working on app that would help log in everyday what I've eaten, what exercise did I do, what activities I've done, what was my well-being, how many hours I've slept. I will be able to go back to any day in the past and see what I was up to, so it will have to look up all entries for that particular date.
So I understand I need separate tables for food type, exercise type, activities types, event types and I need main table "diary" which will log each time date and id referencing another table. So I'm wondering if in that diary table I can have date column, id column and lets say type column (which will differentiate which table does id column references) or should I rather have date column and column for each of the other tables ids, even though I will be logging only one type at the time?
Also, would indexing the date column be a good idea?
Or maybe there is a better way to design that database? Any suggestions will be appreciated.

So I understand I need separate tables for food type, exercise type,
activities types, event types
If normalising then perhaps consider a single table for all types with a column to indicate the type.
So I'm wondering if in that diary table I can have date column, id
column and lets say type column (which will differentiate which table
does id column references) or should I rather have date column and
column for each of the other tables ids, even though I will be logging
only one type at the time?
If you are logging and assuming human input (as opposed to automated) then it is highly likely that a timestamp would be sufficient to uniquely identify a row.
As such there would be little need for an id column(in theory).
Saying that SQLite, unless you specify WITHOUT ROWID (which you might consider, this may be of use in deciding:-Clustered Indexes and the WITHOUT ROWID Optimization ), automatically creates a unique row identifier column i.e ROWID.
If you code a column as columnname INTEGER PRIMARY KEY or columnname INTEGER PRIMARY KEY AUTOINCREMENT then columnname becomes an alias for ROWID, in which case the unique value will be provided, if you do not provide a value when inserting.
However, if you were to specify timestamp INTEGER PRIMARY KEY and provide the current date/time as a value for the column when inserting, the current date/time would be stored and would also be indexed (it would have to be unique (current date/time would very likely be)).
So I'd suggest that a log entry need only be something like CREATE TABLE log (timestamp INTEGER PRIMARY KEY, eventref INTEGER);, where eventref is a reference to the event type.
As _id is required at times e.g. for a CursorAdapter then you could specify the columns to be extracted as *, timestemp as _id (3 columns timestamp, eventref and _id (timestamp and _id would be identical))or timstemp as _id, * (3 columns but _id, timestamp and eventref) or timestamp as _id, eventref (2 columns _id and eventref).
So using this model as the basis would minimise columns and be indexed automatically.
An example
You may have the events table as :-
Along with log table as :-
A query such as SELECT * FROM log JOIN events WHERE eventref = _id would give:-
Note! made up timestamps for illustration purposes