I've created a SQLite FTS3 table for an Android app with several columns. The table is populated by user's inputs. However not all table columns should have a value. Example: There is a column named PLURAL. The user submits a word which is a verb (not a noun) and so the PLURAL column shouldn't have a value.
What should I input into the PLURAL column, a NULL value or an empty string ""?
Except for the logic that affects the answer, is there any significant difference in the Database performance if I use NULL over empty string values? Are there any known issues (e.g. throw exceptions) if I use NULL values?
I 've searched around the web and I 've found some opinions on other DB but not on SQLite fts3.
Thank you!
Based on logic: you should not insert empty string but NULL, since in SQL word NULL is treated as NO VALUE
There's no preformance issues related to emptyness or nulliness of string. The only consideration can emerge in case of SQL inner join statement. INNER JOIN joins only non NULL fields which fits to join conditions. There's special LEFT/RIGHT JOIN statements which permits usage of NULL fields in joins.
what i understood after going through multiple posts is that it handles null as combination of behavior from different database engines.
The goal is to make SQLite handle NULLs in a standards-compliant way. But the descriptions in the SQL standards on how to handle NULLs seem ambiguous. It is not clear from the standards documents exactly how NULLs should be handled in all circumstances.
refer this for detailed information
http://www.sqlite.org/nulls.html
Related
I have an SQLite DB where I perform a query like
Select * from table where col_name NOT IN ('val1','val2')
Basically I'm getting a huge list of values from server and I need to select the ones which is not present in the list given.
Currently its working fine, No issues. But the number of values from server becomes huge as the server DB is getting updated frequently.
So, I may get thousands of String values which I need to pass to the NOT IN
My question is, Will it cause any perfomance issue in the future? Does the NOT IN parameter have any size restriction? (like max 10000 values you can check)?
Will it cause any crash at some point?
This is an official reference about various limitation in sqlite. I think the Maximum Length Of An SQL Statement may related to your case. Default value is 1000000, and it is adjustable.
Except this I don't think any limitation existed for numbers of parameter of NOT IN clause.
With more than a few values to test for, you're better off putting them in a table that has an index on the column holding them. Then things like
SELECT *
FROM table
WHERE col_name NOT IN (SELECT value_col FROM value_table);
or
SELECT *
FROM table AS t
WHERE NOT EXISTS (SELECT 1 FROM value_table WHERE value_col = t.col_name);
will be reasonably efficient no matter how many records are in value_table because that index will be used to find entries.
Plus, of course, it makes it a lot easier to re-use prepared statements because you don't have to create a new one and re-bind every value (You are using prepared statements with placeholders for these values, right, and not trying to put their contents inline into a string?) every time you add a value to the ones you need to check. You just insert it into value_table instead.
Yes, there is a limit of 999 arguments as reported in the official documentation: https://www.sqlite.org/limits.html#max_variable_number
I need to change the data type for my SQLite. I am worry that it might effect users who update the App. However, after reading the SQLite document in the following link
https://sqlite.org/datatype3.html
It would seem changing the data type when creating a table column shouldn't break the App. From what I read it seems unlike other SQL database engines, SQLite datatype is associated with the value itself and not the column data type that I initially assigned.
I was going to alter the column data type when user updates the App, but it doesn't seem necessary (nor possible without dropping and recreating table). Am I reading this correctly or am I making a mistake?
The App seems to work well when I test updating, but I want to make sure I am not missing anything. Any feedback is appreciated.
Changing the type name in the column definition can affect the affinity.
This might change the type of some values (for example, attempting to store the string '123' in an INTEGER column will result in the number 123), and might change how comparisons work (WHERE SomeColumn = ? will try to convert the value to the same type as the column's affinity).
So you should change the type name only if you are sure that your app handles the values in this column correctly.
For my Android app, I want to save data using sqlite with this format:
name, date, attr1, attr2, attr3,...
These are the requirements:
each date can only contain each name once
there can be a variable number of attributes(numbers) for each name
each specific name has the same number of attributes
The app will be used to track events throughout the day. Events can have zero or more numeric properties.
The questions are: is sqlite the best way to store things here? If so how do I design my database? What other ways are there to store this kind of data?
is sqlite the best way to store things here?
This will depend on a number of other factors, such as how the data will be queried and used, the volume of transactions, data growth and retention, etc. From what you've described, though, SQLite is a great option, offering functionality out-of-the-box that supports some of your requirements directly, and is commonly used in such cases.
If you don't have much experience with relational databases, implementing this functionality may seem difficult at first, but like learning a new language or framework, it will get easier with time.
If so how do I design my database?
Let's step through each of your enumerated requirements...
each date can only contain each name once
SQLite supports the UNIQUE constraint. For example, if your columns were named name and date, you could add the following to your CREATE TABLE statement:
UNIQUE(name, date)
(A more complete CREATE TABLE statement is in the next example below, and it includes this constraint.)
This constraint prevents the insertion of rows with name/date pairs that already exist. Using android.database.sqlite.SQLiteDatabase, if you attempt to insert a row into the table with a duplicate name/date pair, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
there can be a variable number of attributes(numbers) for each name
This is a textbook case for normalizing the database, putting your data into multiple tables. For example:
CREATE TABLE names (
name_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
name TEXT NOT NULL,
date DATETIME,
UNIQUE(name, date));
CREATE TABLE attrs (
name_id INTEGER NOT NULL,
attr_value INTEGER NOT NULL,
FOREIGN KEY(attr_value) REFERENCES names(name_id));
Your queries that retrieve attribute data would then JOIN the two tables. Since you indicated that "Events can have zero or more numeric properties", a LEFT OUTER JOIN might be most appropriate, as it will return names and dates even if there are no attributes.
Here's an example query, by name:
SELECT n.name, n.date, a.attr_value
FROM names AS n
LEFT OUTER JOIN attrs AS a
ON n.name_id = a.name_id
WHERE n.name = 'SMITH'
ORDER BY n.name, n.date, a.attr_value;
This query would return results like the following:
name date attr_value
--------------- ---------- ------------
SMITH 2015-02-13 1027
SMITH 2015-02-13 4426
SMITH 2015-02-13 8390
SMITH 2015-02-20 4426
SMITH 2015-02-20 8152
SMITH 2015-02-20 9328
You can then iterate through and process these results in java. If your results include multiple names and/or dates, then in your loop you should keep track of the last used name and date. If the name/date in the current record is the same, the attribute belongs to the current one. If the name/date is different, then this is a new one.
Note that this approach to your database design is flexible, allowing you to query on the attributes, for instance, to see what name/date pairs are associated.
Also note that there is a FOREIGN KEY constraint on the attrs table, meaning that if you attempt to insert a record into that table with a name_id that does not exist in the names table, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
each specific name has the same number of attributes
You will need to accommodate this requirement in your java code, probably doing some checks in the database prior to performing an INSERT.
What other ways are there to store this kind of data?
Flat files, JSON, XML, third-party data stores (with their own libraries), to name a few.
I'm not sure but I think the best way to achieve your requirement is to use sqlite and to solve your problem you can have 3 columns only. One for the name and one for the date and the other contains a JSON array that represents the rest of the attributes.
I am trying to create sqllite db for my Android application use
Is it bad habit to have all the data types as text? The reason is even though some data can be integer in nature (like number of items.. Etc) but many times I need to display the values as a string or get it as input from user. So I thought for easier manipulation I will just make the datatype in the db as text
Thoughts?
The first thing to understand here is SQLite Storage Classes. SQLite allows any data type to be stored in a table's columns, the actual data type defined by the table is just a hint to the database engine.
This means that even if you define a column as type INTEGER, you can still store text in it.
That being said, I haven't found a use case for storing arbitrary data types in a column, or a case for disregarding the defined data types. For readability purposes, it would probably be a good idea to type things properly and obey the defined types.
There is a thing that I can not understand about Cursor.getType(), can anyone explain why do I get this stupid exception when I want to get columns type if cursor has no record but there are columns? I mean, if cursor has record there is no problem, I can use getType method to get columns type without any problem but if there is no any record it throws this exception out.
The question is why must I need records to get columns type? Why just knowing columns name is not enough to get types of it? Is this ridiculous?
It's because SQLite uses dynamic typing:
Most SQL database engines (every SQL database engine other than
SQLite, as far as we know) uses static, rigid typing. With static
typing, the datatype of a value is determined by its container - the
particular column in which the value is stored.
SQLite uses a more general dynamic type system. In SQLite, the
datatype of a value is associated with the value itself, not with its
container.
http://www.sqlite.org/datatype3.html
So no value, no data type.