I have a .csv file that I turned into an SQLite database with the first column as a date and time (as a TEXT datatype because it's in the format 7/20/2011 12:00:00 PM), the second column as the subsecond (a REAL, like 0.142857), and the rest of the columns as data about that time. I will be using it as a read-only db in my Android app, in which the user selects a start time and an end time and then the data is displayed graphically for that time period.
This is my first time using a database and I'm not sure how to work with this. I believe I need a unique NUMERIC primary key for each row but simply labeling them by row number is a waste of space. The first column is not unique and is a TEXT datatype anyway, though if the subsecond was to be somehow incorporated then it would be unique. But I really can't re-write the database because it has 65534 rows... How do you suggest I access specific times in this database?
In Android you need a column named _ID in your database (else you'll face some issues later on). You will use that as the primary key.
Dates and times can be stored in SQLite in the form of a text column in the following format (See http://www.sqlite.org/datatype3.html):
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS")
If your database is static, simply use a tool such as SQLite Database Browser to convert it to a format convenient for Android.
If your database is local and external(not remote), than you must have _id and another another table android_metadata which will hold the locale.
If your database was remote. Yes, you can it is only matter of speed if you are write, since you don't. Using WHERE clause will do the work.
Every date can be converted to numeric timestamp quite easy:
Date date = new Date();
long timestamp = date.getTime();
Date otherDate = new Date(timestamp);
Numbers are MUCH easier and faster to process than text fields. If you are completely sure, that you have unique data within column you can use it as primary key.
Importing csv file into table should be also easy:
//just open file in some known way and read it line by line
// we have declared String[] data, String line is line read from your csv somewhere earlier
// in code
data = line.split(",");
//here you have to process values, and execute insert statement
You have to create indexes on every column which will be used to search or order data. Please be also aware, that rows in table has no "default", "natural" or any other order. If you execute this same select statement twice you can get two totally different results in meaning of sorting. So simple select statement should look like that:
select
_id, other_colum_name, some_other_column
from
table_name
where
column_name = 5
and other_column_name = 3
order by
column_name;
Related
I have an SQLite database within my Android application, which stores dates as integers. These integers are derived from a call to Java.util.Date.getTime();. I am trying to run a raw query of my database to get a Cursor to pass to a CursorAdapter and display in a ListView, but the date is stored as an integer as returned by getTime().
To keep my program simple, I would like to avoid using a SimpleArrayAdapter, and stick with the CursorAdapter.
Is it somehow possible to format the integer within the date colum as mm-dd-yyyy so that the column of the table, that the cursor is pointing to, contains properly formatted values rather than the integer that was returned by Java.util.Date.getTime(); when I added the item to the database?
SELECT strftime("%m-%d-%Y", date_col, 'unixepoch') AS date_col
Your code will work if it expects a result set column in that format called date_col.
EDIT: One thing you need to watch out for is that getTime uses milliseconds since 1970, while standard UNIX time (including SQLite) uses seconds.
The Java.util.Date.getTime(); method is returning an integer that represents the "unix time".
The simplest way to read this number as a date is by storing it as-is, and reading it using the following Sqlite query:
SELECT strftime('%m-%d-%Y', 1092941466, 'unixepoch');
which returns:
08-19-2004
If you need another format, you can use the strftime function to format is as you like, or any of the other date formats and functions available.
You'll have to, as Matthew Flaschen points out in a commend below, divide the date by 1000 before you are able to use them in this way. "Real" unix times are measured in seconds since the epoch, and Java.util.Date.getTime(); returns milliseconds since epoch.
SQLite uses static rigid typing. With static typing, the datatype of a value is determined by its container - the particular column in which the value is stored.
Any value stored in the SQLite database has one of the following storage class:
NULL
INTEGER
REAL
TEXT
BLOB
so I am not sure what you meant by but the date is stored as a long, unhelpful integer.
For more details please refer to Datatypes In SQLite Version 3. For further information on storing date/time in SQLite please refer to SQL As Understood By SQLite.
I hope this helps.
For my Android app, I want to save data using sqlite with this format:
name, date, attr1, attr2, attr3,...
These are the requirements:
each date can only contain each name once
there can be a variable number of attributes(numbers) for each name
each specific name has the same number of attributes
The app will be used to track events throughout the day. Events can have zero or more numeric properties.
The questions are: is sqlite the best way to store things here? If so how do I design my database? What other ways are there to store this kind of data?
is sqlite the best way to store things here?
This will depend on a number of other factors, such as how the data will be queried and used, the volume of transactions, data growth and retention, etc. From what you've described, though, SQLite is a great option, offering functionality out-of-the-box that supports some of your requirements directly, and is commonly used in such cases.
If you don't have much experience with relational databases, implementing this functionality may seem difficult at first, but like learning a new language or framework, it will get easier with time.
If so how do I design my database?
Let's step through each of your enumerated requirements...
each date can only contain each name once
SQLite supports the UNIQUE constraint. For example, if your columns were named name and date, you could add the following to your CREATE TABLE statement:
UNIQUE(name, date)
(A more complete CREATE TABLE statement is in the next example below, and it includes this constraint.)
This constraint prevents the insertion of rows with name/date pairs that already exist. Using android.database.sqlite.SQLiteDatabase, if you attempt to insert a row into the table with a duplicate name/date pair, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
there can be a variable number of attributes(numbers) for each name
This is a textbook case for normalizing the database, putting your data into multiple tables. For example:
CREATE TABLE names (
name_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
name TEXT NOT NULL,
date DATETIME,
UNIQUE(name, date));
CREATE TABLE attrs (
name_id INTEGER NOT NULL,
attr_value INTEGER NOT NULL,
FOREIGN KEY(attr_value) REFERENCES names(name_id));
Your queries that retrieve attribute data would then JOIN the two tables. Since you indicated that "Events can have zero or more numeric properties", a LEFT OUTER JOIN might be most appropriate, as it will return names and dates even if there are no attributes.
Here's an example query, by name:
SELECT n.name, n.date, a.attr_value
FROM names AS n
LEFT OUTER JOIN attrs AS a
ON n.name_id = a.name_id
WHERE n.name = 'SMITH'
ORDER BY n.name, n.date, a.attr_value;
This query would return results like the following:
name date attr_value
--------------- ---------- ------------
SMITH 2015-02-13 1027
SMITH 2015-02-13 4426
SMITH 2015-02-13 8390
SMITH 2015-02-20 4426
SMITH 2015-02-20 8152
SMITH 2015-02-20 9328
You can then iterate through and process these results in java. If your results include multiple names and/or dates, then in your loop you should keep track of the last used name and date. If the name/date in the current record is the same, the attribute belongs to the current one. If the name/date is different, then this is a new one.
Note that this approach to your database design is flexible, allowing you to query on the attributes, for instance, to see what name/date pairs are associated.
Also note that there is a FOREIGN KEY constraint on the attrs table, meaning that if you attempt to insert a record into that table with a name_id that does not exist in the names table, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
each specific name has the same number of attributes
You will need to accommodate this requirement in your java code, probably doing some checks in the database prior to performing an INSERT.
What other ways are there to store this kind of data?
Flat files, JSON, XML, third-party data stores (with their own libraries), to name a few.
I'm not sure but I think the best way to achieve your requirement is to use sqlite and to solve your problem you can have 3 columns only. One for the name and one for the date and the other contains a JSON array that represents the rest of the attributes.
I found on Stack similar questions like this How to update an entire column in sqlite? But they don't explain me my how to solve my task.
Say, I have a db with 5 columns and 5 records in it. What i need is to update the last column "date" with the values of unix time that differs by 1 sec. So i need to put values 1406820974139, 1406820974140, 1406820974141, 1406820974142, 1406820974143, 1406820974144.
How to do it using ContentValues? As i got i have to loop five times to create new ContentValues object and update one record at a time (maybe using db.startTransaction() syntax).
My question is is there a way to put all values at a time into one ContentValues object and write in them into DB? Or maybe the better way is to use rawQuery using native SQL syntax as explained in How to update an entire column in sqlite?
In theory, it would be possible to put all the values into a single SQL statement:
UPDATE MyTable
SET date = CASE _id
WHEN 5 THEN 1406820974139
WHEN 17 THEN 1406820974140
WHEN 23 THEN 1406820974141
WHEN 69 THEN 1406820974142
WHEN 666 THEN 1406820974143
END;
However, just creating one ContentValues object for each row is easier than constructing this command.
so that we know which date should go to which row? what is the cririteria to differentiate the rows?
a relational db table is different from say an excel table. there is no implicit row order (if you always see the rows in the same order,you can consider it a kind of coincidence,you can not rely on this like you do in excel), in a db table you need to have a column(or a group of them) with unique values which you use in your queries to identify each of your records.
so you need to be more clear in your question. what date should go to which record (identified by what?). there is no implicit row number, if you want it, add an autoincrement PK column.
then you could for instance use something along the lines of
UPDATE table SET column5= 1406820974140+PKcolumn
where 1406820974140 is the start date you have to choose, depending on what you are up to
I have a table named as Attendance, Here is the structure given below.
id fname lname roll_no date time
________________________________________________________
1 Qadir Hussain 08cs18 19/04/2013 8:45am
2 Qadir Hussain 08cs18 19/04/2013 8:50am
_______________________________________________________
i want to insert the record having roll_no = 08cs18 only one time per day. not more than once per day.
I can restrict this via if/else. but is it possible to restrict this via sqlite query?
Edit
Acctually i m making an app of studnets attendance, I m using the QR_code to scan encoded roll_no. once a student scan its card for the first time it should insert the record (i.e id = 1) if user again scan it on the same day it should not insert. means a particular student should have a attendance record only one time per day.
For both databases, make a unique key constraint on roll_no and date.
CREATE UNIQUE INDEX daily_roll_no ON Attendance (roll_no,date)
Then for mysql make your insert with the IGNORE clause. For sqlite, use OR IGNORE.
INSERT IGNORE INTO Attendance ....
INSERT OR IGNORE INTO Attendance ....
mysql:
ignore clause
create index
sqlite:
ignore clause
create index
You can do this in a single SQL:
INSERT OR REPLACE INTO <tablename>
(<identifier>, <time inserted>, column1, column2, ...)
SELECT <identifier>, <time inserted>,
COALESCE(column1,'new value 1'),
COALESCE(column2,'new value 2'), ...
FROM <tablename>
WHERE <identifier> = ...
AND <time inserted> > <now minus one day>
The basic idea is: using INSERT OR REPLACE you can update an existing row or insert a new one depending on one or more conditions. In this case I update the row with the same values, if the row is not older than one day.
If the line is not existing or older than one day, you can assign new values using COALESCE.
I know this is only a kind of outline but may be it helps though .... Cheers!
I have an SQLite database within my Android application, which stores dates as integers. These integers are derived from a call to Java.util.Date.getTime();. I am trying to run a raw query of my database to get a Cursor to pass to a CursorAdapter and display in a ListView, but the date is stored as an integer as returned by getTime().
To keep my program simple, I would like to avoid using a SimpleArrayAdapter, and stick with the CursorAdapter.
Is it somehow possible to format the integer within the date colum as mm-dd-yyyy so that the column of the table, that the cursor is pointing to, contains properly formatted values rather than the integer that was returned by Java.util.Date.getTime(); when I added the item to the database?
SELECT strftime("%m-%d-%Y", date_col, 'unixepoch') AS date_col
Your code will work if it expects a result set column in that format called date_col.
EDIT: One thing you need to watch out for is that getTime uses milliseconds since 1970, while standard UNIX time (including SQLite) uses seconds.
The Java.util.Date.getTime(); method is returning an integer that represents the "unix time".
The simplest way to read this number as a date is by storing it as-is, and reading it using the following Sqlite query:
SELECT strftime('%m-%d-%Y', 1092941466, 'unixepoch');
which returns:
08-19-2004
If you need another format, you can use the strftime function to format is as you like, or any of the other date formats and functions available.
You'll have to, as Matthew Flaschen points out in a commend below, divide the date by 1000 before you are able to use them in this way. "Real" unix times are measured in seconds since the epoch, and Java.util.Date.getTime(); returns milliseconds since epoch.
SQLite uses static rigid typing. With static typing, the datatype of a value is determined by its container - the particular column in which the value is stored.
Any value stored in the SQLite database has one of the following storage class:
NULL
INTEGER
REAL
TEXT
BLOB
so I am not sure what you meant by but the date is stored as a long, unhelpful integer.
For more details please refer to Datatypes In SQLite Version 3. For further information on storing date/time in SQLite please refer to SQL As Understood By SQLite.
I hope this helps.