I have a table with data and having a column with a date value.
Table A
table_id date
So think of the table as to have more than thousands of rows and entries. What I want to do is to select all the distinct months and years from the table, so the output would probably look something like this:
09 2013 09-2013
12 2014 12-2014
01 2015 01-2015
03 2015 03-2015
05 2015 05-2015
Maybe something like this? If this is possible?
dao.queryBuilder().distinct().selectColumns("strftime('%mm-%yyyy', date)").query();
But I can't seem to make the above work the way I want it to. My model also uses the DataType.DATE on its date field.
Is there some way I can use the distinct() method of OrmLite to return this kind of list? Any help would be greatly appreciated! Thank you!
SqlFiddleDemo
CREATE TABLE TableA(table_id INT, `date` DATE);
INSERT INTO TableA
VALUES (1, '2014-01-21'), (2, '2014-01-23'),
(3, '2015-01-21'), (4, '2015-01-01'), (5, '2014-07-21'), (6, '2012-01-01');
SELECT DISTINCT
STRFTIME('%m-%Y', t.`date`) AS Output
FROM TableA t
ORDER BY t.`date`;
I don't know OrmLite but I hope you manage to convert this solution or just use pure SQL with your ORM.
EDIT:
SELECT DISTINCT
STRFTIME('%m', t.`date`) AS Month,
STRFTIME('%Y', t.`date`) AS `Year`,
STRFTIME('%m-%Y', t.`date`) AS Output
FROM TableA t
ORDER BY t.`date`;
To be sure we are talking about the same thing : ORMLite is a "Object Relational Mapping Lite (ORM Lite) provides some simple, lightweight functionality for persisting Java objects to SQL databases [...]."
It's not a just a wrapper to some SQL raw queries : it provides a way to create models (some POJO) that will be persisted in a database, without having to handle SQL constraints and strong knowledge. You will be able to store objects and retrieve them using plain Java objects.
The selectColumns(String) method take a column name as argument. You are passing a raw SQL statement.
If you read the source code of this method, you have this :
public QueryBuilder<T, ID> selectColumns(String... columns) {
for (String column : columns) {
addSelectColumnToList(column);
}
return this;
}
and then
private void addSelectColumnToList(String columnName) {
verifyColumnName(columnName);
addSelectToList(ColumnNameOrRawSql.withColumnName(columnName));
}
and to finish :
/**
* Verify the columnName is valid and return its FieldType.
*
* #throws IllegalArgumentException
* if the column name is not valid.
*/
protected FieldType verifyColumnName(String columnName) {
return tableInfo.getFieldTypeByColumnName(columnName);
}
So, with introspection, ORMLite will try to find the given column name in the fields of your model. Your string is clearly NOT a column name :)
Also, for a more "conceptual" concern : I think you should not convert the date format from your request. Actually, ORMLite is in charge to persist models in your database. He is not designed to format your data correctly for processing or display. If after persisting data in the DB you need to display it another, you will have to make the "parse stuff" at this moment, not before.
To be summarize :
Use the real column name of the date field
Continue to use distinct() the way you do it
Parse the data to display it correctly if needed (let ORMLite choose the way it stores and load it)
Sources
http://ormlite.com/javadoc/ormlite-core/doc-files/ormlite_3.html#QueryBuilder-Capabilities
https://github.com/j256/ormlite-core/blob/master/src/main/java/com/j256/ormlite/stmt/QueryBuilder.java
https://github.com/j256/ormlite-core/blob/master/src/main/java/com/j256/ormlite/stmt/StatementBuilder.java
Related
Interesting issue while using SQLite in Android. I am seeing an inconsistency in the string length and quoting of a string between what is stored in the database and the materialized value seen in Java.
We are using an ORM called SugarORM to query the DB, but I've traced the offending code to the internal android.database.sqlite.SQLiteCursor class used within SugarORM, specifically the cursor.getString(columnIndex) method.
I have a string in the database that is an ISO data string 2019-03-25T19:19:39.664Z and is stored in a VARCHAR column . I have confirmed using DB Browser for SQLite that the length of the string as its stored in the database is indeed 24 characters. SELECT LENGTH(MyStringColumn) FROM MyTable WHERE ...
When I get the value of this string via cursor.getString(columnIndex), it is returning the string "2019-03-25T19:19:39.664Z". Notice the leading and trailing quotes. Java reports to me that the string is 26 characters long.
Any value that I store in this column that is not an ISO data does not have this behavior. I tried tracing the SQLiteCursor source back, but ultimately it ends up being a Native method and that's where my skill set stops.
Can anyone explain what might be going on here? I am probably just going to write a wrapper around my queries to get rid of the quotes, but its all very perplexing. The date string is being fed to a JavaScript interpreter and causing it to fail when creating a JavaScript Date object.
If it helps, I have replicated the behavior on both my S7 physical device and a Pixel 6 emulator.
As a quick get around you could use :-
SELECT length(replace(mystringcolumn,'"','')) FROM mytable;
or before using the original SELECT use :-
UPDATE mytable SET mystringcolumn = replace(mystringcolumn,'"','');
If this doesn't fix the issue, then for some reason it is the code that retrieves the data that is at fault.
e.g. consider :-
DROP TABLE IF EXISTS mytable;
CREATE TABLE IF NOT EXISTS mytable (mystringcolumn VARCHAR);
INSERT INTO mytable VALUES('2019-03-25T19:19:39.664Z'),('"2019-03-25T19:19:39.664Z"');
SELECT length(mystringcolumn), length(replace(mystringcolumn,'"','')) FROM mytable;
which results in :-
i.e. The 2nd row, 2nd column retrieves the appropriate value by using the replace function to strip of the quotes, if they exist.
As to why the quotes exist could depend upon either the way that the data is inserted (perhaps you have inadvertenly coded the quotes but the db being looked at isn't the actual database as copied from the App) or the way in which the data is being retrieved that for some reason adds them.
I don't believe it likely that the Cursor getString method has a bug in which the quotes are added, otherwise such an issue would likely be a recurring issue.
I have an SQLite database within my Android application, which stores dates as integers. These integers are derived from a call to Java.util.Date.getTime();. I am trying to run a raw query of my database to get a Cursor to pass to a CursorAdapter and display in a ListView, but the date is stored as an integer as returned by getTime().
To keep my program simple, I would like to avoid using a SimpleArrayAdapter, and stick with the CursorAdapter.
Is it somehow possible to format the integer within the date colum as mm-dd-yyyy so that the column of the table, that the cursor is pointing to, contains properly formatted values rather than the integer that was returned by Java.util.Date.getTime(); when I added the item to the database?
SELECT strftime("%m-%d-%Y", date_col, 'unixepoch') AS date_col
Your code will work if it expects a result set column in that format called date_col.
EDIT: One thing you need to watch out for is that getTime uses milliseconds since 1970, while standard UNIX time (including SQLite) uses seconds.
The Java.util.Date.getTime(); method is returning an integer that represents the "unix time".
The simplest way to read this number as a date is by storing it as-is, and reading it using the following Sqlite query:
SELECT strftime('%m-%d-%Y', 1092941466, 'unixepoch');
which returns:
08-19-2004
If you need another format, you can use the strftime function to format is as you like, or any of the other date formats and functions available.
You'll have to, as Matthew Flaschen points out in a commend below, divide the date by 1000 before you are able to use them in this way. "Real" unix times are measured in seconds since the epoch, and Java.util.Date.getTime(); returns milliseconds since epoch.
SQLite uses static rigid typing. With static typing, the datatype of a value is determined by its container - the particular column in which the value is stored.
Any value stored in the SQLite database has one of the following storage class:
NULL
INTEGER
REAL
TEXT
BLOB
so I am not sure what you meant by but the date is stored as a long, unhelpful integer.
For more details please refer to Datatypes In SQLite Version 3. For further information on storing date/time in SQLite please refer to SQL As Understood By SQLite.
I hope this helps.
For my Android app, I want to save data using sqlite with this format:
name, date, attr1, attr2, attr3,...
These are the requirements:
each date can only contain each name once
there can be a variable number of attributes(numbers) for each name
each specific name has the same number of attributes
The app will be used to track events throughout the day. Events can have zero or more numeric properties.
The questions are: is sqlite the best way to store things here? If so how do I design my database? What other ways are there to store this kind of data?
is sqlite the best way to store things here?
This will depend on a number of other factors, such as how the data will be queried and used, the volume of transactions, data growth and retention, etc. From what you've described, though, SQLite is a great option, offering functionality out-of-the-box that supports some of your requirements directly, and is commonly used in such cases.
If you don't have much experience with relational databases, implementing this functionality may seem difficult at first, but like learning a new language or framework, it will get easier with time.
If so how do I design my database?
Let's step through each of your enumerated requirements...
each date can only contain each name once
SQLite supports the UNIQUE constraint. For example, if your columns were named name and date, you could add the following to your CREATE TABLE statement:
UNIQUE(name, date)
(A more complete CREATE TABLE statement is in the next example below, and it includes this constraint.)
This constraint prevents the insertion of rows with name/date pairs that already exist. Using android.database.sqlite.SQLiteDatabase, if you attempt to insert a row into the table with a duplicate name/date pair, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
there can be a variable number of attributes(numbers) for each name
This is a textbook case for normalizing the database, putting your data into multiple tables. For example:
CREATE TABLE names (
name_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
name TEXT NOT NULL,
date DATETIME,
UNIQUE(name, date));
CREATE TABLE attrs (
name_id INTEGER NOT NULL,
attr_value INTEGER NOT NULL,
FOREIGN KEY(attr_value) REFERENCES names(name_id));
Your queries that retrieve attribute data would then JOIN the two tables. Since you indicated that "Events can have zero or more numeric properties", a LEFT OUTER JOIN might be most appropriate, as it will return names and dates even if there are no attributes.
Here's an example query, by name:
SELECT n.name, n.date, a.attr_value
FROM names AS n
LEFT OUTER JOIN attrs AS a
ON n.name_id = a.name_id
WHERE n.name = 'SMITH'
ORDER BY n.name, n.date, a.attr_value;
This query would return results like the following:
name date attr_value
--------------- ---------- ------------
SMITH 2015-02-13 1027
SMITH 2015-02-13 4426
SMITH 2015-02-13 8390
SMITH 2015-02-20 4426
SMITH 2015-02-20 8152
SMITH 2015-02-20 9328
You can then iterate through and process these results in java. If your results include multiple names and/or dates, then in your loop you should keep track of the last used name and date. If the name/date in the current record is the same, the attribute belongs to the current one. If the name/date is different, then this is a new one.
Note that this approach to your database design is flexible, allowing you to query on the attributes, for instance, to see what name/date pairs are associated.
Also note that there is a FOREIGN KEY constraint on the attrs table, meaning that if you attempt to insert a record into that table with a name_id that does not exist in the names table, a SQLiteConstraintException will be thrown at runtime. You will need to handle this exception in your java code.
each specific name has the same number of attributes
You will need to accommodate this requirement in your java code, probably doing some checks in the database prior to performing an INSERT.
What other ways are there to store this kind of data?
Flat files, JSON, XML, third-party data stores (with their own libraries), to name a few.
I'm not sure but I think the best way to achieve your requirement is to use sqlite and to solve your problem you can have 3 columns only. One for the name and one for the date and the other contains a JSON array that represents the rest of the attributes.
I currently have a .csv file with several unlabeled columns of data, which to my knowledge translate to the following datatypes in sqlite:
datetime (in the format 7/19/2011 12:00:00 PM) -> numeric
double -> real
char(1) -> text
float -> real
I can create the database by doing the following:
sqlite> create table myTable (myVar1 numeric, myVar2 real, myVar3 text, myVar4 real);
sqlite> .separator ","
sqlite> .import myFile.csv myTable
Then I copy and paste the newly created myTable.db into the "assets" folder in my project in eclipse. I make a DatabaseHelper class that extends SQLiteOpenHelper, and then I can start using and reading from the database in my Android project.
Am I getting this right? I've never used a database before and I've seen so many vastly different instructions on doing this. Some of my questions are-- do I have to label the columns of my .csv file? Is my .csv file not "simple" enough to just use .import and I'll need to find a program to translate it? I've come across sites saying that I need to rename something (which I don't seem to have) to "_id", and I don't know what this is, where this is, or how to do this, or if it's even necessary, or what it's for. What else am I missing?
I think you are getting it "right" except for that first datetime column. You should use the TEXT type, not a numeric type.
Also, you can inspect your data after the import to see if all is well, especially with that datetime field:
SELECT * FROM myTable ORDER BY RANDOM() LIMIT 10;
UPDATE
In response to the OP's last comment: my understanding of how you store date(time) is that it depends on your context. So if the date in the flat file is in the format "7/19/2011 12:00:00 PM", then without any transformation it'll be imported as TEXT anyway.
Importing csv into database is 15 lines of code task and it gives you more control over this process.
Table columns names like "var3" are just terrible, however there is no need to rename it in database you can just use sql aliases:
select myVar1 _id, myVar2 from myTable
_id is common name for primary key column of table (it's usually numeric column witch must be unique). Every ADK class using datastore assumes relays on it, so it's nice to use this convention.
If you just want read only database you can prepare db locally and find some tutorials how to include it into your app.
I have an SQLite database within my Android application, which stores dates as integers. These integers are derived from a call to Java.util.Date.getTime();. I am trying to run a raw query of my database to get a Cursor to pass to a CursorAdapter and display in a ListView, but the date is stored as an integer as returned by getTime().
To keep my program simple, I would like to avoid using a SimpleArrayAdapter, and stick with the CursorAdapter.
Is it somehow possible to format the integer within the date colum as mm-dd-yyyy so that the column of the table, that the cursor is pointing to, contains properly formatted values rather than the integer that was returned by Java.util.Date.getTime(); when I added the item to the database?
SELECT strftime("%m-%d-%Y", date_col, 'unixepoch') AS date_col
Your code will work if it expects a result set column in that format called date_col.
EDIT: One thing you need to watch out for is that getTime uses milliseconds since 1970, while standard UNIX time (including SQLite) uses seconds.
The Java.util.Date.getTime(); method is returning an integer that represents the "unix time".
The simplest way to read this number as a date is by storing it as-is, and reading it using the following Sqlite query:
SELECT strftime('%m-%d-%Y', 1092941466, 'unixepoch');
which returns:
08-19-2004
If you need another format, you can use the strftime function to format is as you like, or any of the other date formats and functions available.
You'll have to, as Matthew Flaschen points out in a commend below, divide the date by 1000 before you are able to use them in this way. "Real" unix times are measured in seconds since the epoch, and Java.util.Date.getTime(); returns milliseconds since epoch.
SQLite uses static rigid typing. With static typing, the datatype of a value is determined by its container - the particular column in which the value is stored.
Any value stored in the SQLite database has one of the following storage class:
NULL
INTEGER
REAL
TEXT
BLOB
so I am not sure what you meant by but the date is stored as a long, unhelpful integer.
For more details please refer to Datatypes In SQLite Version 3. For further information on storing date/time in SQLite please refer to SQL As Understood By SQLite.
I hope this helps.