This is very similar to another question that got closed as not a real question. I tried to edit it to make it valid for reopening but was told I would be better off asking a new question.
I'm developing on android and need to store datetime values in a sqlite database to track repeating events that will generate notifications. I will also need to be able to query the database based on time ranges.
The SQLite documentation states that it does not support specific date types but that dates can be represented using TEXT, REAL, or INTEGER types:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
The advantages of each initially seem to be:
TEXT useful for readability in the database with ability to be displayed directly later (no transformation necessary), but costly if calculations need to be performed on them. Introduces possible error from time zones.
REAL useful for dates before 1970, good for calculations or date comparisons. Does not represent time of day, only days.
INTEGER useful for calculations or datetime comparisons, very good compatibility since it is a widely supported standard.
Does this sound correct? Would using an INTEGER for datetimes make querying time ranges noticeably faster than when using TEXT? Anything else I haven't considered?
Given my use case, which of these solutions would be best?
TEXT useful for readability in the database with ability to be displayed directly later (no transformation necessary)
The ISO format is usually not used for display; you have to transform this, too.
(This format is more useful for debugging.)
costly if calculations need to be performed on them
With databases, the bottleneck usually is the I/O.
I doubt you will ever see a query where the actual format of date values would make a noticeable difference.
Introduces possible error from time zones.
The TEXT format does not have a time zone specifier, but neither have the other formats.
If you say that all your DB values are in UTC, there is no difference.
REAL useful for dates before 1970
All formats support all years between 0 and 9999. (Integers can be negative.)
Does not represent time of day, only days.
Time of day is represented as a fractional value.
INTEGER ... very good compatibility since it is a widely supported standard.
But Java uses milliseconds, not seconds.
Anything else I haven't considered?
The default output format of most of the built-in date functions is TEXT.
Given my use case, which of these solutions would be best?
I'd say TEXT, but I don't think there would be much of a difference.
Does not represent time of day, only days.
That's what the fractional portion of the REAL is for.
Would using an INTEGER for datetimes make querying time ranges noticeably faster than when using TEXT?
Most likely. I cannot envision a scenario in which string comparisons would be faster than integer comparisons, particularly for indexed columns in queries.
Anything else I haven't considered?
I have no idea if you have considered the effects of gamma rays on man-in-the-moon marigolds, but that's not important here. :-)
Given my use case, which of these solutions would be best?
INTEGER, IMHO.
Answers here are going to be primarily opinion, but if I were you I'd use the INTEGER type and store the unix timestamp. It seems less dependent on format conversions/parsing, and is the most universally supported standard.
Store UTC date time as eight byte 64 bit integer in SQLite. Millis sence 1970.
long time= System.currentTimeMillis();
With an indexed database you'll have excellent response for about 10k rows on order by and between datetime reading data.
Leave the time zone manipulations to the view layer because a database with all UTC times will work world wide.
Don't store dates as text that's so ancient and has no place in a modern application.
I would avoid storing just the dates because modern apps take into account when an event occured. You can store datetimes with 8 bytes. But if you must you can simply put the iso date into an integer 1999-12-31 as 19991231 and store a 4 byte integer in sqllite.
Related
This question already has answers here:
System.currentTimeMillis() vs. new Date() vs. Calendar.getInstance().getTime()
(8 answers)
Closed 3 years ago.
I searched a lot I found some solutions for that and got confused to select which method to use to get a timestamp in android and which one is the best.
found
Get by using the android Calendar
Calendar.getInstance().getTimeInMillis()
By using the Date
new Date().getTime()
By using System
Long tsLong = System.currentTimeMillis()/1000;
String ts = tsLong.toString();
Please help me to understand.
System.currentTimeMillis() is obviously the most efficient since it does not even create an object, but new Date() is really just a thin wrapper about a long, so it is not far behind. Calendar, on the other hand, is relatively slow and very complex, since it has to deal with the considerable complexity and all the oddities that are inherent to dates and times (leap years, daylight savings, timezones, etc.).
It's generally a good idea to deal only with long timestamps or Date objects within your application, and only use Calendar when you actually need to perform date/time calculations or to format dates for displaying them to the user. If you have to do a lot of this, using Joda Time is probably a good idea, for the cleaner interface and better performance.
for full discussion please check the link
There are answers all over StackOverflow that quote the same section of the documentation (as follows):
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.
However I think that some people (question askers and answer-ers) are confused about or omit the following detail regarding table creation, where DateTime is actually a NUMERIC type internally, and should be created as a DATETIME or NUMERIC to avoid performance issues relating to casting.
I may be misunderstanding something, but I do see answers out there telling people to create a TEXT column for datetime, which doesn't seem efficient.
Question
Given that I can query a datetime column as TEXT, REAL, or INTEGER, what is the most efficient way to handle datetimes, especially since a column of DATETIME results in a NUMERIC.
Does a columnType of NUMERIC/DateTime result in casting delays due to comparisons of TEXT, REAL, or Integer?
Could a TEXT column type and a TEXT query type for SQLite be faster?
Could a REAL column type and a REAL query type for SQLite be faster?
Could a INTEGER column type and a INTEGER query type for SQLite be faster?
How would I convert a DATETIME into a NUMERIC for faster queries?
For the benefit of those asking the question above, or answering it with excessive focus on theory, I offer an answer that attempts to address the intent of your question... "what is the most efficient way to handle datetimes" in the context of "especially since a column of DATETIME results in a NUMERIC." And you also mention mobile devices in your comments.
First, in the context of mobile, rarely will you need to address large datasets. So the most efficient code you write is code that isn't confusing when you need to change it and works even as the framework evolves.
So a string or an integer may work, depending on what you are most comfortable working with. If you get confused working with dates as integers, use Strings. If you have to expose the work to a group that likes strings, go for it.
That said, if you are truly device performance driven, then integer comparisons are the fastest when using indexes. And if you have a performance problem, it is usually best solved using an index.
So, to answer question all of the questions - yes, casting results in delays, but those delays usually do not matter because you are dealing with small datasets. However, when it does matter, integer comparisons are the fastest (which is well documented like the questions you cite), so convert your DATETIME values to integers to optimize query performance.
Your title then specifically mentions "INSERT" and "UPDATE" - which then raises the question, "do you really need an index if INSERT and UPDATE performance is your concern? Indexing adds overhead to INSERT and UPDATE statements." This is also usually does not matter due to the relatively high processing power and the inability to create situations with large numbers of INSERT or UPDATE statements in mobile devices in general.
However, if this is really an issue, then standard DB optimization for transaction vs. query becomes the issue. You may need to build two databases, one to support INSERT/UPDATE activity and another for querying, with some synchronization process. Or you may have to make conscious trade-offs for the INSERT/UPDATE vs. SELECT situations you encounter.
In other words, like a very many detailed questions, the answer largely depends on the particular problems your app/software is facing, the needs/demands of the user, the hardware you target and/or support and the team that is building/maintaining the product.
Even so, the short answer is "it probably doesn't matter" and if it does, then use INTEGERs unless you can be more specific about your particular use case.
These questions presuppose facts which are not actually true.
SQLite has no column types. There are only column affinities, which do not restrict what types can be actually stored in a column.
There is no special DATETIME type. When the documentation mentions TEXT and REAL and INTEGER, it means that you actually have to use one of those types for your values. And once you are using one such value, SQLite will not automatically convert it into another type, because it does not know that this value is a date/time value.
Does a columnType of NUMERIC/DateTime result in casting delays due to comparisons of TEXT, REAL, or Integer?
You get casting delays only when you have values of one type stored in the column, and are comparing against another type.
(And a string like '1970-01-01' is never equal with a number like 0, so such comparisons would not make sense.)
Could a TEXT column type and a TEXT query type for SQLite be faster?
The column type does not really matter.
If you store TEXT values in the column, you must query with TEXT values.
If you store REAL values in the column, you must query with REAL values.
If you store INTEGER values in the column, you must query with INTEGER values.
How would I convert a DATETIME into a NUMERIC for faster queries?
There is no DATETIME type.
I am currently working on an Android app where users can order their list via drag and drop in any order they want. Therefore, I must store the sort order in a variable and a column. I was thinking of giving each row a number like 100000, 200000, 300000, etc and if a user moves an item between 100000 and 200000, then its sort number becomes the average of its neighbours ie 150000. So the farther apart the numbers, the fewer times I have to "reset" the sort numbers when they converge onto each other
There are a few things I am worried about:
First is what is the most efficient? Do large numbers use more resources or take longer to sort? I only expect ~40 rows so if large numbers take longer to sort, I might be better off using smaller numbers and "resetting" more often.
Second is ensuring cross-platform compatibility in the future. For now, I only have to worry about this working with my Android app which uses Java and SQLite which haves longs with max of 2 ^ 63-1. But in the future I may have to worry about things like syncing between an iOS app which uses Objective C and maybe a web client. I am not familiar with those technologies so it would be helpful if anyone could point out any cross-compatibility problems I might have with them and how I can prevent them now so I won't have to modify my stuff later.
Thanks
Edit: Sorry I forgot to mention this but another reason I don't want to just update all the rows every time with an incrementing sort order is because not only do I have to save to the local db but I have to sync any updated changes to a BAAS. A lot of BAAS like Parse and Kinvey (what im using) do not allow you to batch save objects so every time I "reset" all the sort orders I have to make a request to my BAAS for every row instead of just one.
You can just stick with the SortOrder being separated by one. I think you are worried about having to perform an update on tons of rows, but if you Index on SortOrder this won't be a problem. They will all get updated extremely fast.
I think you'll be surprised how fast a query like
UPDATE TableName SET SortOrder = SortOrder + 1 WHERE SortOrder > {InsertLocation + 1};
will run with an Index on your SortOrder column. If you have a lot of data and don't have the Index, you'll notice the query will take significantly longer. Try it out for yourself!
Regarding cross platform SQLite support, both iOS and the Web support SQLite, so you shouldn't have a problem there.
I have a date/time that I get from a sqlite database and want to show in my Android app. I want it in the format :
9 November, 12:12. How do I do this? Is it best to format it before or after I enter it into the database?
See the answers to this question.
In your database, I would suggest you keep timestamps in there rather than the formatted strings. That way your program would still work if you decide to change the format later.
Can anyone please tell me the LIST OF DATATYPES in SQLITE supporting ANDROID.
I want to confirm about TIME and DATE datatypes.
Here is a list of SQLite's data types
TIME and DATE are supported indirectly.
As said at Datatypes In SQLite Version 3:
Each value stored in an SQLite database (or manipulated by the database engine) has one of the following storage classes:
NULL. The value is a NULL value.
INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value.
REAL. The value is a floating point value, stored as an 8-byte IEEE floating point number.
TEXT. The value is a text string, stored using the database encoding (UTF-8, UTF-16BE or UTF-16LE).
BLOB. The value is a blob of data, stored exactly as it was input.
And for TIME and DATE storage the following is stated:
Date and Time Datatype
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
First of all check the Sqlite3 version for each android API version: https://stackoverflow.com/a/4377116/737636
I don't think that this is the full list of versions and I think that also OEM can change the SQLite's version installed inside.
Anyway, if you take a quick look you can see that the base version is 3.x
This is the list of supported datatypes by SQLite3 (the same used inside android): http://www.sqlite.org/datatype3.html
and here you are :)