In my Android App I need to keep track of the longest streak, and current streak, of consecutive dates that are saved in a database table. I don't even know where to start to get this to work. The best I can come up with is to query every row in the table and iterate through all of them programmatically to find where there's a gap. Not very efficient. Anyone have any better ideas?
Here is an SQL only solution that I thought was really cool. Assuming the dates in your table are unique (not that it would be too hard to just group on them) you can use the method adapted from here http://www.sqlteam.com/article/detecting-runs-or-streaks-in-your-data. I ran through the example and there are some syntax errors, so hopefully I didn't repeat them in my answer below. I probably used some reserved keywords, so you may need to adapt that.
First create a table of Dates that is significantly large to cover your needs. I'm not sure what the best method for SQLite is, but in SQL Server you can insert integers into a Datetime field and it will do an implicit conversion of integers to dates. There are many methods to insert integers into tables...
Anyway, once the Dates table is created, do a left join from your Dates table to your Streak table using your min and max dates from your streak table as your range limiter. You will then have the following code.
Let’s call it SQL 0
SELECT Dates.Date,
CASE
WHEN StreakTable.DATE IS NULL THEN 0
ELSE 1
END AS Result
FROM Dates
LEFT JOIN StreakTable
ON Dates.DATE = StreakTable.DATE
WHERE Dates.DATE BETWEEN (SELECT MIN(DATE) AS MinDate
FROM StreakTable) AND (SELECT MAX(DATE) AS MaxDate
FROM StreakTable)
Let’s call the following SQL 1
SELECT Date,
Result,
(SELECT COUNT(*)
FROM (SQL 0) S
WHERE S.Result <> SV.Result
AND S.GameDate <= SV.GameDate) AS RunGroup
FROM (SQL 0) SV
Let’s call the following SQL 2
SELECT Result,
MIN(Date) AS StartDate,
MAX(Date) AS EndDate,
COUNT(*) AS Days
FROM (SQL 1) A
GROUP BY Result,
RunGroup
ORDER BY MIN(Date)
At this point you can do some pretty cool stuff like answer:
What was the longest streak?
SELECT TOP 1 *
FROM (SQL 2) A
WHERE Result = 1
ORDER BY Games DESC
What is the current streak as of the most recent date?
SELECT *
FROM (SQL2) A
WHERE EndDate = (SELECT Max(Date)
FROM Streak)
How many streaks of 3 or more did we have?
SELECT Result,
COUNT(*) as NumberOfStreaks
FROM (SQL 2) A
GROUP BY Result
WHERE Days >= 3
Basically you have a month and days in a month
so you just compare the days count to the needed number.
If there's a gap you can easily find it out by substracting the count from days in a month. E.g. you have count(days_visited) where month=1
and it returns you 20 days but January has 31 so there's a gap in 11 days and here're the date functions of sqlite
http://www.sqlite.org/lang_datefunc.html
You can use following functions
like SELECT date('now','start of year','+9 months','weekday 2');
EDIT
sorry everyone solution is ugly. it is I know.
create table visits(day_visited timestamp,visited int);
You create a record everyday in this table indicating
whether a user was online or offline with
'now',1 or 0 (online/offline). Then you run through there records.
Your records for month will be an int array with 1s and 0s.
called vistedrecordsformonth
pseudo code:
int online=0;
int offline=0;
for(int i=0;i<vistedrecordsformonth.size();i++){
boolean flag=false;
if(vistedrecordsformonth[i]==1){ //visited
if(!flag&&i!=0) //check for flag and not 0 index to insert a record
{
streaksMap.put(online,offline); //or insert a record into another streakmap or table
online=0;
offline=0;
}
flag=true;
online++;
}
else{
flag=false;
offline++;
}
} //end of for
The map or table will contain a pair of online=offline days for a month.
with usual order by you can see what was the biggest streak in online or offline days.
It is ugly I know I'm sure there should be something more elegant but as
quick and dirty it should work.
hope it helps abit.
Related
I need get the total SUM for each rows in my query, but I don't want go twice in the table.
I tried do this:
SELECT id, value, SUM(value) as total FROM product
But my result was this:
id value total
3 30 60
If I do the bellow query I get my wanted result, but I need go twice in the table:
SELECT id, value, (SELECT SUM(value) FROM product) as total FROM product
Or if I use 'WITH' clause, but this is not supported before Android 5:
WITH data AS (SELECT id, value FROM product)
SELECT id, value, (SELECT SUM(value) FROM data) as total FROM data
Wanted result:
id value total
1 10 60
2 20 60
3 30 60
Thank you!
It's not possible using your SQLite version. You'll have to use two selects.
Basically you have to use a subquery.
However, perhaps you may be less concerned about the 2nd table as I believe that the Query Planner will determine that it only needs to calculate the sum once and does away with the need for a variable as it stores the value in cache.
I believe that the results of using EXPLAIN QUERY PLAN your_query shows this. i.e. using
EXPLAIN QUERY PLAN SELECT id, value, (SELECT sum(value) FROM products) AS total FROM products;
results in :-
This being explained as (see bolded statements) :-
1.3. Subqueries
In all the examples above, the first column (column "selectid") is
always set to 0. If a query contains sub-selects, either as part of
the FROM clause or as part of SQL expressions, then the output of
EXPLAIN QUERY PLAN also includes a report for each sub-select. Each
sub-select is assigned a distinct, non-zero "selectid" value. The
top-level SELECT statement is always assigned the selectid value 0.
For example:
sqlite> EXPLAIN QUERY PLAN SELECT (SELECT b FROM t1 WHERE a=0), (SELECT a FROM t1 WHERE b=t2.c) FROM t2;
0|0|0|SCAN TABLE t2
0|0|0|EXECUTE SCALAR SUBQUERY 1
1|0|0|SEARCH TABLE t1 USING COVERING INDEX i2 (a=?)
0|0|0|EXECUTE CORRELATED SCALAR SUBQUERY 2
2|0|0|SEARCH TABLE t1 USING INDEX i3 (b=?)
The example above contains a pair of scalar subqueries assigned
selectid values 1 and 2. As well as a SCAN record, there are also 2
"EXECUTE" records associated with the top level subquery (selectid 0),
indicating that subqueries 1 and 2 are executed by the top level query
in a scalar context. The CORRELATED qualifier present in the EXECUTE
record associated with scalar subquery 2 indicates that the query must
be run separately for each row visited by the top level query. Its
absence in the record associated with subquery 1 means that the
subquery is only run once and the result cached. In other words,
subquery 2 may be more performance critical, as it may be run many
times whereas subquery 1 is only ever run once.
Unless the flattening optimization is applied, if a subquery appears
in the FROM clause of a SELECT statement, SQLite executes the subquery
and stores the results in a temporary table. It then uses the contents
of the temporary table in place of the subquery to execute the parent
query. This is shown in the output of EXPLAIN QUERY PLAN by
substituting a "SCAN SUBQUERY" record for the "SCAN TABLE" record that
normally appears for each element in the FROM clause. For example:
sqlite> EXPLAIN QUERY PLAN SELECT count(*) FROM (SELECT max(b) AS x FROM t1 GROUP BY a) GROUP BY x;
1|0|0|SCAN TABLE t1 USING COVERING INDEX i2
0|0|0|SCAN SUBQUERY 1
0|0|0|USE TEMP B-TREE FOR GROUP BY
If the flattening optimization is used on a subquery in the FROM
clause of a SELECT statement, then the output of EXPLAIN QUERY PLAN
reflects this. For example, in the following there is no "SCAN
SUBQUERY" record even though there is a subquery in the FROM clause of
the top level SELECT. Instead, since the flattening optimization does
apply in this case, the EXPLAIN QUERY PLAN report shows that the top
level query is implemented using a nested loop join of tables t1 and
t2.
sqlite> EXPLAIN QUERY PLAN SELECT * FROM (SELECT * FROM t2 WHERE c=1), t1;
0|0|0|SEARCH TABLE t2 USING INDEX i4 (c=?)
0|1|1|SCAN TABLE t1
EXPLAIN QUERY PLAN
End Note
Perhaps of relevance is this statement :-
The best feature of SQL (in all its implementations, not just SQLite)
is that it is a declarative language, not a procedural language. When
programming in SQL you tell the system what you want to compute, not
how to compute it. The task of figuring out the how is delegated to
the query planner subsystem within the SQL database engine.
Query Planning
You may also find this of interest he SQLite Query Optimizer Overview noting that as of release 3.8.0 The Next-Generation Query Planner is utilised.
I want to get the latest 5 records in my table, so far i tried this but, it did not work out very well. So, what is the cleanest and efficient way to get last 5 records in the table ?
"select * from (select * from People order by Date DESC limit 5) order by Date ASC;"
Your query works just fine.
To make it efficient, ensure that there is an index on the Date column; then SQLite will just read the last five entries from the index and the table and does not need to scan the entire table.
If this table has an autoincrementing ID column, and if "latest" means the insertion order, then you can use that ID for sorting; this will be as efficient as your original query with an index on Date:
SELECT * FROM (SELECT * FROM People
ORDER BY _id DESC
LIMIT 5)
ORDER BY Date ASC
I have a problem in displaying the data in my application for each week?
I can show the data by day using this code in my SQLite
SELECT substr(_id, 1, 7) as _id, sum(value) as total FROM pd_table GROUP BY _id order by _id desc
I was also able to display the data based on the month by using this code
SELECT substr(_id, 1, 10) as _id, sum(value) as total FROM pd_table GROUP BY _id order by _id desc
_id using SimpleDateFormat ("yyyy-MM-dd HH: mm: ss")
now i want to display it every week. How?
Any answer is very useful for me :)
SQLite has internal date/time formatting function strftime, and one of possible format option is a week number. So your query should look like this:
SELECT strftime('%Y-%W',_id) as week_of_year, sum(value) as total
FROM pd_table
GROUP BY week_of_year order by _id desc
Though I didn't try it on Android. See more here: http://sqlite.org/lang_datefunc.html
I have been stuck with this problem for a while (Android SQLITE). I have 2 columns - Date and Amount. Date is stored in YYYYMMDD format.
DATE AMOUNT
20120521-------50
20120506-------40
20120311-------30
20120202-------20
20120125-------10
What I need is a SQL query (Android SQLITE), which will output two columns - Month and cumulative total till that month..If a month does not have any transaction, it still should evaluate the cummulative total.
So the output I need here (notice there are no transactions for April)
Month Cumulative-Total
MAY-------150
APR-------60
MAR-------60
FEB-------30
JAN-------10
It should work:
SELECT strftime('%m', date), SUM(Amount)
FROM myTable
GROUP BY strftime('%m', date)
Sorry i forgot this part of your question "..If a month does not have any transaction, it still should evaluate the cummulative total."
Solution: The simple solution is to have a DUMMY entry in the table for all months.
Difficult since the easy way to do this is to write a stored procedure to do it but which isnt supported in sqlite. However take a look at this - maybe this way might be useful
Android - easy/efficient way to maintain a "cumulative sum" for a SQLite column
Also you can try doing the cumulative addition on the android end
you can not just comulate like this. You can do this 2 ways:
Group by month and then sum it up on the Android end and add the Months that missing.
Try make that with a store procedure. (this I am not entierly sure possable on sql lite for your case.) Read this
So I would go with the first option.
I have seen number of post about storing date.
I am still not getting the fine and exact approach about saving it to a sqlite database.
I am able to store it, but during sorting I need to consider only month and day just like birthday where years doesn't matter.
What will be the query if I want to get the row whose date is 2 or 3 days in advance, like 2nd march row if searched on 28 Feb?
You should start by checking out the SQLite documentation of date & time functions.
For instance, to solve your problem "And what will be the query if i want to get the row whose date is 2 or 3 days in advance" you'd use julian day calculations, such as this example that you can execute directly in the sqlite3 shell:
create TABLE example (_id INTEGER PRIMARY KEY NOT NULL, date TEXT NOT NULL);
insert into example (date) values ('2011-01-02');
insert into example (date) values ('2011-04-02');
insert into example (date) values ('2012-02-26');
insert into example (date) values ('2012-02-27');
insert into example (date) values ('2012-02-28');
insert into example (date) values ('2012-02-29');
insert into example (date) values ('2012-03-01');
insert into example (date) values ('2012-03-02');
insert into example (date) values ('2012-03-03');
select date from example where julianday(date) - julianday('now') < 3 AND julianday(date) - julianday('now') > 0;
This would return (given that "today" is feb 28th) all the days that are one, two or three days in the future:
2012-02-29
2012-03-01
2012-03-02
Edit: To only return rows, regardless of year, you could do something like this - using a VIEW (again, exampl is directly in SQLite):
create view v_example as select _id, date,
strftime("%Y", 'now')||'-'||strftime("%m-%d", date) as v_date from example;
This VIEW would return the date & times in your database "rebased" on the current year - which, of course could introduce all manner of wonky behavior with leap years.
You can select all the dates like this in that case:
select date from v_example where
julianday(v_date) - julianday('now') < 3 AND
julianday(v_date) - julianday('now') > 0 ORDER BY v_date;
Which would return:
2012-02-29
2012-03-01
2001-03-01
2012-03-02
2010-03-02
If you want to sort by day and month consider storing the date as string in the format ddMMyyyy (you need two digits for day and month, otherwise the sorting will be flawed). Sorting by increasing values will give you dates sorted by day and month (and then year).
You can even do range query with string but you have to compute the query string.
Alternatively you may store the date as milliseconds in an additional column (this is the usual format for dates in the database) and do the range queries more easily with integer arithmetic.
One option is to use strftime() in SQLite to strip of the year and then do a comparison.
select * from sometable where strftime('%m-%d', somecolumn) = '02-28'
This will do a query of all rows for February 28th. But performance might be hurt if you have a large table and need to do a string conversion of every row for comparison. Maybe store the day and month in two additional columns if this query is performed often?