Develop Reference

How to get daily sum in SQLite

I have a simple SQLite database with two columns.
Col 1 SimpleDateFormat("yyyyDDD") For example 2017001 (for jan 1st, 2017)
Col 2 int of hourly occurances
So each day has a unique code and each day has 24 rows of varying occurances.
How can I get the sum of occurrences for each day sent to an ArrayList(Float)? Eventually this ArrayList will populate an MPAndroidChart. I've successfully pulled the data (without the sum) with the following code. But getting the daily sum with help of my date code has eluded me for days. I've tried many variations of GROUP BY and SUM only to have each crash.
public ArrayList<Float> getOccChartData(){
SQLiteDatabase db = this.getWritableDatabase();
ArrayList<Float> yNewData = new ArrayList<>();
String query = "SELECT OCC FROM user_table ORDER BY ID DESC LIMIT 96";
Cursor c = db.rawQuery(query, null);
for (c.moveToFirst(); !c.isAfterLast(); c.moveToNext()) {
yNewData.add(c.getFloat(c.getColumnIndex("OCC")));
}
c.close();
return yNewData;
}

It's unclear whether or not you want the sums instead of or with the rows, so assuming that you just want the sums, then perhaps you could do:-
public ArrayList<Float> getOccChartData(){
SQLiteDatabase db = this.getWritableDatabase();
ArrayList<Float> yNewData = new ArrayList<>();
String query = "SELECT SUM(OCC) AS OCC FROM user_table GROUP BY ID ORDER BY ID DESC"
Cursor c = db.rawQuery(query, null);
while(c.moveToNext) {
yNewData.add(c.getFloat(c.getColumnIndex("OCC")));
}
c.close();
return yNewData;
}
Example
For example assuming your table has (not 24 rows per table for brevity) :-
The the above would result in the cursor containing the following :-
Note! reversal due to ORDER BY DESC (so latest data first).
i.e. for 2017001, 15 + 20 + 30 + 40 = 105 the last row.
yNewData would have 3 elements accordingly:-
yNewData[0] would be 48,
yNewData[1] would be 50 and
yNewData[2] would be 105.
Alternative considerations
If you wanted a range of days then you change the SELECT to be something like :-
SELECT SUM(OCC) FROM user_table WHERE ID BETWEEN 2017001 AND 2017002 GROUP BY ID ORDER BY ID DESC
Where the values 2017001 and 2017002 would be generated according to the required logic and placed into the query string.
In which case the resultant returned data would be :-
yNewData[0] would be 50 and
yNewData[1] would be 105.
If you want all intermediate values as well as the sums then matters would be more complex as in effect you are returning two types of data in a single dimension array.
The above as an SQL Fiddle
Here's the above as a SQL Fiddle

FTS4 sqlite MATCH not working

I've tried several methods from here:
SQLite FTS example doesn't work
and here:
Full text search example in Android (best tutorial so far i think)
However, my search returns 0 results!
Here is what I've tried:
String key = "a";
Cursor c = db.query(true, "texts_virtual",
new String[]{"id","title_normalized"},
"title_normalized MATCH '"+key+"'",
null, null, null, null, null);
= 0 Results;
String query = "a";
String[] params = {"%" +query+ "%"};
Cursor c = db.rawQuery("SELECT * FROM texts_virtual WHERE title_normalized MATCH ?", params);
= 0 Results too
I know that the virtual table is correctly working because I can do this:
String queryText = "a"; //here i test other texts and they worked too
String query = "select * from texts_virtual where title_normalized like ? order by number";
String[] params = {"%" + queryText + "%"};
Cursor c = db.rawQuery(query, params);
so this prove that the texts_virtual is working, what is not working are the queries, but I don't know why, not error, nothing, just 0 results.
Also after I make it work, I'm planning to use multiple terms search in 2 columns
user type "WordA WordB WordC"
it search for each word in the 2columns and return the results, but this if for a future task....
Edit
Table Code Creation:
CREATE TABLE texts (id INTEGER PRIMARY KEY AUTOINCREMENT, title_normalized....);
INSERT INTO texts (id, titulo_normalized...) VALUES (1, 'aaaaaa', ...);
and go on for more inserts, and at the end the virtual creation
CREATE VIRTUAL TABLE texts_virtual USING fts4(content="texts", id, title_normalized, ..other fields);
i can query texts_virtual using LIKE but not MATCH, match return 0 results =/
Edit 2 how the table looks:
Table: texts_virtual
----------------------------
id --- title_normalized
--------------------------
1 --- aaaaaaaaab
2 --- abbbbbbbbb
3 --- bbbbbabbbb
4 --- bbbbbbbbbb

The FTS module searches for words (where the exact definition depends on the tokenizer used), or at best for words with a prefix.
MATCH words as designed; it does not find "a" because there is no word "a" in your data.
If you want to find substrings inside words, you must use LIKE.

You are using % as a joker. In FTS requests, You have to use * instead.
LIKE "%word%"
MATCH "*word*"
I've noticed that for very short words (less than 3 letters), LIKE is faster than MATCH. For longer words, MATCH is faster.

Anyway to change row number in SQLITE rawquery?

I've got the following cursor set up to fill a dialog box with a users payment history
Cursor PaymentsCursor = db.getReadableDatabase().rawQuery(
"SELECT _id, Date, Payment FROM tblPaymentHistory WHERE DebtName = '"
+ debtname + "'" + "ORDER BY _id ASC", null);
SimpleCursorAdapter HistoryAdapter = new SimpleCursorAdapter(this,
R.layout.paymenthistoryrow, PaymentsCursor, from, to);
The problem is though that if there is more than one type of debt, and payments are made to each debt out of order, when the payment history returns it's results, it returns as out-of-order row numbers, for example 1,2,6,7,9,12,etc. I know it's pulling the _id (unique key) from the database, but is there a way to re-base or change the row number in the query, so that each result returns as "1,2,3,4,5,etc" regardless of original ID?
I thought that the ORDER BY _id or even ORDER BY Date ASC would fix this but it didn't.
My rows in the database look something like this:
1, TEST, 4/13/2012, 250
2, TEST, 4/13/2012, 300
3, TEST, 4/14/2012, 222
4, TEST2, 4/14/2012, 500
5, TEST, 4/15/2012, 600
When the user clicks history for "TEST", it returns back as 1,2,3,5... and if they pull up history for "TEST2", it shows as "4", I'm trying to get it so TEST shows "1,2,3,4" and TEST2 shows "1"
Damn I can't answer my own answer, but here's what I ended up doing:
Thanks guys. I found an alternate option that modified the view, so as not having to touch the SqLite db. heres the link that i foundModifying SimpleCursorAdapter's data
And here is the result:
PaymentsCursor = db.getReadableDatabase().rawQuery(
" SELECT _id, Date, Payment FROM tblPaymentHistory WHERE DebtName = '"
+ debtname + "'" + "ORDER BY _id ASC", null);
String[] from = new String[] { DbAdapter.KEY_HISTORY_ID,
DbAdapter.HISTORY_DATE, DbAdapter.HISTORY_PAYMENT };
int[] to = new int[] { R.id.PAYMENTNO, R.id.PAYMENTDATE,
R.id.PAYMENTAMOUNT };
SimpleCursorAdapter HistoryAdapter = new SimpleCursorAdapter(this,
R.layout.paymenthistoryrow, PaymentsCursor, from, to);
HistoryAdapter.setViewBinder(new SimpleCursorAdapter.ViewBinder() {
#Override
public boolean setViewValue(View view, Cursor cursor, int column) {
if (column == 0) { // let's suppose that the column 0 is the
// date
TextView tv = (TextView) view;
String rownum = String.valueOf(cursor.getPosition() + 1);
// here you use SimpleDateFormat to bla blah blah
tv.setText(rownum);
return true;
}
return false;
}
});
paymenthistory.setAdapter(HistoryAdapter);
It may not be the most glamourous way, but now each time the window comes up with the history, it's using the row number (plus one) to indicate which # it is.
Thanks all!

Here is one way to get the "re-based" ids. In this example, the "new ids" are based on the grade (i.e. the "old ids" in your case):
.headers on
create table foo (name text, grade int);
insert into foo values ('Joe', 45);
insert into foo values ('Anna', 98);
insert into foo values ('Julie', 78);
select name,
grade,
(select count(*) from foo t1 where t1.grade>=t2.grade) as rank
from foo t2;
select name,
grade,
(select count(*) from foo t1 where t1.grade>=t2.grade) as rank
from foo t2
order by rank;
Having saved this as foo.sql, I get this:
[someone#somewhere tmp]$ sqlite3 < foo.sql
name|grade|rank
Joe|45|3
Anna|98|1
Julie|78|2
name|grade|rank
Anna|98|1
Julie|78|2
Joe|45|3

I've played a bit with what #sixfeetsix answered and since that fails to give 1,2,3,4,.. numbering in combination with the WHERE you might need to put in more subqueries (maybe not but, I'm not that good with queries):
SELECT (
SELECT count( * ) + 1
FROM (
SELECT *
FROM tblPaymentHistory
WHERE DebtName = ?
)
AS t1
WHERE t1._id < t2._id
)
AS _id,
Date,
Payment
FROM tblPaymentHistory AS t2
WHERE DebtName = ?
ORDER BY _id;
put in java String and leave the ? in there to get escaped values (injection safe):
Cursor PaymentsCursor = db.getReadableDatabase().rawQuery(
"...WHERE DebtName=? .. WHERE DebtName=? .. ", new String[]{ debtname, debtname });
SimpleCursorAdapter HistoryAdapter = new SimpleCursorAdapter(this,
R.layout.paymenthistoryrow, PaymentsCursor, from, to);

This query worked for me after long research...
Empirical results I derived :
1)You need to define where condition in sub-query also.
2)if ids (t2._id <= t1._id) compared in relation operator will be primary keys then it
will work fine in all cases.
3)Regarding orderby condition you have to decide that according to your choice or need.
SELECT
(SELECT COUNT(*) FROM table t2 WHERE t2._id <= t1._id AND t2.Recipe_id = 2) AS RowNumber,_id,Recipe_id,col2,col3,col4
FROM table t1
WHERE Recipe_id = 2
ORDER BY _id
How it works:-
Say we have a sequence of primary keys 1,2,3,4,5,6 in some table t
Now we create two aliases of it using table t1 and table t2
Now both have same sequence table t1 -> 11,12,13,14,15,16
table t2 -> 11,12,13,14,15,16
now this condition ( WHERE t2._id <= t1._id ) compares first primary key "11" of t2 with
the first primary key "11" of t2 as 11=11 it will return count() that only one row exists, hence we get "1" in row number..
*** remember for every row in Outer query the sub-query is executed ***
Hence now outer query is at row second having primary key "12"
now it will again compare the ( WHERE t2._id <= t1._id ) this time again t2._id contains "11" while t1._id contains "12"..
Quiet clear it will return that TWO rows are there which are having ids <= 12 that is 11 and 12
this way it will generate the desired sequence.........
This is a simple trick to generate the sequence.. not simple actually in one look but really simple when you get into depth of it..
I am not expert but this is what i understood..
Hope the explanation helps...
As there are various solutions or same solutions available on net but no explanation..
:)

Delete first N rows in android sqlite database

Please let me know how to delete n-rows in android sqlite database. I used this code:
String ALTER_TBL ="delete from " + MYDATABASE_TABLE +
"where"+KEY_ID+"in (select top 3"+ KEY_ID +"from"+ MYDATABASE_TABLE+"order by _id );";
sqLiteDatabase.execSQL(ALTER_TBL);
But it shows an error.
03-21 13:19:39.217: INFO/Database(1616): sqlite returned: error code = 1, msg = near "in": syntax error
03-21 13:19:39.226: ERROR/Database(1616): Failure 1 (near "in": syntax error) on 0x23fed8 when preparing 'delete from detail1where_id in (select top 3_idfromdetail1order by _id );'.

String ALTER_TBL ="delete from " + MYDATABASE_TABLE +
" where "+KEY_ID+" in (select "+ KEY_ID +" from "+ MYDATABASE_TABLE+" order by _id LIMIT 3);";
there is no "top 3" command in sqlite I know of, you have to add a limit
watch out for spaces when you add strings together : "delete from" + TABLE + "where" = "delete frommytablewhere"
This approach uses two steps to delete the first N rows.
Find the first N rows:
SELECT id_column FROM table_name ORDER BY id_column LIMIT 3
The result is a list of ids that represent the first N (here: 3) rows. The ORDER BY part is important since SQLite does not guarantee any order without that clause. Without ORDER BY the statement could delete 3 random rows.
Delete any row from the table that matches the list of ids:
DELETE FROM table_name WHERE id_column IN ( {Result of step 1} )
If the result from step 1 is empty nothing will happen, if there are less than N rows just these will be deleted.
It is important to note that the id_column has to be unique, otherwise more than the intended rows will be deleted. In case the column that is used for ordering is not unique the whole statement can be changed to DELETE FROM table_name WHERE unique_column IN (SELECT unique_column FROM table_name ORDER BY sort_column LIMIT 3). Hint: SQLite's ROWID is a good candidate for unique_column when deleting on tables (may not work when deleting on views - not sure here).
To delete the last N rows the sort order has to be reversed to descending (DESC):
DELETE FROM table_name WHERE unique_column IN (
SELECT unique_column FROM table_name ORDER BY sort_column DESC LIMIT 3
)
To delete the Nth to Mth row the LIMIT clause can be extended by an OFFSET. Example below would skip the first 2 rows and return / delete the next 3.
SELECT unique_column FROM table_name ORDER BY sort_column LIMIT 3 OFFSET 2
Setting the LIMIT to a negative value (e.g. LIMIT -1 OFFSET 2) would return all rows besides the first 2 resulting in deletion of everything but the first 2 rows - that could also be accomplished by turning the SELECT .. WHERE .. IN () into SELECT .. WHERE .. NOT IN ()
SQLite has an option to enable the ORDER BY x LIMIT n part directly in the DELETE statement without a sub-query. That option is not enabled on Android and can't be activated but this might be of interest to people using SQLite on other systems:
DELETE FROM table_name ORDER BY sort_column LIMIT 3

It seems that you've missed some spaces:
"where"+KEY_ID+"in..
must be:
"where "+KEY_ID+" in...
Furthermore you need to use the limit statement instead of top:

I'll do:
db.delete(MYDATABASE_TABLE, "KEY_ID > "+ value, null);

you can try this code
int id;
public void deleteRow(int id) {
myDataBase.delete(TABLE_NAME, KEY_ID + "=" + id, null);
}
String id;
public void deleteRow(String id) {
myDataBase.delete(TABLE_NAME, KEY_ID + "=\" " + id+"\"", null);
}

It is a bit long procedure but you can do it like this
first get the ids column of table from which which you want to delete certain values
public Cursor KEY_IDS() {
Cursor mCursor = db.rawQuery("SELECT KEYID " +
" FROM MYDATABASE_TABLE ;", null);
if (mCursor != null)
{
mCursor.moveToFirst();
}
return mCursor;
}
Collect it in an array list
ArrayList<String> first = new ArrayList<String>();
cursor1 = db.KEY_IDS();
cursor1.moveToFirst();
startManagingCursor(cursor1);
for (int i = 0; i < cursor1.getCount(); i++) {
reciv1 = cursor1.getString(cursor1
.getColumnIndex(DBManager.Player_Name));
second.add(reciv1);
}
and the fire delete query
for(int i = 0 ;i<second.size(); i++)
{
db.delete(MYDATABASE_TABLE KEYID +"=" + second.get(i) , null);
}

Delete first N (100) rows in sqlite database
Delete from table WHERE id IN
(SELECT id FROM table limit 100)

You can make use of the following mode: (in addition to the response provided by "zapl").
**DELETE FROM {Table-X} WHERE _ID NOT IN
(SELECT _ID FROM {Table-X} ORDER BY _ID DESC/ASC LIMIT (SELECT {Limit-Column} FROM {SpecificationTable}) );**
Where {Table-X} refers to the table you want to delete, _ID is the main unique-column
DESC/ASC - Based on whether you want to delete the top records or the last records, and finally in the "LIMIT" clause, we provide the "n" factor using another query, which calls in the {Limit-Column} from {SpecificationTable}: Which holds the value against which you want to delete them.
Hope this helps out someone.
Happy Coding.

How do I improve the performance of my SELECT query in Sqlite / Android?

I'm getting poor performance and possibly strange behavior with a simple SELECT query in Sqlite & Android. SqliteDatabase.query() executes my query in only 1 ms, but fetching the results with Cursor.get*() takes over 150 ms to return only 8 rows!
I am trying to find all the rows in the table english where the column word starts with "prefix" (an arbitrary string), sort the results by the row column, and return only the first 8 results.
Here is my code:
String columns[] = {"word", "rank"};
Cursor cursor = mDB.query("english", columns, "word LIKE '" + prefix + "%'", null, null, null, "rank,word", "8");
// It takes only 1 ms to get here
String word = "";
int rank = 0;
if (cursor.moveToFirst()){
do {
word = cursor.getString(0);
rank = cursor.getInt(1);
}
while(cursor.moveToNext());
}
cursor.close();
// It takes over 150 ms to get here!
The table definition for english is:
CREATE TABLE en (_id INTEGER PRIMARY KEY, word TEXT, rank INTEGER)
It contains about 65,000 entries. It also also indexes on word and rank, with a third index for both (I was getting desperate!):
CREATE INDEX "rank" ON "en" ("rank" ASC)
CREATE INDEX "word" ON "en" ("word" ASC)
CREATE INDEX "word_rank" ON "en" ("word" ASC, "rank" ASC)
Thanks in advance!

The query method doesn't actually retrieve all the data, the cursor retrieves it as it moves through the rows. So it makes sense that the Cursor.move*() methods are slower then the query.
This 'lazy-loading' concept helps save memory as only the relevant data is retrieved as it's needed.
As for performance, you really aren't doing anything wrong. Are you trying this on the emulator? Perhaps try it on an actual device and test the performance.