I have a SQLite table (with more than 100000 rows) that I want its rows streamed.
I do:
Cursor cursor = db.rawQuery("SELECT _id FROM hugetable", new String[] {});
The problem is that just getting the first row as:
cursor.moveToFirst()
seems to load the whole table, being quite slow.
Is there a way to stream a query without having wait to load all the rows when fetching the first row?
(like I can do with the plain C SQLite API outside Android)
The Android Cursor is designed to load all data for caching.
To work arouns this, you have to manually query the data in steps:
SELECT _id
FROM hugetable
ORDER BY _id
LIMIT 100;
SELECT _id
FROM hugetable
WHERE _id > last_id_from_previous_query
ORDER BY _id
LIMIT 100;
Related
I need to retrieve last 25 rows from a table using ContentProvider. I have searched and came up with a solution which works fine on simple SQL Database.
SELECT * FROM( SELECT * FROM [Customers] ORDER BY CustomerID DESC limit 25)ORDER BY CustomerID ASC;
But as i said , m using ContentProvider, and in my onCreateLoader the procedure is like this :
CursorLoader(getActivity(), uri, projection, selection, null, sortOrder);
in which the bold portion of the above query i.e. FROM can only be a Uri.
And if i use
( SELECT * FROM [Customers] ORDER BY CustomerID DESC limit 25) as a Uri then Exception says that is not a valid Uri.
Is there any other way to achieve my objective or any modification in this query. Thanx.
Tell the content provider to return rows sorted by CustomerID DESC.
Once you have that cursor, just take the first 25 rows out of it, and sort them again, manually. (In this case, just reverse the order of the rows.)
If you still need to have the data in a cursor afterwards, store the values in your own MatrixCursor.
As I'm fetching only one column from SqLite database but getting more than 1MB of data in my cursor object and I can't split my database query. Is it possible to break cursor processing as soon as cursor fetches first row and at that particular time I want to hold this cursor object values in another object. After that cursor clear this value and move to next for fetching second row in database, this continues till the end of records ?
What if you do the following? (This is just an idea)
Fetch all rows you need with the id column only (fetch the id instead of the blob column).
Iterate throw that cursor and for each line fetch only one row for the given id with your blob. Then you close that Cursor and you open a new one for the next id row:
//just fetch the ids of the wanted rows
Cursor idCursor = db.query("TABLE_NAME",new String[]{"_id"}, null, null, null,null,null);
Cursor blobCursor;
//for each row (id)
while(idCursor.moveToNext())
{
//fetch one row with the blob of the given id
blobCursor = db.query("TABLE_NAME",new String[]{"image"}, "_id = ?", new String[] {new Long(idCursor.getLong(0)).toString()}, null,null,null);
if(blobCursor.moveToFirst())
{
//get the blob and store it
blobCursor.getBlob(0);
}
blobCursor.close(); //close the cursor (and release resources)
}
idCursor.close();
If you are using Cursor(SQLiteCursor) - there is no way to prevent cursor from "eating memory"(break processing as you says) after you fetched first row.
android.database.sqlite is a java wrapper for sqlite3 library which is written on C.
The fact is that sqlite3 has no function to count how much records statement will produce, so you have to scan whole resultset with help of sqlite3_step function until it returns SQLITE3_DONE. SQLiteCursor is derived from CursorWindow.
CursorWindow (has some native methods) at the moment Cursors getCount() method is called first time - it does two things : calculates count of row and caches these rows.
There is custom port(russian) of sqlite3 for android with functionality you need.
If you can not read russian:
java code
native code
native sources
I'm working to develop an application that has to query at some time, a database with over 4k rows, and each row has 90 fields (Strings). The problem is that if I select * from database, my cursor gets really big (over 4MB). And the cursor in android is limited to 1MB.
How can I solve this, or what's the most elegant method to workaround this?
It is possible to split database in smaller chunks and query them out?
I found a way to handle this and I want to share with all who need it.
int limit = 0;
while (limit + 100 < numberOfRows) {
//Compose the statement
String statement = "SELECT * FROM Table ORDER someField LIMIT '"+ limit+"', 100";
//Execute the query
Cursor cursor = myDataBase.rawQuery(statement, null);
while (cursor.moveToNext()) {
Product product = new Product();
product.setAllValuesFromCursor(cursor);
productsArrayList.add(product);
}
cursor.close();
limit += 100;
}
//Compose the statement
String statement = "SELECT * FROM Table ORDER someField LIMIT '"+ (numberOfRows - limit)+"', 100";
//Execute the query
Cursor cursor = myDataBase.rawQuery(statement, null);
while (cursor.moveToNext()) {
Product product = new Product();
product.setAllValuesFromCursor(cursor);
productsArrayList.add(product);
}
cursor.close();
The main idea is to split your data, so you can use the cursor as it should be used. It's working under 2 s for 5k rows if you have indexed table.
Thanks,
Arkde
Well as a rule you never do select *. For a start each row will have a unique identifier, and your user will want to select only certain rows and columns - ie what they can see on an android screen. Without appearing to be rude this is a pretty basic question. You only return the columns and rows you want to display for that screen on the phone - otherwise you consume unnecssary battery life transfering never to be diaplayed data. the standard approach is to used parameterised stored procedures. Google parameterised stored procedures and do a little reading - by the by - you cant update any table unlees you return the unique row identifier for that table.
Do you need all these rows at the same time? Can you fetch them in parts? This question has been asked several times: Android SQLite and huge data sets
Here's one more suggestion: If you have 90 fields that you need to modify, split them into 10 different views. On each view have a left arrow and right arrow so you can horizontally traverse across screens. Hence each view will show 9 fields. Or some strategy like that. Essentially these are all the same views except for column names so you shouldn't have to modify much code.
I have a listview populated from an SQLite database. I have several items that I successfully populate into the listview, however I'm having trouble with one last thing.
I'm trying to queue the sum total of the column KEY_CONTENT6 which is a string type, however it only contains numbers. I'd like to keep it as a string, so to add it up I'm using Double.valueOf(). The problem is this code force closes on queue and I cant figure out whats wrong:
public Cursor queueAll(){
String[] columns =
new String[]{KEY_ID, "sum("+ Double.valueOf(KEY_CONTENT6) +")",
KEY_CONTENT9, KEY_CONTENT10 };
Cursor cursor = sqLiteDatabase.query(MYDATABASE_TABLE, columns,
null , null, KEY_CONTENT10, null, KEY_CONTENT9+ " DESC");
return cursor;
}
simply use SUM, no need to use anything else..
String[] columns =
new String[]{KEY_ID, "sum(KEY_CONTENT6)",
KEY_CONTENT9, KEY_CONTENT10 };
It is valid for SQLite. Because, no matter what you set data type in SQLite, it stores values as string. So, type conversion is somewhat built-in in SQLite.
You can't use java in a SQL statement, either stick to strait sql or iterate over the cursor and use java to do your calculation.
You can find everything there is to know about sqlite here http://www.sqlite.org/docs.html
SQLite is basically typeless, so you might be able to use SUM on your column even though it is a string. However, if it's meant to be a numeric column, why not give it a number type??
I have created a database and all works fine. But How I can get out last five rows, which one's column value is for example 1.
The select and insert function have synchronized function, so the reading and inserting doesn't happen same time. There has more than 300 hundreds rows, but I only need get cursor object last 5 rows (so I get all columns in one row) which one's column value is 1.
Thanks for any helps!
SELECT * -- list of the columns you want
FROM table
WHERE column1 = 1 -- rows with column1 = 1
ORDER BY (ordering columns) -- columns by which you want to order
LIMIT 5 -- and get the last 5
I think you want to select the first five rows which have all have a certain column with a value of one. If so, using a WHERE and LIMIT statement should help:
SELECT * FROM tbl WHERE the_column = 1 LIMIT 5
You can Use the method database.query();
as follows:
Cursor cursor = database.query(false, TABLE,new String[] {COLUMNS_TO_SELECT},"YOUR_WHERE_CLAUSE", null, null, null, null /*ORDER BY*/, "5"/*YOUR_LIMIT*/);
Have a look at Documentation