i am trying to optimize my application. I noticed that cursor.movetofirst() method somehow slowing the performance of my code.
Cursor cursor = myDbHelper.getDayInfo(new SimpleDateFormat("yyyy-MM-dd").format(myCalendar.getTime());
above line executes in 10 ms in 2.1 emulator, and
if(cursor != null && cursor.moveToFirst())
this line took about 1.6 seconds. I made little search about this. Somepeople say make it in another thread or in asynctask, but this will make the code more complicated.
I 'm just trying to figure out what is actually happening to this cursor.
Can anyone simplify or give a hint about database performance increase related to my question?
It's been a while since this was marked answered but you never really got a great answer here. Even on a very large table a single row query should usually take much less than 1.6 seconds. The reason it's slow is that you don't have an index for the column(s) that you are querying on so it has to scan the entire table to find those values. If you create an index you can cut the time down to a fraction of a second.
It is natural, that moveToFirst() method gets much more time that other code. The data enquariyng actually takes place, which invokes database communication, data reading and so on. You actually can't do anything with it. The advice was right - move all the long operations to AsyncTask.
If the dataset held by the cursor is large, moving the cursor around takes time. You have to perform such data-intensive operations on a separate thread. Using an AsyncTask might make your code a tiny bit complex but it's worth it. This against the user thread being blocked at the cost of a broken UI experience.
Related
Conventional wisdom says to avoid calls to Cursor.getCount() because it's an expensive operation.
After digging through AbstractCursor source code, it seems all standard operations including moveToNext, moveToFirst, and moveToPosition all call getCount internally!
So what's the point of avoiding calling it yourself at all?
EDIT: added hyperlink to Android docs
EDIT: Perhaps I stand corrected?
I was under the impression that a Cursor can provide a moving window over large data sets, whose full length can only be known once the window has perused the entire set at least once. Hence calling getCount would cause a full load/iteration, and moveToNext advances the window only as necessary.
EDIT the assumption of this question is wrong, Android doesn't keep the query open, but it can cause other problems. See discussion bellow...
The Android Cursor implementation keeps a scrolling window of the underlining SQLite cursor, so it can implement "move forwards".
But I found a old SQLite documentation saying this is a bad practice, and you should not leave query open for UI inputs:
https://www.sqlite.org/cvstrac/wiki?p=ScrollingCursor
Also for example Telegram use it's own wrapper of SQLite, and don't have the scrolling cursor. https://github.com/DrKLO/Telegram/blob/master/TMessagesProj/src/main/java/org/telegram/SQLite/SQLiteCursor.java . They only have "next"
Also there is the "best practice" to call "getCount" in CursorLoader, if you look at the implementation, it actually tries to loop through all the data in the underlining query.
So is Android's wrapper implementation bad? Is SQLite doc outdated and it is ok to keep query open?
The Android Cursor implementation keeps a scrolling window of the
underlining SQLite cursor, so it can implement "move forwards".
Both scrollable and non-scrollable cursors implement move forward, i think you meant move backward here.
The documentation you specified, doesn't tell that using scrollable cursor with UI is a bad practice. It says
Get in, grab your data, then get out. Later on, when the user decides
to scroll up or down (which will usually be eons of time later from
the point of view of your CPU), run another query to refresh the
screen with new data.
As far as I know, android cursors work exactly this way. Android's SQLiteCursor (that is actually used with SQLiteDatabase class) is a subclass of the AbstractWindowedCursor. AbstractWindowedCursor uses CursorWindow within it, that is, as per documentation:
A buffer containing multiple cursor rows.
Each time the limit of this buffer is reached, AbstractWindowedCursor makes new CursorWindow and fills it with data.
P.S. In fact, these windows keep fairly big amount of data. It,s not limited by the screen size and far more than 5 or 50 rows (to be precise, it can be limited by 5 or even 1 row, if your rows contain, say, a thousand columns), and we as developers rarely face such behaviour.
I have a ListView which potentially contains thousands of rows, generated by a CursorAdapter. Only eight or so rows are visible at any one time. I've had reports that starting up this view can take many seconds, and can cause an ANR (force close).
I'm doing the DB query in a background thread. I've verified that newView and bindView in my adapter are only being called for the number of visible rows.
Once the list is displayed, actually scrolling up and down the list is very fast.
The delay is in the call to ListView.setListAdapter, which has to run on the UI thread. Why does this seem to depend on the total number of rows in the result set, rather than the (much smaller) number of rows which are actually being displayed? Is there any way I can optimize it?
This question was asked a couple of years ago in this thread. I'm hoping to get some fresh insight and more concrete examples of potential workarounds.
UPDATE
I have tried to work around this by using CommonsWare's EndlessAdapter. I limit the initial query to (say) 20 rows using a LIMIT clause in my DB query, and I increase this limit and resubmit the query every time I hit the bottom of the list.
(As an aside, I haven't found a way of appending just the new results to an existing Cursor, so I'm increasing the LIMIT value each time and then re-fetching the whole lot up to the new limit in a new Cursor.)
Strangely, this workaround doesn't seem to improve the time it takes to perform the initial setListAdapter call. When I run it on a data set containing only 20 rows, the call to setListAdapter is really quick. When I run it on a data set containing hundreds of rows, but limited to return just 20, it takes over a second.
UPDATE 2
By forcing the query to execute in the background thread with a simple getCount(), as suggested by CommonsWare, I've cured the initial blocking of the UI thread on starting the activity. The UI is still blocked, though, when returning to this activity from a child activity. The ListActivity by default seems to want to re-run the query on the UI thread.
I have worked around this by removing the adapter in onStop(), and recreating it in onStart(). Thus the query is always performed in the background regardless of the direction we're moving through the activity stack.
I'm doing the DB query in a background thread
If all you do in the background is call query() or rawQuery(), the query is not actually executed. It will be lazy-executed when you first try using the Cursor (e.g., getCount()). So, the right recipe for doing a query in the background is a rawQuery() followed by something like getCount() in the background thread, to ensure the query really is executed.
When I run it on a data set containing hundreds of rows, but limited to return just 20, it takes over a second.
Off the cuff, that would suggest that the speed issue is not the time required to read in the results, but rather in computing the results in the first place.
I have a simple query returning a Cursor, and then I walk the cursor and create objects that I throw in an ArrayList, like this:
List<Element> myElements = new ArrayList<Element>();
Cursor c = db.query(...);
c.moveToFirst();
while (c != null && !c.isAfterLast()) {
myElements.add(new Element(cursor.getString(0).........)); <-- CREATING THE ELEMENT
c.moveToNext();
}
...
You get the idea.
The problem is that I need to run 4 queries like this hitting different tables, etc, but they all return the same Element object in the end (after walking the cursor).
Being a good Android citizen I created a class extending AsyncTask to avoid hogging the UI Thread. Also, I want to run the 4 queries in 4 threads to speed things up.
The question:
in my onPostExecute(Cursor c), I'm running the logic marked as "CREATING THE ELEMENT" above. If I run 4 threads with 4 queries and all modifying the List, will I have thread conflicts touching the same variable from them? How do I prevent that? Do I gain anything by threading this if the list I need to modify is synchronized? I mean, the threads will have to wait in line anyway, I might as well write the 4 queries and run them sequentially... or not?
I understand I want to keep this out of the UI Thread. The question is if I want to create 4 threads (each running in an AsyncTask) or just ONE AsyncTask that runs the 4 queries sequentially.
Thanks!
Llappall
will I have thread conflicts touching the same variable from them?
You will certainly have race conditions - if you are fine with it then no issues.
How do I prevent that? Do I gain anything by threading this if the list I need to modify is synchronized?
I don't think so.
I mean, the threads will have to wait in line anyway, I might as well write the 4 queries and run them sequentially... or not?
The question is if I want to create 4 threads (each running in an AsyncTask) or just ONE AsyncTask that runs the 4 queries sequentially.
I would run all the 4 queries in one AsyncTask, creating 4 AsyncTasks would be a lot to do and maintain.
Vector, as opposed to ArrayList, is synchronized and thread safe, so I would suggest to use it instead.
http://download.oracle.com/javase/6/docs/api/java/util/Vector.html
Another alternative would be to create a new List per thread and then use Collections.addAll() to incorporate the elements to the original list.
To answer the question whether you would gain anything by starting multiple threads, probably the answer will depend on how expensive are the queries you are doing. Starting a new thread has an intrinsic overhead, so you want to make sure that the query you are starting is worth the cost.
I'm having a hard time figuring out what my problem is here. I'm receiving this error in my program, but it does not cause a crash or anything like that. I have an update I'd like to release, but I don't want to release it with this error being thrown at certain times. I've read all related posts on this error, but none apply to my situation.
I've made sure that I am closing my DatabaseHelper and SQLiteDatabase objects. I've also made sure that I'm closing all of my cursors. This error is pointing toward my method getActiveScheduleInfo, which returns a Cursor object. I've made sure that whenever I call this method, the returned cursor is closed in a Finally block.
Is this incorrect to do it this way? In my methods that call getActiveScheduleInfo, I have multiple return statements in them, based on certain conditions. So, instead of closing the cursor before each return line, I surround the condition testing with a Try, and close everything down in my Finally.
Everything looks like it should be working, so I'd really appreciate any help!
Thanks a lot!
Paul
I was able to figure this out! I hope that this helps someone else out there having the same problem.
I wasn't doing anything inherently incorrect here, but was rather taking too long to close some of my cursors. To give you a very brief background, I could not use a Managed Query or use startManagingCursor, since this code was in a custom class, not an activity. I am building against Android 2.0 (API level 5) so I am not using the new CursorLoader object.
I was taking the following steps:
Opening the database.
Creating a new Cursor and performing my query.
Iterating through the cursor and performing the needed tasks
Performing some other logic
Closing the Cursor and Database in a Finally block.
I found out that my step 4, performing some other logic, coming before closing my Cursor, was causing it to, for lack of a better term, timeout and cause this error. From now on, I read the necessary data from the Cursor, and not ONE LINE OF CODE FURTHER, I close the Cursor. :) This has completely eliminated these random errors, and I have clean-running code again.
I hope that helps others having the same problem! Take care,
Paul