I am dealing with a performance problem in one of my apps. Maybe one of you guys can help me out?!
I have a database with somewhat around 10k entries.
I query for elements using the default query method from the SQLiteDatabase-class.
The query itself is fast enough.
Once the query is complete I have to display the results on a Google Map.
For that I generate a result array from the cursor which holds the marker information.
The method I use looks somewhat like this:
final ArrayList< MarkerElement > result = new ArrayList< MarkerElement >();
cursor.moveToFirst();
while ( !cursor.isAfterLast() ) {
result.add( new MarkerElement(
cursor.getString( COL_TITLE ),
cursor.getString( COL_SNIPPET ),
new LatLng(
cursor.getDouble( COL_LAT ),
cursor.getDouble( COL_LNG ) ),
cursor.getString( COL_OTHER_USEFUL_DATA ) );
cursor.moveToNext();
}
cursor.close();
Where MarkerElement is simply a class that holds the required values for a Marker on a Google Map.
The problem now is, that looping through all cursor elements takes by far to long.
Also I cannot think of a smart way to lazy load the results like in a ListView because I need to show all results simultaneously.
Is there anything I can do to speed up this process significantly?
Any help is greatly appreciated!
Best regards
I'm not so sure the query is really as fast as you think it is. Most likely the actual query is only executed with the cursor.moveToFirst() statement and not when calling query() or rawQuery() (or whatever other query method you're using).
Anyway the query should be fast enough to keep the user waiting only shortly. If not then you might want to consider loading it in chunks using SELECT * FROM your_table LIMIT START, COUNT (e.g. SELECT * FROM your_table LIMIT 0, 1000 to retrieve the first 1000 rows).
The query can't happen on the ui thread so you want to run it in an AsyncTaskLoader or better even a CursorLoader. A CursorLoader without ContentProvider is possible as you can see here: https://stackoverflow.com/a/7422343/534471.
Let's assume you need to run 10 queries for 1000 records each, then you would have 10 CursorLoaders that you could manage using a LoaderManager. LoaderManager manages the cursors (opens and closes them), retains the cursors across orientation changes and runs everything in a background task so no issues with blocking the ui thread. LoaderManager also re-queries the db should the content change (see: https://stackoverflow.com/a/5603959/534471).
When the LoaderManager notifies your fragment or your activity that a cursor has finished loading it will call onLoadFinished() (see: http://developer.android.com/reference/android/support/v4/app/LoaderManager.LoaderCallbacks.html).
What slows down your code isn't just querying the database but also creating 10'000 MarkerElement and another 10'000 LatLng objects. I don't know your requirements but if you see any chance to work without these objects that would certainly speed up your code. Another desirable effect of eliminating the MarkerElements/LatLng is memory usage. For an app targeting phones 20'000 objects with 3 Strings and 2 Double is considerable.
Using CursorLoaders and LoaderManager would allow you to retrieve the values from the cursor and to populate your ui views directly without MarkerElements and LatLng. It would also allow you to load lazily. You can populate the views whenever onLoadFinished() is called for one of the CursorLoaders (onLoadFinished() is called on the ui thread unless the initLoader/restartLoader is called from a non-ui thread). If 1000 views is too much to populate at once, either break down the query into smaller pieces or add a mechanism to populate the views in chunks of 10s or 100s (for each cursor).
In case you need the information you currently store in MarkerElement e.g. when the user selects one of the markers, use setTag() on the view that displays the marker to store the primary key of the db record. Use the key to either retrieve the record from the db or better even from the already queried cursors (that will need some mapping mechanism but it's doable).
Summary:
Split the query into several sub queries to retrieve smaller sets of data
Use CursorLoaders and CursorManager to manage the different queries/cursors
Don't create a MarkerElement and a LatLng for each row but populate the views directly from the returned cursors
Possibly do the populate for each cursor in several steps to keep the ui responsive
use the setTag() on the views to be able to retrieve the data behind the view
Related
Suppose, In my app I have a sqlite table that can contain at most 20 row. Each row has 2 column(id, name). Where I frequently need to search by Id to get Name. For this frequent need I have two solution:
Solution 1: Get rows in a arraylist<model> and then find name from array.
Solution 2: Every time search on sqlite table.
Now please give your opinion which one is better?
Remember again, I need this search in my recycleView item, so it call so frequently.
Thanks
I don't really get what is your real intent, but if your task is to search by id often, I would use
LongSparseArray<String> idsToNames; // or LongSparseArray<Model>
Which will map primitive long to Strings in a more memory-efficient way than Map and will have a better performance than ArrayList when searching.
The advantage over querying SQLite here is that you can do it in a blocking manner instead of having to query database on a background thread every time the lookup runs.
The disadvantage is that whenever data changes in SQLite, you will have to rebuild your idsToNames map. Also, if the number of entries in SQLite will eventually grow, you will end up in a large collection. So I would recommend this approach only if the updates to the database during this session will not happen, and if the data size is always predictable or fixed.
In most of the Android sample codes, populating a ListView from SQLite database is done in two ways,
Prefetch data to List - Execute query, create Model objects for each row then add it to a List and close the Cursor, then populate ListView with List.
Without List and Model objects - Execute query and populate ListView by following the Cursor using moveToFirst, moveToLast, move, as required.
Now I want to know, which of the above method is more memory efficient, in Android ?
The Cursor approach is more memory efficient:
Suppose you have 1000 entries in your database and you have a ListView which can show 10 entries at the same time. If you create a list at first, you'll have to create 1000 model objects (each of which in turn consists of several objects depending on the number of columns of your table) and the listview creates additional 10 views (actually some more, depending on the layout of the list) for displaying the 10 items. Now when the user scrolls the list, in your Adapter you end up copying data from your model objects to the list item views currently in view.
On the other hand, if you use a CursorAdapter, whenever you have to fill a list item with data, you are provided with the Cursor holding exactly the data for that row and you can simply pick the data of the columns you actually need to be displayed in the list item. No need for creating the 1000 model objects.
From a code readability perspective, a model approach would be better because working with Cursors is quite low level, you'll need to know the names of the columns in the database and so on.
I think you need to use Service or at least Thread/Async so your UI thread will not be blocked. Service is better because people can go to other apps while downloading. You can use BroadcastReceiver to communicate with your running Service.
I'm currently working on a SQLite code base with a table that can hold a large amount of records. It has so many records that using a LoaderManager to asynchronously retrieve Cursor objects is becoming slow to display them in a ListView with a CursorAdapter.
If there is one row change in the table being queried, the LoaderManager is notified, and a new Cursor is retrieved. But, this seems inefficient because the Cursor queries for all the rows in the table for the ListView. The GUI isn't being blocked because the Cursor loading is being done in another thread, the problem is that the retrieval of the table rows can take a while. 5-10 seconds can pass on some slower phones before the new record information is displayed.
I'm trying to find a way to efficiently retrieve row changes to update the ListViews's rows without reloading everything.
I've looked into rewriting my code as a internal ContentProvider (hiding SQLiteDatabase) because I've seen it can be used with the app's ContentResolver to send out individual row change notifications via notifyChange().
If I switch to a ContentProvider, will it be as efficient as I've assumed? Upon individual record changes, can the ContentProvider send out events that will allow a ListView to reload only the row change information, and not require a complete requery of all the table information?
when implementing query() method of your ContentProvider return a custom. AbstractWindowedCursor, that way even if the final data set is huge you just fill the small window
I have two methods which read the same data from database, the first returns Cursor and the second returns List of objects.Now I show my items in activity using SimpleCursorAdapter and the first method, byt I can also use the second method and appropriate adapter.
Which of these two ways is beter to use and in the second way which adapter I should use?
P.S sorry for poor english
Definitely go with SimpleCursorAdapter. If possible, always use Cursor if your data comes from database, you save memory by not creating List of objects. Creating objects in Java is expensive with regards to time and memory consumption and you have to bear in mind you are on mobile platform with limited resources. If you are using List of objects for your ListView than use custom adapter extending from ArrayAdapter.
It's not always straightforward to use Cursor although your data comes from database. Let's say you store places in the database defined by its name and location and you want to display them in a ListView sorted by distance from current location. It makes it difficult to execute a query which returns sorted results unless you don't store relative distance in additional column. But you can get Cursor convert it to List of objects and sort this collection before sending it to your ListView.
I have a custom view that loads a model object (let's call it Person, why not). These objects are stored in a DB, obtained through a Loader and inserted into a ListView through a CursorAdapter that instantiates said views. So far, so good.
Now, Person has a reference to another model object, let's say Country. Countries are in its own table, and I need the name of the country (having the ID, of course) to represent it in the list items.
I see three options:
Query the database from the view method that loads the Person data (setPerson()?).
Deep pre-load (I think I just made a term up, sorry) my model objects with the Country information.
Request that the Country data be asynchronously queried and then brought back to the UI.
The problem with (1) is that the UI may block. The problem with (2) is that it leads to heavy data duplication in memory. The problem with (3) is that it complicates flow, maybe unnecessarily.
What should I do? Is the performance hit of (1) important? Maybe (1), query the data from the View, but implement a cache to avoid hitting the database repeatedly for the same Country? Maybe (2), with said cache layer to ensure instances of the object are unique? A 4th option I haven't considered? What do ORMs do?
Thanks!
In your query that you're using for your CursoLoader do an INNER JOIN on the Person and Country tables. The result of the query will then have all the information you want in the single cursor.
EDIT IN RESPONSE TO COMMENT
This is probably the best/cleanest way of going about things. Don't worry about duplication in memory at this point, that's a premature optimization. Besides, how big are your tables really going to be? Let's do a little back of the envelope calculation here. If each row of the joined table takes up 100 bytes (which is a huge row, so I'm thinking worse case scenario here), then even if you had 10000 rows in your result query (once again, that's preeeeetttty large), you'd only be using 1,000,000 bytes, or less than 1 meg of memory.