I have a custom view that loads a model object (let's call it Person, why not). These objects are stored in a DB, obtained through a Loader and inserted into a ListView through a CursorAdapter that instantiates said views. So far, so good.
Now, Person has a reference to another model object, let's say Country. Countries are in its own table, and I need the name of the country (having the ID, of course) to represent it in the list items.
I see three options:
Query the database from the view method that loads the Person data (setPerson()?).
Deep pre-load (I think I just made a term up, sorry) my model objects with the Country information.
Request that the Country data be asynchronously queried and then brought back to the UI.
The problem with (1) is that the UI may block. The problem with (2) is that it leads to heavy data duplication in memory. The problem with (3) is that it complicates flow, maybe unnecessarily.
What should I do? Is the performance hit of (1) important? Maybe (1), query the data from the View, but implement a cache to avoid hitting the database repeatedly for the same Country? Maybe (2), with said cache layer to ensure instances of the object are unique? A 4th option I haven't considered? What do ORMs do?
Thanks!
In your query that you're using for your CursoLoader do an INNER JOIN on the Person and Country tables. The result of the query will then have all the information you want in the single cursor.
EDIT IN RESPONSE TO COMMENT
This is probably the best/cleanest way of going about things. Don't worry about duplication in memory at this point, that's a premature optimization. Besides, how big are your tables really going to be? Let's do a little back of the envelope calculation here. If each row of the joined table takes up 100 bytes (which is a huge row, so I'm thinking worse case scenario here), then even if you had 10000 rows in your result query (once again, that's preeeeetttty large), you'd only be using 1,000,000 bytes, or less than 1 meg of memory.
Related
I've only observed this on Android 9 and possibly only on Samsung devices. I'm storing multiple JSON responses to multiple serialized strings into my DB later to be typeConverted using Moshi again into a model.
The query that causes this error is:
#Query(“SELECT * FROM tasks”)
public abstract Flowable<List<TaskEntity>> getAll();
The last instance had a total of about 392,000 characters TOTAL in the table. These are actually split up into about 5500 character size entries within the table.
Why would the cursor have a problem with ~11k byte sized entries? Does the fact that I'm selecting * mean the cursor is grabbing the whole table into memory and not a single row at a time?
Why only Android 9?
Thanks.
Does the fact that I'm selecting * mean the cursor is grabbing the whole table into memory and not a single row at a time?
SELECT * means you are retrieving all columns. A SELECT without a WHERE clause (or other types of constraints) means that you are retrieving all rows. So, SELECT * FROM tasks will attempt to retrieve the entire table contents into memory.
You could add #Transaction to this function, as that may help get past this error. Quoting the documentation:
When used on a Query method that has a SELECT statement, the generated code for the Query will be run in a transaction. There are 2 main cases where you may want to do that:
If the result of the query is fairly big, it is better to run it inside a transaction to receive a consistent result. Otherwise, if the query result does not fit into a single CursorWindow, the query result may be corrupted due to changes in the database in between cursor window swaps.
If the result of the query is a POJO with Relation fields, these fields are queried separately. To receive consistent results between these queries, you also want to run them in a single transaction.
Even better would be to not load the entire table's content's into memory (and then convert the entire table's rows into entity objects). Heap space is limited.
Why only Android 9?
No clue. I wouldn't worry about that — if you focus on retrieving less data, that will have benefits for all your users.
Suppose, In my app I have a sqlite table that can contain at most 20 row. Each row has 2 column(id, name). Where I frequently need to search by Id to get Name. For this frequent need I have two solution:
Solution 1: Get rows in a arraylist<model> and then find name from array.
Solution 2: Every time search on sqlite table.
Now please give your opinion which one is better?
Remember again, I need this search in my recycleView item, so it call so frequently.
Thanks
I don't really get what is your real intent, but if your task is to search by id often, I would use
LongSparseArray<String> idsToNames; // or LongSparseArray<Model>
Which will map primitive long to Strings in a more memory-efficient way than Map and will have a better performance than ArrayList when searching.
The advantage over querying SQLite here is that you can do it in a blocking manner instead of having to query database on a background thread every time the lookup runs.
The disadvantage is that whenever data changes in SQLite, you will have to rebuild your idsToNames map. Also, if the number of entries in SQLite will eventually grow, you will end up in a large collection. So I would recommend this approach only if the updates to the database during this session will not happen, and if the data size is always predictable or fixed.
I have some Items with a foreign key to a parent Category. Is there an efficient way to query some Items and get fully populated Category objects?
The only approach I'm aware of is to use foreignAutoRefresh, or to refresh the Categories manually after the Item query, but this would result in an extra db hit for EACH Item object.
This can be done with a single JOIN, but if I do that is there any support for automatically building out the Category objects? Part (maybe all) of the problem is I don't fully understand the QueryBuilder's join functionality, but based on this answer it sounds like it doesn't do this:
Notice, however, that you can only get entities from the query builder
using this mechanism. If you want to get your two description fields
from different objects then you would have to still use a raw-query.
Alternatively, is there a way to refresh a collection of Categories in place with a single query, so that I can take the bare id-only Categories from the Item query and refresh them all?
In case it's of interest, the goal is to display these hierarchically in an ExpandableListView. Please let me know if I can provide any more info. I am comfortable throwing some SQL at it and populating Java objects myself if need be, but I'd rather stay within the framework if possible.
I need to populate ListView with List of objects returned from my Dao object.
The items get returned after 3 seconds, obviously to much time for the user to wait...
I'm using BaseAdapter as the ListView adapter.
2 questions:
How can get rid of the 3 seconds waiting time? Should I just retrieve the entire list of objects in a seperate worker Thread and display dialog in the meanwhile? Is there any mechanism that allows me to get the first, let's say... 20 records, display them and fetch the rest of the records while the user scrolls down the list?
If I would use cursors, rather than ORMLite, the list would then query the DB as the user scrolls down the list, releasing the objects of the hidden cells and the cells themselves, and not keeping all the objects of the cursor in the memory. How can I achieve this behavior with ORMLite?
I hope I was clear enough, despite the bad English ;)
Thanks.
You might want to load the data in an AsyncTask, and display a ProgressDialog while it loads. Lot of Android apps do this.
Cannot OrmLite return a DataProvider instead of the while list? (I too wanted to look into ORM on Android but the management decided against it "Its slow", but I still badly want it)
I maintain an application that is collecting a lot of information and is storing these information in an ArrayList.
In detail this ArrayList is defined as ArrayList<FileInformation> which has some member like:
private File mFile;
private Long mSize;
private int mCount;
private Long mFilteredSize;
private int mFilteredCount;
private int mNumberOfFilters;
etc.
This approach is working but is not very flexible when I would like to introduce some new functionality. It also has some limitations in terms of memory usage and scale-ability. Because of this I did some tests if a database is the better approach. From the flexibility there is no question, but somehow I'm not able to make it running fast enough to become a real alternative.
Right now the database has just one table like this:
CREATE TABLE ExtContent (
"path" TEXT not null,
"folderpath" TEXT not null,
"filename" TEXT,
"extention" TEXT,
"size" NUMERIC,
"filedate" NUMERIC,
"isfolder" INTEGER not null,
"firstfound" NUMERIC not null,
"lastfound" NUMERIC not null,
"filtered" INTEGER not null
);
The performance issue is immense. Collecting and writing ~14000 items takes ~3mins! when writing into the database and just 4-5secs if written into the ArrayList.
Creating the database in-memory does not make a big difference.
As my experience in terms of SQLITE is rather limited, I started by creating the entries via the android.database.sqlite.SQLiteDatabase.insert methode.
As there was no meaningful difference between a file based and a in-memory database, I guess using BEGIN TRANSACTION and COMMIT TRANSACTION will not make any difference.
Is there some way to optimize this behavior?
Just for clarification, putting BEGIN TRANSACTION and END TRANSACTION will increase the performance greatly. Quoted from http://www.sqlite.org/faq.html#q19 :
SQLite will easily do 50,000 or more INSERT statements per second on an average desktop computer. But it will only do a few dozen transactions per second. By default, each INSERT statement is its own transaction...
I had a similar issue on an app I was coding on the weekend.
Is the data in the database to be included in the app when it's released? If so, bulk inserts aren't they way to go, instead you want to look at creating the database and including it in the assets directory and copying it over to the device. Here's a great link.
Otherwise I'm not sure you can do much to improve performance, this link explains methods on bulk inserting into an SqlLite Database.
Edit: You may also want to post your insert code too.
This is opretty obvious. Assuming you already allocated object to insert into. ( This is the same workload for bot solutions ) Let's compare alternatives:
Inserting in ArrayList does:
- (optional) allocate new chinks of cells for pointers if necessary
- insert object pointer into array list on the end
... really fast
INserting into sqlite:
-prepare insertion query ( I hope you use prepared query, and do not construct it from strings)
-perform database table insertion with modifications of indexes etc.
... a lot of work
Only advantage of database is that you can:
- query it later
- it handles external storage transparently allowing you to have much more entities
But it comes at cost of performance.
Depending on what you are for, there could be better alternatives.
For example, in my android games I store highscore entries in JSON file and utilise
GSON Pull parser / databinding layer ( https://github.com/ko5tik/jsonserializer ) to create objects out of it. Typical load time for 2000 entries from external storage is about 2-3 seconds