In an app I'm working on, I need to very simply fetch data from a Room database. There are relations involved. Everything was working perfectly with simple queries returning LiveData, however, the size of the data that needs to be retrieved grew much larger than expected and it also contains blobs (images), which can make the query very slow. I've resolved to using the Paging library, implemented as follows, but for some reason the #relation annotation simply doesn't work anymore.
The entity fetched is a DTO, looks basically like this:
data class EntityOtherAnotherDTO(
var id: Long? = null,
var name: String? = null,
...,
#Relation(parentColumn = "id", entityColumn = "entity_id", entity = OtherEntity::class)
var others: List<OtherEntity>,
#Relation(parentColumn = "id", entityColumn = "entity_id", entity = AnotherEntity::class)
var anothers: List<AnotherEntity>
)
The query:
#Query("SELECT * FROM other
JOIN entity ON entity.id = other.entity_id
JOIN another ON entity.id = another.entity_id
WHERE entity.deleted = 0
ORDER BY
CASE WHEN other.some_column IS NULL THEN 1 ELSE 0 END,
other.some_column ASC,
entity.some_other_column DESC")
fun getAllEntityOtherAnotherDTOs(): DataSource.Factory<Int, EntityOtherAnotherDTO>
When the query was like this: fun getAllEntityOtherAnotherDTOs(): LiveData<List<EntityOtherAnotherDTO>> everything worked just fine. The results were ordered as required and all data was fetched, including the lists annotated with #relation. But after changing the return type to DataSource.Factory and of course implementing a paging adapter, the relations return empty.
The ordering still works perfectly, the query appears to be working exactly as before, the paging also works, but the data is simply missing. All the columns in the entity table are there (name, some_other_column etc.), the relations are the only but major problem.
(I can provide further details about the paging implementation, if that's of any relevance.)
Turns out this is an issue in Room which can happen even when you don't use the Paging library, given a large-ish (hundreds of results+) query.
There is no solution, but there is a workaround for 1:1 relations: using #embedded instead of #relation. That can however complicate things through the need of setting a prefix and enumerating all columns in the query and giving them aliases. That's pain but such is life. Room sucks.
Or if the joined entity doesn't have too many columns and there aren't any duplicate names, it works just as fine to copy those columns/properties in the DTO that is returned by the query.
Related
I have a situation where I need some information about my results, in my code I have this data class/Room's entity.
#Entity(primaryKeys = ["searchId","page","pr_id"])
data class ProductResponse(
var searchId: Int,
val page: Int,
#Embedded(prefix = "pr_")
val products: ProductSearchFormatted
)
Where you can see in the Android Studios App Inspector:
The data is saved as spected in ROOM, when I try to load it from ROOM:
#Query(
"SELECT * FROM PagedSearchResponse WHERE searchId ==:input"
)
fun loadPagedSearchResponse(input: Int): PagingSource<Int, ProductResponse>
I just need the data in the same order that was previously saved, and got the data in different order (ordered by pr_id):
I found out that if I change the primaryKeys order, like
#Entity(primaryKeys = ["pr_id","searchId","page"])
data class ProductResponse(
var searchId: Int,
val page: Int,
#Embedded(prefix = "pr_")
val products: ProductSearchFormatted
)
Now the data's order from ROOM is correct.
Why does this happen? Does the primaryKeys order matter?
Changing primaryKeys order changes the order of the data saved in ROOM
NO it does not, the data is ALWAYS saved in the order in which it is inserted. What is changing in your case, is the ORDER in which the data is extracted. That is because you aren't saying in what ORDER you want the data to be extracted and are leaving that choice to the query planner.
Why is this happened?, does the primaryKeys order matter?.
Yes it can do, especially in the absence of other indexes. Certainly pr_id before search_id will make a difference as the order within the index will be different and that as the WHERE clause is on the search_id then it is likely that the primary key index will be used (as in both cases search_id is an initial column (see the links below))
The query planner is an AI that tries to pick the fastest and most efficient algorithm for each SQL statement.
see :-
https://www.sqlite.org/queryplanner.html
https://www.sqlite.org/optoverview.html
https://www.sqlite.org/queryplanner-ng.html
If you want data to be in ORDER then you should specify an ORDER clause (ORDER BY ....). That is the only way to guarantee an ORDER. Assuming an ORDER without an ORDER clause will very likely result in issues.
Saying that using pr_id prior to search_id makes the composite (multiplte column) index likely to be more beneficial as the pr_id (according to the data shown) is less repeated than the search_id.
The way I fix it, it was simpler than I expected.
In the entity, I added a field called "order", and in my Mediator (I'm using Paging 3) basically did this:
list.mapIndex { s,t ->
ProductResponse(
...
order = s
...
)
}
That way, I'm following the EXACT order from Backend without the need to modify any primaryKeys.
I know it's not the best practice to store an image in DB directly and we should store a path instead. But in my case this is a must.
I am able to store list of images perfectly fine defined as:
#ColumnInfo(name = "picture")
var picture: ByteArray? = null
I came across solutions that suggests using (typeAffinity = ColumnInfo.BLOB). So I changed my column to:
#ColumnInfo(name = "picture", typeAffinity = ColumnInfo.BLOB)
var picture: ByteArray? = null
I haven't noticed anything significant in performance. I wonder what are the possible advantages of using typeAffinity or disadvantages of not using it?
It maybe worth mentioning my images are always under 1 megabytes.
There is no real advantage/disadvantage, certainly not at run time perhaps marginally at compile time.
That is all that using typeAffinity=? does is override the typeAffinity being determined by the type of the field/column of the variable.
As you have var picture: ByteArray this would be resolved to a column type of BLOB anyway.
If you wished you could compile with both and see the resultant SQL used by looking at the generated java.
Perhaps consider the following Entity that uses both:-
#Entity(tableName = "user")
data class UserEntity(
#PrimaryKey
val userid: Long = 0,
var picture1: ByteArray? = null,
#ColumnInfo(typeAffinity = ColumnInfo.BLOB)
var pitcure2: ByteArray? = null
)
In the generated Java (use Android view as highlihted) then in the #Database class (UserDatabase in the example) suffixed by _Impl (so UserDatabase_Impl in the example) the following is a screen shot of the generated Java :-
Android highlighted indicates where to select the Android view.
The highlight in the code explorer shows the respective code (UserDatabase_Impl) in the expanded java (generated) directory
The createAllTables method is the method used to create the table(s)
room_master_table is a room specific table used for verification of an existing table with the schema to detect if there are differences.
The code (SQL) generated for the creation of the table is :-
_db.execSQL("CREATE TABLE IF NOT EXISTS `user` (`userid` INTEGER NOT NULL, `picture1` BLOB, `pitcure2` BLOB, PRIMARY KEY(`userid`))");
i.e. the definition of the columns picture1 and picture2 are identical bar the column names.
NOTE please heed the WARNING in regards to not changing the generated code.
I'm trying to update my Android knowledge with the new Android Architecture Components. I'm looking at how to persist data with Room, but it seems like it is practically unusable if you actually have relational data and I'm hoping you can help me understand something I've missed or confirm that Room really doesn't work for any kind of serious relational data.
Here's my problem in as small a nutshell as I can offer.
I've got an app for animal breeders and I have an Animal entity. Let's say I have a Weight entity which is a record of how much an animal weighed on a given date. An Animal has a to-many relationship to Weight to create a history of how much they weighed over time.
According to the Room training documentation, I would need to create a 3rd class called AnimalWithWeights like:
data class AnimalWithWeights(
#Embedded val Animal: Animal,
#Relation(
parentColumn = "AnimalId",
entityColumn = "AnimalId"
)
val Weights: List<Weight>
)
and add a function to my DAO like fun getAnimalsWithWeights(): List<AnimalWithWeights>.
#Transaction
#Query("SELECT * FROM Animal")
fun getAnimalsWithWeights(): List<AnimalWithWeights>
But I don't have just one relationship.
My Animal entity has relationships to other Animals like father, mother, surrogate mother, breeding parter, and relationships to other entity types like Breed, Breeder, Buyer, DNA, and to-many relationships for Weights, Notes, MedicalTreatments, MatingRecords, Ultrasounds, etc.
Unless I'm missing something, it sounds like I need to create a class that is AnimalWithWeights and then wrap that in a 4th class called AnimalWithWeightsAndNotes
and wrap that in a 5th class called AnimalWithWeightsAndNotesAndMedicalTreatments
and wrap that in a 6th class called AnimalWithWeightsAndNotesAndMedicalTreatmentsAndMatingRecords
and wrap that in a 7th class called AnimalWithWeightsAndNotesAndMedicalTreatmentsAndMatingRecordsAndUltrasounds...
and I haven't even gotten to ...AndFatherAndMotherAndSurrogateMotherAndBreedingParterAndBreedAndBreederAndBuyerAndDNA....
I'm currently using ORMLite to handle my sqlite persistence and it handles relational data well, but it seems like Room is only really designed to handle flat data.
Am I missing something with Room?
I'd like to be able to use some of the new features like LiveData<> so that I can get automatic callbacks when my data is updated and so that's one of the reasons I was looking into Room, but it just seems too rudimentary to be able to handle a realistic relational data model.
It's not clear from your post whether you're aware of possibility to link several #Relations in mentioned auxilary 3rd class. With Animal having one-to-many and one-to-one relations to other entities technically you need add just one final class AnimalWithAllStuff:
data class AnimalWithAllStuff(
#Embedded val Animal: Animal,
#Relation(
parentColumn = "AnimalId",
entityColumn = "AnimalId"
)
val weights: List<Weight>,
#Relation(
parentColumn = "AnimalId",
entityColumn = "AnimalId"
)
val notes: List<Notes>,
....
#Relation(
parentColumn = "fatherId",
entityColumn = "AnimalId"
)
val father: Animal,
#Relation(
parentColumn = "motherId",
entityColumn = "AnimalId"
)
val mother: Animal,
....
)
Technically on each #Relation Room internally invokes separate query and if you use LiveData Room sets tracker to notify observer on any change in Weight, Notes and so on.
I doubt though that with that structure you can get, for example, father's and mother's weight, notes and so on.
Let's say I have an Order and a User Entity that resembles something like this:
User.java:
#Entity
public class User {
#PrimaryKey
public long id;
#NonNull
public String username;
}
Order.java:
#Entity(foreignKeys = #ForeignKey(entity = User.class,
parentColumns = "id",
childColumns = "user_id",
onDelete = ForeignKey.CASCADE,
onUpdate = ForeignKey.CASCADE),
indices = #Index("user_id"))
public class Order {
#PrimaryKey
public long id;
public String description;
#ColumnInfo(name = "user_id")
public long userId;
#Ignore
private User user;
}
So this way, I have an Order with a User inside it, on the database I am only saving this relation with the user id as a Foreign Key and not the object itself (it's #Ignored), so everything is nice and relational and can be expanded to be used with other types in several different ways and queries and etc.
Now, the question is, how do I get an Order with the User object automatically populated by Room inside it?
If I use #Embedded, then Order and User will live on the same table, which is not good for relational separation of the types. I could return them both together with a JOIN but still this only works for data types that have different names (maybe they both have a column named "description"?), besides, inserting an Order with a User inside it wont be simple with a JOIN. #Relation only works for one-to-many and it needs a List, which is not my case.
I thought that maybe a #TypeConverter would be the best option here, converting between long and User, but this also is tricky. The #TypeConverter would need a reference to the DAO so it could query the User and the #TypeConverter is a static method, invoked by Room, so passing the DAO can be tricky and lead to many code smells, besides this extra query for each User would trigger multiple searches that won't be in the same #Transaction.
I am new to Room, but I bet there's a proper way to fix this, to use Room with relational types as it was intended to be used, I just can't find out how to make this work simply and nicely nor I can find it in any documentations.
Now, the question is, how do I get an Order with the User object automatically populated by Room inside it?
I think there is not an out of box solution for this.
IMHO, the entity model (i.e. the Order mapping a FK with User.id) should not be propagated to upper layer like domain/presentation layer, providing a DataMapper to transform the entity model to a domain model (i.e. a User contains an Order) for the upper layer may be a better option.
I have three tables Notes, Tags and join table with foreign keys of Note and Tag called NoteTagJoin, but how can I return Note with all Tags as one response?
Here is query to get Note and Tags:
SELECT n.*,t.* FROM notes n INNER JOIN note_tag_join nt ON n.entryId = nt.noteId INNER JOIN tags t ON t.tagId = nt.tagId WHERE n.entryId =:noteId
And here is response class which has to hold note and list of tags:
data class NoteResponse(
#Embedded
var note: Note? = null,
#Relation(parentColumn = "entryId", entityColumn = "tagId", entity = Tag::class)
var tags: List<Tag>? = null
)
But tags list is empty on response just note is there, I know for sure that Tags and Note exists in db and Join table has right foreign keys because all other queries works so my guess is that Query is wrong or NoteResponse class is wrong because I use annotation #Relation which I don't need, but if I don't add #Relation annotation on tags it throws error that room doesn't know what is this list so how to do it? I can't find any references for this, documentation only mentions embedding one class in POJO but no examples for Lists and all similar posts talks only about inserting list.
please see my realization of many-to-many relationship CinemaActorJoinDao
You can instead my code , replace with your , if you have any question , i will try ask to you :)