Develop Reference

Saving complex Class data in Room - "Cannot figure out how to save this field into database. You can consider adding a type converter for it"

I started to learn Room and I'm facing an issue:
Given two classes, one is a Car, and the other one is an Engine iside a Car.
#Entity
class Car{
#PrimaryKey
var id = 0
var name: String? = null
var engine: Engine? = null
}
...
#Entity
class Engine{
#PrimaryKey
var id = 0
var manufacturer: String? = null
}
I also have these classes initalized to tables in my AppDatabase class.
#Database(entities = [Car::class, Engine::class], version = 1)
abstract class AppDatabase : RoomDatabase() {
//...
}
The problem is whenever I simply want to run the project I get the following error message which points to the Car's engine field:
Cannot figure out how to save this field into database. You can consider adding a type converter for it.
Is there no simple way for this? I'm looking for something which saves my data with the least amount of code, like Firestore which do all the work with simple annotations.
Thanks in advance.

AS a car would only have a single engine and that you have a table for the engine as well as a table for the car. Then you have a 1 to many relationship. That is a car can have an engine but the same engine can be used by many cars.
So instead of trying to embed the engine within the car you make a relationship, the car (the child) referencing the engine (the parent).
This is as simple as changing the Car to be:-
#Entity
class Car{
#PrimaryKey
var id = 0
var name: String? = null
var engine: Int? = null
}
An alternative, that would not need the relationship nor a TypeConverter would be to not have the Engine as a table but to use the #Embedded annotation prior to the engine. e.g.
#Entity
class Car{
#PrimaryKey
var id = 0
var name: String? = null
#Embedded
var engine: Engine? = null
}
...
class Engine{
#PrimaryKey
#ColumnInfo(name = "engineId")
var id = 0
var manufacturer: String? = null
}
the name of the column used to store the Engine's id changed as otherwise there would be 2 columns with the same name.
Note that with this way there is no need for the #Entity annotation as you are storing the Engine values within the Car.
This is not considered good practice as if the same engine is used by many cars then you are duplicating data and thus that it is not normalised.
The third and least desirable way from a database perspective is to store a representation of the engine object in a single column. That is to convert the object into a singular storable representation. Typically a JSON string. Thus you need code (a function) to convert from the object to the single value (JSON string) and (another function) to convert from the JSON String to the Object.
With this method not only are you not normalising the data but additionally you end up storing the bloat required to enable the object to be represented. That bloat, from a database, perspective, obfuscating the actual useful stored data to some extent.
In addition there is not a single set/standard library providing the functionality of converting objects to/from JSON, so you have to select a flavour and then include that library in the project.
Here is a class that contains Type Converters that could be used (see comment re library):-
class CarAndEngineTypeConverters{
/* Using Library as per dependency implementation 'com.google.code.gson:gson:2.10.1' */
#TypeConverter
fun convertEngineToJSONString(engine: Engine): String = Gson().toJson(engine)
#TypeConverter
fun convertJSONStringToEngine(jsonString: String): Engine = Gson().fromJson(jsonString,Engine::class.java)
}
This would suit your original classes.
Room needs to be told to use these classes (it works out when) via a #TypeConverters annotation (note the plural and not singular) this it immediately before or after the #Database annotation has the highest level of scope. The annotation itself could be #TypeConverters(value = [CarAndEngineTypeConverters::class])
To demonstrate all three together consider this over the top Car class:-
#Entity
class Car{
#PrimaryKey
var id = 0
var name: String? = null
var engine: Int? = null
#Embedded
var alternativeEngine: Engine? = null
var jsonConvertedEngine: Engine? = null
}
Over the top as the engine is stored 3 times (could be different engines)
The *Engine class
#Entity
class Engine{
#PrimaryKey
#ColumnInfo(name = "engineId")
var id = 0
var manufacturer: String? = null
}
The Type Converters as above.
With the above in place and using within an activity (noting that for brevity/convenience .allowMainThreadQueries has been used):-
db = TheDatabase.getInstance(this)
carAndEngineDAO = db.getCarAndEngineDAO()
var engine1 = Engine()
engine1.manufacturer = "Ford"
engine1.id = carAndEngineDAO.insert(engine1).toInt()
var car1 = Car()
car1.name = "Escort"
car1.engine = engine1.id /* id of the engine */
car1.alternativeEngine = engine1
car1.jsonConvertedEngine = engine1
carAndEngineDAO.insert(car1)
Using Android Studios App inspection the view the database then
The Columns id and name and obviously as expected
The engine column contains the value 0, this is the id of the respective engine in the engine table (maximum 8 bytes to store the id)
The JsonConvertedEngine column stores the JSON representation of the Engine (31 bytes)
The engineId column and manufacturer column stores the respective values (12 bytes).
The Engine Table (only needed for the relationship) is :-

You should use TypeConverters:
At first add this dependency to your project to convert Engine to Json and vice versa
implementation 'com.squareup.retrofit2:converter-gson:2.5.0'
Now you should create an Object class that convert Engine to Json. This class make Engine understandable for Room :
object CommonTypeConverters {
#TypeConverter
#JvmStatic
fun stringToEngine(value: String): Engine = fromJson(value)
#TypeConverter
#JvmStatic
fun engineToString(items: Engine?): String = toJson(items)
inline fun <reified T> toJson(value: T): String {
return if (value == null) "" else Gson().toJson(value)
}
inline fun <reified T> fromJson(value: String): T {
return Gson().fromJson(value, object : TypeToken<T>() {}.type)
}
In the end Engine is not a entity and you should add #Typeconverter annotation to your database class :
#Database(entities = [Car::class], version = 1)
#TypeConverters(CommonTypeConverters::class)
abstract class AppDatabase : RoomDatabase() {
//...
}

Android - How to fetch sub list based on WHERE conditions from Room DB?

I am building an Android application in which I would like to fetch the list of active devices under the project manager.
Trying to put it in different way for better understanding
Project Manager table has list of employees
Employee table has list of devices
Now, we need the list of Project Managers with list of employees with device status either with 1 or 0 based on UI selection.
Entities
#Entity(tableName = TABLE_PROJECT_MANAGER)
data class ProjectManager(
#PrimaryKey
val id: String,
val firstName: String?,
val middleName: String?,
val lastName: String?,
#TypeConverters(EmployeesConverter::class)
var employees: List<Employee>
)
#Parcelize
data class Employee(
val id: String,
val name: String?,
#TypeConverters(DeviceListTypeConverter::class)
val devices : List<Device>? = null
)
#Parcelize
data class Device(
#ColumnInfo(name = "device_id")
#SerializedName("id")
val id: String,
val manufacturer: String?,
val model: String?,
val status: Int,
) : Parcelable
Type Converters:
EmployeesConverter
class EmployeesConverter {
private val moshi = Moshi.Builder().build()
private val membersType = Types.newParameterizedType(List::class.java, Employee::class.java)
private val membersAdapter = moshi.adapter<List<Employee>>(membersType)
#TypeConverter
fun stringToMembers(member: String?): List<Employee>? {
return member?.let {
membersAdapter.fromJson(member)
}
}
#TypeConverter
fun membersToString(members: List<Employee>?): String? {
return members?.let {
membersAdapter.toJson(members)
}
}
}
DeviceListTypeConverter
class DeviceListTypeConverter {
private val moshi = Moshi.Builder().build()
private val membersType = Types.newParameterizedType(List::class.java, Device::class.java)
private val membersAdapter = moshi.adapter<List<Device>>(membersType)
#TypeConverter
fun stringToMembers(member: String?): List<Device>? {
return member?.let {
membersAdapter.fromJson(member)
}
}
#TypeConverter
fun membersToString(members: List<Device>?): String? {
return members?.let {
membersAdapter.toJson(members)
}
}
}
I am little confused on how to achieve this. Please help me out on this.

With what you currently have, you will need a query that has a WHERE clause that will find the appropriate status within the employees column. This is dependant upon how the Type Converter converts the List and the List.
This could be along the lines of:-
#Query("SELECT * FROM $TABLE_PROJECT_MANAGER WHERE instr(employees,'status='||:status)")
fun findProjectManagerWithDevicesAccordingToDeviceStatus(status: String): List<ProjectManager>
NOTE the above will very likely not work as is, you will very likely have to change 'status='||:status according to how the TypeConverter converts the employee list and the device list into the single employees column.
You would call the function with "0" or "1" respectively.
Of course you could use Int for status (Room will convert it to an SQLite string anyway)
In short you are embedding a List with an embedded List into a single value and thus finding the anecdotal needle in that haystack is complicated.
If this were approached from a database perspective then you would have tables (#Entity annotated classes) for each of the List's and as the relationships are probably many-many then tables that map/reference/associate/relate.
So rather than just the ProjectManager table, you would have an Employee table and a Device table and then a table for the mapping of a ProjectManager to the Employee(s) and a table for mapping the Employee to the Device(s). In which case you would have columns with specific values that can be queried relatively efficiently rather than an inefficient search through a complex single relatively large value bloated by the inclusion of data needed solely for the conversion to/from the underlying objects.

Jetpack Compose: Modify Room data class using TextField

Modifying simple values and data classes using EditText is fairly straight forward, and generally looks like this:
data class Person(var firstName: String, var lastName: Int)
// ...
val (person, setPerson) = remember { mutableStateOf(Person()) }
// common `onChange` function handles both class properties, ensuring maximum code re-use
fun <T> onChange(field: KMutableProperty1<Person, T>, value: T) {
val nextPerson = person.copy()
field.set(nextPerson, value)
setPerson(nextPerson)
}
// text field for first name
TextField(
value = person.firstName,
onChange = { it -> onChange(Person::firstName, it) })
// text field for last name name
TextField(
value = person.lastName,
onChange = { it -> onChange(Person::lastName, it) })
As you can see, the code in this example is highly reusable: thanks to Kotlin's reflection features, we can use a single onChange function to modify every property in this class.
However, a problem arises when the Person class is not instantiated from scratch, but rather pulled from disk via Room. For example, a PersonDao might contain a `findOne() function like so:
#Query("SELECT * FROM peopleTable WHERE id=:personId LIMIT 1")
fun findOne(personId: String): LiveData<Person>
However, you cannot really use this LiveData in a remember {} for many reasons:
While LiveData has a function called observeAsState(), it returns State<T> and not MutableState<T>, meaning that you cannot modify it with the TextFields. As such this does not work:
remember { personFromDb.observeAsState()}
You cannot .copy() the Person that you get from your database because your component will render before the Room query is returned, meaning that you cannot do this, because the Person class instance will be remembered as null:
remember { mutableStateOf(findPersonQueryResult.value) }
Given that, what is the proper way to handle this? Should the component that contains the TextFields be wrapped in another component that handles the Room query, and only displays the form when the query is returned? What would that look like with this case of LiveData<Person>?

I would do it with a copy and an immutable data class
typealias PersonID = Long?
#Entity
data class Person(val firstName: String, val lastName: String) {
#PrimaryKey(autoGenerate = true)
val personID: PersonID = null
}
//VM or sth
object VM {
val liveData: LiveData<Person> = MutableLiveData() // your db call
val personDao: PersonDao? = null // Pretending it exists
}
#Dao
abstract class PersonDao {
abstract fun upsert(person: Person)
}
#Composable
fun test() {
val personState = VM.liveData.observeAsState(Person("", ""))
TextField(
value = personState.value.firstName,
onValueChange = { fName -> VM.personDao?.upsert(personState.value.copy(firstName = fName))}
)
}

Room Dao Returns Insert IDs but Data is Missing from Database

When saving a list of objects in my room database using a Dao
#Insert()
fun saveCharmRankMaterialCosts(materialCosts: List<CharmRankCraftingCost>) : List<Long>
And this is used from my repository class to save results from an API call:
val charmRankCosts = CharmRankCraftingCost.fromJsonCraftingCost(
charmRankId.toInt(),
jsonCharmRank.crafting
)
// save crafting/upgrade costs for the rank
val results = charmDao.saveCharmRankMaterialCosts(charmRankCosts)
Log.d("CharmRepository", "Saved charm material costs: ${results.toString()}");
assert(!results.contains(-1))
When running this code, insert ID's are returned and the assertion is never triggered (i.e. no inserts fail).
But when I inspect the data base on the device, most of the supposedly inserted IDs are missing from the table. I'm very confused as to what is going on here. I've debugged this issue for many hours and have been unsuccessful in getting this to work. Is there something obvious I'm missing?

The issue seems to have been related to foreign key constraints. I had a CharmRank data class with multiple related data objects. See below:
/**
* Copyright Paul, 2020
* Part of the MHW Database project.
*
* Licensed under the MIT License
*/
#Entity(tableName = "charm_ranks")
data class CharmRank(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "charm_rank_id")
var id: Int = 0,
#ColumnInfo(name = "charm_id")
var charmId : Int,
#ColumnInfo(name = "charm_rank_level")
var level: Int = 0, // 3
#ColumnInfo(name = "charm_rank_rarity")
var rarity: Int = 0, // 6
#ColumnInfo(name = "charm_rank_name")
var name: String = "",
#ColumnInfo(name = "craftable")
var craftable: Boolean
)
Each charm rank has associated skills and items to craft said rank. These objects are simply relational objects in that they hold the ID of the CharmRank and a SkillRank in the case of the skills object, or the ID of the CharmRank and the ID of the Item object.
data class CharmRankSkill(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "charm_rank_skill_id")
var id: Int,
var charmRankId : Int,
var skillRankId: Int
)
data class CharmRankCraftingCost(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "charm_rank_crafting_cost_id")
var id: Int,
#ColumnInfo(name = "charm_rank_id")
var charmRankId: Int,
#ColumnInfo(name = "charm_rank_crafting_cost_item_quantity")
val quantity: Int,
val itemId: Int
)
Originally in CharmRankCraftingCost, I had a foreign key constraint on the Item object and the CharmRank object. Below is the foreign key constraint on the Item object:
ForeignKey(
entity = Item::class,
parentColumns = ["item_id"],
childColumns = ["itemId"],
onDelete = ForeignKey.CASCADE
)
The Item data object has IDs provided by the remote data source, so when I insert items into it's respective table, the conflict resolution is set to Replace. During the process of saving the relational items to the data base for the CharmRanks, I also have to save the Item objects prior to saving CharmRankCraftingCosts. It seems that what was happening is that when the Item objects are inserted, sometimes the items would get replaced, which would trigger the cascade action of the foreign key resulting in the CharmRankCraftingCosts items I just saved for the CharmRank to be deleted due to the cascading effect.
Removing the foreign key constraint on the Item table solved my issue.

As I understood from the comments, you make a delete before inserts. The problem is that it happens that the insert gets completed before the delete since you do them in separate threads. What you need is to do both in one transaction. Create a method in the the DAO class with #Transaction annotation (Make sure your dao is an abstract class so you can implement the body of this method):
#Dao
public abstract class YourDao{
#Insert(onConflict = OnConflictStrategy.IGNORE)
public abstract List<Long> insertData(List<Data> list);
#Query("DELETE FROM your_table")
public abstract void deleteData();
#Transaction
public void insertAndDeleteInTransaction(List<Data> list) {
// Anything inside this method runs in a single transaction.
deleteData();
insertData(list);
}
}
Read this for Kotlin Version of the code.

"no such column" SQL error from Android Room and multiple #Embedded fields on POJO

The original question is at the very bottom. I have created a minimal (non-)working example of my problem which is hopefully easier to read through. The example is here at gitlab. There is a readme describing the problem. I am pasting some parts of the project here.
The data model is plain simple:
Owner <--(1:N)-- Child --(N:1)--> ReferencedByChild
All I want to do is to read an Owner from the database with all its associated Child objects and for each Child object
also the ReferencedByChild object that it references.
The whole code that reproduces my problem is below. What I am not 100% sure about is the #Relation on the OwnerWithEverything POJO. See below please.
#Database(
entities = [
Owner::class,
Child::class,
ReferencedByChild::class
],
version = 1
)
abstract class AppDatabase : RoomDatabase() {
abstract fun appDao(): AppDao
}
#Dao
abstract class AppDao {
#Insert
abstract fun insertOwner(owner: Owner): Long
#Insert
abstract fun insertChild(child: Child): Long
#Insert
abstract fun insertReferencedByChild(referencedByChild: ReferencedByChild): Long
#Query("SELECT * FROM Child INNER JOIN ReferencedByChild ON Child.referencedByChildId = ReferencedByChild.refByChildId ORDER BY Child.childText")
abstract fun findAllChildrenWithReferencedClasses(): List<ChildWithReferenced>
// Commenting this query out makes the build pass, so something here is incorrect.
#Query("SELECT * FROM Owner")
abstract fun findOwnersWithEverything(): List<OwnerWithEverything>
}
// ENTITIES
#Entity
data class Owner(
#PrimaryKey(autoGenerate = true)
val ownerId: Long,
val ownerText: String
)
#Entity(
foreignKeys = [
ForeignKey(
entity = Owner::class,
parentColumns = arrayOf("ownerId"),
childColumns = arrayOf("referencedOwnerId"),
onDelete = ForeignKey.CASCADE
),
ForeignKey(
entity = ReferencedByChild::class,
parentColumns = arrayOf("refByChildId"),
childColumns = arrayOf("referencedByChildId"),
onDelete = ForeignKey.CASCADE
)
]
)
data class Child(
#PrimaryKey(autoGenerate = true)
val childId: Long,
val childText: String,
val referencedOwnerId: Long,
val referencedByChildId: Long
)
#Entity
data class ReferencedByChild(
#PrimaryKey(autoGenerate = true)
val refByChildId: Long,
val refText: String
)
// POJOS
// The Child has exactly one ReferencedByChild reference. This POJO joins those two
class ChildWithReferenced(
#Embedded
var child: Child,
#Embedded
var referencedByChild: ReferencedByChild
)
class OwnerWithEverything {
#Embedded
var owner: Owner? = null
#Relation(
parentColumn = "ownerId",
entityColumn = "referencedOwnerId",
entity = Child::class // which entity should be defined here?
)
var childrenWithReferenced: List<ChildWithReferenced>? = null
}
Building this code results in this error message:
error: There is a problem with the query: [SQLITE_ERROR] SQL error or missing database (no such column: refByChildId)
I think that the Owner query is badly constructed, but I am not entirely sure. If that is the problem, what is the correct way to construct the query?
This is the original question
I have a nested POJO structure that should represent a single Game having multiple Rounds and each Round has a single Topic associated with it:
class GameWithRounds {
#Embedded
var game: Game? = null
#Relation(
parentColumn = "id",
entityColumn = "gameId",
entity = RoundRoom::class
)
var rounds: List<RoundWithTopic>? = null
}
class RoundWithTopic(
#Embedded
var round: RoundRoom,
#Embedded(prefix = "topic_")
var topic: Topic
)
The embedded annotation on Topic specifies a prefix because there are clashing id properties.
The Room Query that can fetch those classes is:
#Query("SELECT Topic.id as topic_id, Topic.name as topic_name, (...), RoundRoom.* FROM RoundRoom INNER JOIN Topic ON RoundRoom.topicId = Topic.id")
abstract fun findRoundsWithTopics(): List<RoundWithTopic>
However, building the project gives me Room errors:
There is a problem with the query: [SQLITE_ERROR] SQL error or missing database (no such column: topic_id)
Even though when I induce a warning about which fields are actually present, this is what Room tells me:
Columns returned by the query: topic_id, topic_name, topic_description, topic_language, topic_keywords, topic_sourceUrls, topic_furtherUrls, topic_questions, order, gameId, topicId, status, id. Fields in cz.melkamar.sklapecka.model.RoundWithTopic: order, gameId, topicId, status, id, topic_id, topic_name, topic_description, topic_language, topic_keywords, topic_sourceUrls, topic_furtherUrls, topic_questions, topic_image.
The topic_id column is there in the query result! Why am I getting this error?
For completeness, this is the entities:
#Entity
data class Game(
#PrimaryKey(autoGenerate = true)
val id: Long = 0,
#Embedded
val gameConfigurationEmbed: GameConfigurationEmbed
)
data class GameConfigurationEmbed(
var secondsPerTurn: Int,
var maxSecondsPerTurn: Int,
var bonusSecondsPerAnswer: Int
)
#Entity(
foreignKeys = [
ForeignKey(
entity = Game::class,
parentColumns = arrayOf("id"),
childColumns = arrayOf("gameId"),
onDelete = ForeignKey.CASCADE
),
ForeignKey(
entity = Topic::class,
parentColumns = arrayOf("id"),
childColumns = arrayOf("topicId"),
onDelete = ForeignKey.CASCADE
)
]
)
#TypeConverters(RoomConverters::class)
data class RoundRoom(
val order: Int,
var gameId: Long,
val topicId: String,
var status: RoundStatus = RoundStatus.CREATED,
#PrimaryKey(autoGenerate = true)
val id: Long = 0
) {
enum class RoundStatus {
CREATED, UPCOMING, IN_PROGRESS, FINISHED
}
}
#Entity
data class Topic(
#PrimaryKey val id: String,
val name: String,
val description: String,
val language: String,
val keywords: List<String>,
val sourceUrls: List<String>,
val furtherUrls: List<String>,
val questions: List<String>,
val image: ByteArray?
)

After some research, specifically looking at this link : How can I represent a many to many relation with Android Room?
the only answers we found would be to
either create a hand-made sql query to handle this type of situation
where you have a many-to-many relationship
OR
to alternatively have an additional joining entity which gets
updated as the rest of the objects are updated. With this approach, you can get the ID's and then create additional queries as needed

It seems embedded fields and type converter is not properly used on observing the question. I don't want to go in detail in the solution of the question since it is trying to use complex relations and I cannot test it replicating in my machine.
But I want to provide insight on using Embedded fields and TypeConverter.
Let's take an example from the question above:
Game table has fields
id, secondsPerTurn, maxSecondsPerTurn, bonusSecondsPerAnswer.
It is okay to create entity like below.
#Entity
data class Game(
#PrimaryKey(autoGenerate = true)
val id: Long = 0,
#Embedded
val gameConfigurationEmbed: GameConfigurationEmbed
)
data class GameConfigurationEmbed(
var secondsPerTurn: Int,
var maxSecondsPerTurn: Int,
var bonusSecondsPerAnswer: Int
)
Here in the SQLite table, data is actually stored in four different columns but Room does CRUD operation based on data class structure giving higher feasibilty to the developers.
TypeConverter
Type converter will be helpful if we want to store non primitive data types or same type of data which will not be covered by #Embedded.
For example, a football game can be held in two places: home and away. Home and Away can have the same field names like placeName, latitude, longitude. In this case, we can create data class and type converters like below:
data class GamePlace(
val placeName:String,
val latitude:String,
val longitude:String
)
#Entity
data class Game(
#PrimaryKey(autoGenerate = true)
val id: Long = 0,
#Embedded
val gameConfigurationEmbed: GameConfigurationEmbed
#TypeConverters
var home: GamePlace? = null,
#TypeConverters
var away: GamePlace? = null,
)
object Converters {
private val gson = Gson()
#TypeConverter
#JvmStatic
fun fromGamePlace(gamePlace: GamePlace?): String? {
return if (gamePlace == null) null else gson.toJson(gamePlace)
}
#TypeConverter
#JvmStatic
fun toGamePlace(jsonData: String?): GamePlace? {
return if (jsonData.isNullOrEmpty()) null
else gson.fromJson(jsonData, object : TypeToken<GamePlace?>() {}.type)
}
}
While using type converter, converter class should be defined in database class as below:
#Database(
entities = [Game::class /* ,more classes here*/],
version = 1
)
#TypeConverters(Converters::class)
abstract class AppDatabase : RoomDatabase() {
abstract fun gameDao(): GameDao
//.....
}
I hope this will help to deal with Embedded and TypeConverter in Room.