Develop Reference

Save complex JSON response in SQLite with Room

I'm trying to implement caching of a JSON API response with Room.
The response I get in JSON follows this data class structure:
#Serializable
data class ApiDataResponse(
val success: Boolean,
val message: String? = null,
val albums: List<AlbumResponse> = emptyList()
)
#Serializable
data class AlbumResponse(
val id: String,
val title: String,
val createdBy: String,
val enabled: Boolean,
val keywords: List<String>,
val pics: List<PicResponse>
)
#Serializable
data class PicResponse(
val picUrl: String,
val emojis: List<String>
)
Notes:
#Serializable is from kotlinx.serialization library to parse the JSON response.
These response data classes are only used inside my datasource layer, the view layer doesn't care about an ApiDataResponse and only knows a "pure" version of AlbumResponse called Album and a "pure" version of PicResponse called Pic (by "pure" I mean a data class without external library annotations).
So to implement this cache with Room I could discard the ApiDataResponse and save only the contents of AlbumResponse (and consequently PicResponse), having new data classes for Room entities following this idea:
#Entity(tableName = "albums")
data class AlbumEntity(
#PrimaryKey(autoGenerate = false)
val id: String,
val title: String,
val createdBy: String,
val enabled: Boolean,
val keywords: List<String>, // obstacle here
val pics: List<PicEntity> // obstacle here
)
// obstacle here
// #Entity
data class PicEntity(
val picUrl: String,
val emojis: List<String>
)
I already know how to save simple data in Room, with the simplest JSON I was able to do this task, the problem is that in this more complex scenario I have no idea how to achieve this goal. So I wish someone could guide me in this situation.

Maybe it's a little late, but I would still like to add some interesting information regarding MikeT's answer.
It is not necessary to create a new data class just to transform a custom object into a JSON with TypeConverter, for example:
#Entity(tableName = "albums")
data class AlbumEntity(
#PrimaryKey(autoGenerate = false)
val id: String,
val title: String,
val createdBy: String,
val enabled: Boolean,
val keywords: List<String>,
val pics: List<PicEntity> // can be converted directly
)
import kotlinx.serialization.Serializable
#Serializable // to be able to do the serialize with the kotlinx.serialization
data class PicEntity(
val picUrl: String,
val emojis: List<String>
)
With just these two data classes we can build the TypeConverters as follows:
import androidx.room.TypeConverter
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
class DatabaseConverter {
private val json = Json
#TypeConverter
fun convertStringListToString(strings: List<String>): String =
json.encodeToString(strings)
#TypeConverter
fun convertStringToStringList(string: String): List<String> =
json.decodeFromString(string)
#TypeConverter
fun convertPicEntityListToString(picsEntity: List<PicEntity>): String =
json.encodeToString(picsEntity)
#TypeConverter
fun convertStringToPicEntityList(string: String): List<PicEntity> =
json.decodeFromString(string)
}
Code to create an example dummy list:
object DummyAlbums {
fun createList(): List<AlbumEntity> = listOf(
AlbumEntity(
id = "0001",
title = "Album AB",
createdBy = "Created by AB",
enabled = true,
keywords = listOf("ab"),
pics = dummyPics(albumId = "0001", size = 0)
),
AlbumEntity(
id = "0002",
title = "Album CD",
createdBy = "Created by CD",
enabled = false,
keywords = listOf("cd", "c", "d"),
pics = dummyPics(albumId = "0002", size = 1)
),
AlbumEntity(
id = "0003",
title = "Album EF",
createdBy = "Created by EF",
enabled = true,
keywords = listOf(),
pics = dummyPics(albumId = "0003", size = 2)
)
)
private fun dummyPics(
albumId: String,
size: Int
) = List(size = size) { index ->
PicEntity(
picUrl = "url.com/$albumId/${index + 1}",
emojis = listOf(":)", "^^")
)
}
}
So we can have the following data in table:
I wanted to highlight this detail because maybe it can be important for someone to have a table with the cleanest data. And in even more specific cases, to have it clean, you can do the conversion manually using Kotlin functions, such as joinToString(), split(), etc.

I believe the issue is with columns as lists.
What you could do is add the following classes so the Lists are embedded within a class:-
data class StringList(
val stringList: List<String>
)
data class PicEntityList(
val picEntityList: List<PicEntity>
)
and then change AlbumEntity to use the above instead of the Lists, as per:-
#Entity(tableName = "albums")
data class AlbumEntity(
#PrimaryKey(autoGenerate = false)
val id: String,
val title: String,
val createdBy: String,
val enabled: Boolean,
//val keywords: List<String>, // obstacle here
val keywords: StringList, /// now not an obstacle
//val pics: List<PicEntity> // obstacle here
val emojis: PicEntityList// now not an obstacle
)
To be able to store the "complex" (single object) you need to convert this so some TypeConverters e.g.
class RoomTypeConverters{
#TypeConverter
fun convertStringListToJSON(stringList: StringList): String = Gson().toJson(stringList)
#TypeConverter
fun convertJSONToStringList(json: String): StringList = Gson().fromJson(json,StringList::class.java)
#TypeConverter
fun convertPicEntityListToJSON(picEntityList: PicEntityList): String = Gson().toJson(picEntityList)
#TypeConverter
fun convertJSONToPicEntityList(json: String): PicEntityList = Gson().fromJson(json,PicEntityList::class.java)
}
note this utilises the dependency com.google.code.gson
You then need to have the #TypeConverters annotation to cover the appropriate scope (at the #Database level is the most scope). Note the plural rather than singular, they are different.
To demonstrate the above works, First some functions in an interface annotated with #Dao :-
#Dao
interface AlbumDao {
#Insert(onConflict = OnConflictStrategy.IGNORE)
fun insert(albumEntity: AlbumEntity): Long
#Query("SELECT * FROM albums")
fun getAllAlbums(): List<AlbumEntity>
}
Second an #Database annotated class (note the #TypeConverters annotation) :-
#TypeConverters(RoomTypeConverters::class)
#Database(entities = [AlbumEntity::class], exportSchema = false, version = 1)
abstract class TheDatabase: RoomDatabase() {
abstract fun getAlbumDao(): AlbumDao
companion object {
#Volatile
private var instance: TheDatabase?=null
fun getInstance(context: Context): TheDatabase {
if (instance==null) {
instance = Room.databaseBuilder(context,TheDatabase::class.java,"album.db")
.allowMainThreadQueries()
.build()
}
return instance as TheDatabase
}
}
}
Third some activity code to actually do something (insert some Albums and then extract them writing the extracted data to the Log) :-
class MainActivity : AppCompatActivity() {
lateinit var db: TheDatabase
lateinit var dao: AlbumDao
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
db = TheDatabase.getInstance(this)
dao = db.getAlbumDao()
dao.insert(AlbumEntity(
"Album001", "The First Album","Fred",false,
StringList(listOf("The","First","Album")),
PicEntityList(
listOf(
PicEntity("PE001", listOf("emoji1","emoji2","emoji3")),
PicEntity("PE002",listOf("emoji10")),
PicEntity("PE003", listOf("emoji20","emoji21"))
))
))
dao.insert(AlbumEntity(
"Album002","This is the Second Album","Mary", true,
StringList(listOf("keya","keyb","keyc","keyd","keye")),
PicEntityList(
listOf(
PicEntity("PE011", listOf("emoji30","emoji31")),
PicEntity("PE012", listOf("emoji1","emoji10","emoji20","emoji30"))
))
))
for (a in dao.getAllAlbums()) {
logAlbum(a)
}
}
fun logAlbum(albumEntity: AlbumEntity) {
val keywords = StringBuilder()
for(s in albumEntity.keywords.stringList) {
keywords.append("\n\t$s")
}
val pelog = StringBuilder()
for (pe in albumEntity.emojis.picEntityList) {
pelog.append("\n\tURL is ${pe.picUrl}")
for (emoji in pe.emojis) {
pelog.append("\n\t\tEmoji is ${emoji}")
}
}
Log.d(
"ALBUMINFO",
"Album id is ${albumEntity.id} " +
"Title is ${albumEntity.title} " +
"CreateBy ${albumEntity.createdBy} " +
"Enabled=${albumEntity.enabled}. " +
"It has ${albumEntity.keywords.stringList.size} keywords. " +
"They are $keywords\n. " +
"It has ${albumEntity.emojis.picEntityList.size} emojis. " +
"They are ${pelog}"
)
}
}
Run on the main thread for convenience and brevity
When run then the log contains:-
D/ALBUMINFO: Album id is Album001 Title is The First Album CreateBy Fred Enabled=false. It has 3 keywords. They are
The
First
Album
. It has 3 emojis. They are
URL is PE001
Emoji is emoji1
Emoji is emoji2
Emoji is emoji3
URL is PE002
Emoji is emoji10
URL is PE003
Emoji is emoji20
Emoji is emoji21
D/ALBUMINFO: Album id is Album002 Title is This is the Second Album CreateBy Mary Enabled=true. It has 5 keywords. They are
keya
keyb
keyc
keyd
keye
. It has 2 emojis. They are
URL is PE011
Emoji is emoji30
Emoji is emoji31
URL is PE012
Emoji is emoji1
Emoji is emoji10
Emoji is emoji20
Emoji is emoji30
i.e. the 2 albums have been extracted along with the appropriate embedded lists.
The Albums table itself (via App Inspection) consists of :-
An Alternative, and from a Database perspective, better approach, instead of embedding lists as a single value (String), would have the lists as related tables (with a one-many or a many-many relationship).

How to get Array type data in Firebase in Kotlin?

I have this type of array in firebase but how to fetch it and use in kotlin
I was able to get as String but how to get its as a data class Like this
data class Comment(
val uid: String,
val comment: String,
val stamp: Timestamp
)
and here's the code of getting string
var text by remember { mutableStateOf("loading...") }
FirebaseFirestore.getInstance().collection("MyApp")
.document("Something").get().addOnSuccessListener {
text = it.get("Comments").toString()
}

Firebase has a toObject method that can be used to turn your document into a custom object.
db.collection("Comments")
.get()
.addOnSuccessListener { documents ->
for (document in documents) {
val comment = document.toObject<Comment>()
}
}
The Comment data class should also define default values. So, it should be like...
data class Comment(
val uid: String = "",
val comment: String = "",
#ServerTimeStamp val stamp: Date? = null
)

I got ArrayLists with HashMaps represents my entities entities just using this:
val cleanAnswers: List<Answer> = (document.get(FIELD_ANSWERS)
as ArrayList<HashMap<String, Any>>).map {
Answer(
it[FIELD_VARIANT] as String,
it[FIELD_IS_CORRECT] as Boolean
)
}
My entity:
class Answer(val answer: String,
val isCorrect: Boolean) : Serializable

Compare two array lists of obejcts with different ids

What is the most optimized way to compare two ArrayLists of different object type? Let's say I have an Array Lists of Remedies and Adherences and them being:
ObjectA
data class ObjectA(val data: List<Objecta> = emptyList()) {
data class Objecta(
val _id: String,
val remedy_id: String,
val patient_id: String,
val date_created: Long,
val is_ongoing: Boolean?,
val start_date: Long,
val medicine_id: String,
val instruction: String,
val medicine_name: String,
val medicine_type: String,
val end_date: Long,
val repeat_cycle: Int,
val allow_edit: Boolean,
//val schedule: List<Schedule>?,
val is_current: Boolean,
//val medicine: Medicine?,
val price: Float
)
}
ObjectB
data class ObjectB(val data: List<Objectb> = emptyList()) {
data class Objectb(
val _id: String,
val adherence_id: String,
val patient_id: String,
val remedy_id: String,
val alarm_time: String,
val action: String,
val action_time: String,
val dose_discrete: String,
val dose_quantity: Int,
val note: String
)
}
How can I compare both those two array lists "remedyList" and "adherencesList" where remedy_id is the same and remove items in adherencesList when remedy_id is not present in remedyList.

This should work:
Prepare the input
data class Rdata (val r_id:String)
data class Adata (val a_id:String,val r_id:String)
val rList = (1..10).map { Rdata("$it")}
val aList = (5..20).map { Adata("foo", "$it")}
Do the filtering:
val rIds = rList.map { it.r_id }.toSet()
val resultList = aList.filter { it.r_id in rIds}
After this, the resultList contains objects with r_id 5-10. (that is, 11-20 have been removed)

Try this code:
val allRemedies = remedies.flatMap { it.data }.associateBy { it.remedy_id }
val allAdherences = adherences.flatMap { it.data }.associateBy { it.remedy_id }
val allPairs = allAdherences.mapValues { allRemedies[it.key] to it.value }
Playground example
remedies is the List<Remedies> and adherences is the List<Adherences>
First we flatten the List<Remedies> to List<Remedy> and associate each Remedy to its remedy_id. allRemedies is Map<String, Remedy>.
Similarly for the adherences, we prepare a Map<String, Adherence>
Then we map each value of allAdherences map to a pair of Remedy? and Adherence. If there is no Remedy for that remedy_id it will be null.

If all you care about is filtering Adherences that don't have a matching Remedy, you can create a Set of remedy_ids from the Remedy List, and use that to efficiently check if they exist and filter on that.
val validRemedyIds: Set<String> = remedies.data.mapTo(mutableSetOf(), Remedy::remedy_id)
val filteredAdherences = Adherences(adherences.data.filter { it.remedy_id in validRemedyIds })
If you need to match them up later, you should use a map instead of set:
val remediesById = remedies.data.associateBy(Remedy::remedy_id)
And then there are several ways you can combine to compare. One example:
val adherencesToRemedies: List<Pair<Adherence, Remedy>> =
adherences.data.mapNotNull { remediesById[it.remedy_id]?.run { it to this } }

Gson deserialization for a list in Kotlin

This is what my json looks like
{
"sub": "9",
"auth_time": 1559381757,
"idp": "idsrv",
"role": [
"Employer",
"Employee",
"Student"
],
"iss": "",
"aud": "",
"exp": 1574933757,
"nbf": 1559381757
}
This is the object I want to convert this Json into.
data class Claims (
#SerializedName("nameid") val nameId: String,
#SerializedName("unique_id") val uniqueId: String,
#SerializedName("sub") val sub: String,
#SerializedName("unifiedNumber") val unifiedNumber: String,
#SerializedName("role") var roleList: List<Role>
)
I wrote a custom Deserializer (which works in Java) for the List type
class RoleDeserializer : JsonDeserializer<List<Role>> {
private var roleId = 0
#Throws(JsonParseException::class)
override fun deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): MutableList<Role> {
val resultList = ArrayList<Role>()
if (json.isJsonArray) {
for (e in json.asJsonArray) {
resultList.add(Role(id = roleId++, name = e.asString))
}
} else if (json.isJsonObject) {
resultList.add(Role(id = roleId++, name = json.asString))
} else if (json.isJsonPrimitive) {
if ((json as JsonPrimitive).isString)
resultList.add(Role(id = roleId++, name = json.getAsString()))
} else {
throw RuntimeException("Unexpected JSON type: " + json.javaClass)
}
return resultList
}
}
This is how I register my type adapter
val listType: Type = object : TypeToken<List<Role>>() {}.type
val gson = GsonBuilder().registerTypeAdapter(listType, RoleDeserializer()).create()
val claims = gson.fromJson(stringJson, Claims::class.java)
I still get a parse exception stating that
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 161 path $.role[0]
and my RoleDeserializer is never called. Am I doing something wrong while registering the type adapter?

Try to replace
val listType: Type = object : TypeToken<List<Role>>() {}.type
with
val listType: Type = object : TypeToken<MutableList<Role>>() {}.type

The role is String array in JSON
Use this
#SerializedName("role") var roleList: List<String>
Instead of this
#SerializedName("role") var roleList: List<Role>
Try this
data class Claims (
#SerializedName("nameid") val nameId: String,
#SerializedName("unique_id") val uniqueId: String,
#SerializedName("sub") val sub: String,
#SerializedName("unifiedNumber") val unifiedNumber: String,
#SerializedName("role") var roleList: List<String>
)

Serialized Data Class combined with built-in modifications

I am working on updating the parsing of an API response that uses a Serialized Data Class to parse the JSON response. The serialization works perfectly fine right now, but the new data that I'm attempting to parse into data class is not fully reliant on data in the json. Here is what I mean by that:
The data class is Career, and the new data I need to parse is a set of skills and each have a rating. The json data is very simple and contains the skills as such:
{
// other career data
...
"mathematics_skill": 8,
"critical_thinking_skill": 6
... // the remaining skills
}
Using straight serialization, I would only be able to store the data as such:
data class Career(
// Other career data
#serializableName("mathematic_skill") val mathSkill: Int,
#serializableName("critical_thinking_skill") val mathSkill: Int,
// remaining skills
)
However, I would like to store all skills in an array variable of a custom skills class that not only contains the rating, but also the name of the skill and a color. Basically, when I access the skills data of a career, I would like to access it like such:
val careerMathSkill = career.skills[0]
val mathRating = careerMathSkill.rating
val mathColor = careerMathSkill.color
Is it possible to use the serialized data from the data class to add non-serialized data to the same data class? (Sorry for the weird wording, not sure how else to explain it)
EDIT: Here is what I have:
class CareersRemote(
#SerializedName("careers") val careers: List<Career>
) {
companion object {
fun parseResponse(response: Response<CareersRemote>): CareersResponse {
return if (response.isSuccessful) {
response.body()!!.format()
} else
CareersResponse(listOf(CareersResponse.ErrorType.Generic()))
}
}
fun format(): CareersResponse {
val careers = topCareers.map {
Career(
id = it.id,
title = it.title,
)
}.toMutableList()
return CareersResponse(CareersResponse.SuccessData(careers = careers))
}
data class Career(
#SerializedName("id") val id: String,
#SerializedName("title") val title: String,
)
}
Here is what I am hoping to do in a way
class CareersRemote(
#SerializedName("careers") val careers: List<Career>
) {
companion object {
fun parseResponse(response: Response<CareersRemote>): CareersResponse {
return if (response.isSuccessful) {
response.body()!!.format()
} else
CareersResponse(listOf(CareersResponse.ErrorType.Generic()))
}
}
fun format(): CareersResponse {
val careers = topCareers.map {
Career(
id = it.id,
title = it.title,
)
}.toMutableList()
return CareersResponse(CareersResponse.SuccessData(careers = careers))
}
data class Career(
#SerializedName("id") val id: String,
#SerializedName("title") val title: String,
// skills array that will need to be filled out based on the data I got in the json
var skills: List<Skill>
)
}
EDIT: The suggested solution
class CareersRemote(
#SerializedName("careers") val careers: List<Career>
) {
companion object {
fun parseResponse(response: Response<CareersRemote>): CareersResponse {
return if (response.isSuccessful) {
response.body()!!.format()
} else
CareersResponse(listOf(CareersResponse.ErrorType.Generic()))
}
}
fun format(): CareersResponse {
val careers = topCareers.map {
Career(
id = it.id,
title = it.title,
)
}.toMutableList()
return CareersResponse(CareersResponse.SuccessData(careers = careers))
}
data class Career(
#SerializedName("id") val id: String,
#SerializedName("title") val title: String,
#SerializedName("math_skill") val mathSkill: Int
#SerializedName("other_skill") val mathSkill: Int
) {
var skills: List<Skill> = {
val mathSkill = Skill(name: "Math", rating: mathSkill, color: /**some color*/)
val otherSkill = Skill(name: "Other", rating: otherSkill, color: /**some color*/)
return listOf(mathSkill, otherSkill)
}
}
}

Yes, you can create a custom JsonDeserializer to modify how the JSON is parsed.
Here is a basic example of what that would look like.
class CareerDeserializer : JsonDeserializer<Career> {
override fun deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): Career {
val obj = json.asJsonObject
// standard career data
val id = obj.get("id")?.asString
val name = obj.get("name").asString
// making a Skill object
val skill = Skill(
obj.get("mathematic_skill").asInt,
obj.get("critical_thinking_skill").asInt,
obj.get("swimming_skill").asInt
// etc
)
return Career(id, name, skill)
}
}
And make sure to register that within your GsonBuilder.
val gson = GsonBuilder()
.registerTypeAdapter(Career::class.java, CareerDeserializer())
.create()
Note, you'll also have to create a JsonSerializer if you want to go the other way too.
Edit:
However, if you're just looking to change the syntax of how you're accessing that data, you can do something like this.
data class Career(
// Other career data
val mathSkill: Int,
val thinkSkill: Int
// remaining skills
) {
val skills: List<Int>
get() = listOf(mathSkill, thinkSkill)
}
This would give you a skills list back whenever you needed it, and it would be created when you accessed it, so you won't have to worry about the data being out of sync. This would allow you to access your data as such.
career.skills[0] // get the math skill.
And you can take this another step further by adding a get operator to your Career class.
data class Career(
// Other career data
val mathSkill: Int,
val thinkSkill: Int
// remaining skills
) {
...
operator fun get(pos: Int) = skills[pos]
}
Now, you can simply do
career[0] // get the math skill.
Warning, this is dangerous because you're accessing an Array so you could get OutOfBoundsExceptions. Use constants to help you out.
Edit 2:
val skills = {
listOf(Skill("Math", mathSkill, /** some color */ ),
Skill("Other", otherSkill, /** some color */ ))
}