Room very slow read operation - android

I have a room database and I'm trying to read data from it, when reading 16k records, the operation takes 15seconds.
this is my code
#Entity(tableName = "reading_table")
data class DatabaseReading(
#PrimaryKey(autoGenerate = true)
val readingId: Int = 0,
val readingNum: Int,
val deviceId: String,
val readingTime: Long,
val topRh: Double,
val topTemp: Double,
val botRh: Double,
val botTemp: Double,
)
this is my DAO
#Query("SELECT * FROM reading_table WHERE deviceId = :id ORDER BY readingTime ASC")
suspend fun getDeviceReadings(id: String): List<DatabaseReading>
then from my viewmodel I'm running the query
fun getReadings() {
Timber.d("get readings start")
viewModelScope.launch {
withContext(Dispatchers.IO) {
val time = measureTimeMillis {
devicesRepository.database.deviceDatabaseDao.getDeviceReadings(
deviceID
)
}
Timber.d("get readings time $time")
}
}
}
getReadings is taking 15563ms for 16155 records.
how can I improve this?

it turns out that the issue was caused by the database inspector plugin on android studio.
as long as I don't use it everything runs normally.

I mean for a mobile device that's a pretty sizable amount of records, chances are you dont need and probably wont use all 16k of them.
You should probably use the Paging library for room to only load chunks of data instead of everything

Related

Android - How to fetch sub list based on WHERE conditions from Room DB?

I am building an Android application in which I would like to fetch the list of active devices under the project manager.
Trying to put it in different way for better understanding
Project Manager table has list of employees
Employee table has list of devices
Now, we need the list of Project Managers with list of employees with device status either with 1 or 0 based on UI selection.
Entities
#Entity(tableName = TABLE_PROJECT_MANAGER)
data class ProjectManager(
#PrimaryKey
val id: String,
val firstName: String?,
val middleName: String?,
val lastName: String?,
#TypeConverters(EmployeesConverter::class)
var employees: List<Employee>
)
#Parcelize
data class Employee(
val id: String,
val name: String?,
#TypeConverters(DeviceListTypeConverter::class)
val devices : List<Device>? = null
)
#Parcelize
data class Device(
#ColumnInfo(name = "device_id")
#SerializedName("id")
val id: String,
val manufacturer: String?,
val model: String?,
val status: Int,
) : Parcelable
Type Converters:
EmployeesConverter
class EmployeesConverter {
private val moshi = Moshi.Builder().build()
private val membersType = Types.newParameterizedType(List::class.java, Employee::class.java)
private val membersAdapter = moshi.adapter<List<Employee>>(membersType)
#TypeConverter
fun stringToMembers(member: String?): List<Employee>? {
return member?.let {
membersAdapter.fromJson(member)
}
}
#TypeConverter
fun membersToString(members: List<Employee>?): String? {
return members?.let {
membersAdapter.toJson(members)
}
}
}
DeviceListTypeConverter
class DeviceListTypeConverter {
private val moshi = Moshi.Builder().build()
private val membersType = Types.newParameterizedType(List::class.java, Device::class.java)
private val membersAdapter = moshi.adapter<List<Device>>(membersType)
#TypeConverter
fun stringToMembers(member: String?): List<Device>? {
return member?.let {
membersAdapter.fromJson(member)
}
}
#TypeConverter
fun membersToString(members: List<Device>?): String? {
return members?.let {
membersAdapter.toJson(members)
}
}
}
I am little confused on how to achieve this. Please help me out on this.
With what you currently have, you will need a query that has a WHERE clause that will find the appropriate status within the employees column. This is dependant upon how the Type Converter converts the List and the List.
This could be along the lines of:-
#Query("SELECT * FROM $TABLE_PROJECT_MANAGER WHERE instr(employees,'status='||:status)")
fun findProjectManagerWithDevicesAccordingToDeviceStatus(status: String): List<ProjectManager>
NOTE the above will very likely not work as is, you will very likely have to change 'status='||:status according to how the TypeConverter converts the employee list and the device list into the single employees column.
You would call the function with "0" or "1" respectively.
Of course you could use Int for status (Room will convert it to an SQLite string anyway)
In short you are embedding a List with an embedded List into a single value and thus finding the anecdotal needle in that haystack is complicated.
If this were approached from a database perspective then you would have tables (#Entity annotated classes) for each of the List's and as the relationships are probably many-many then tables that map/reference/associate/relate.
So rather than just the ProjectManager table, you would have an Employee table and a Device table and then a table for the mapping of a ProjectManager to the Employee(s) and a table for mapping the Employee to the Device(s). In which case you would have columns with specific values that can be queried relatively efficiently rather than an inefficient search through a complex single relatively large value bloated by the inclusion of data needed solely for the conversion to/from the underlying objects.

Unable to Insert values in Room Database in Android [Kotlin]

I am trying to insert values inside my room database but the App Inspection is showing nothing. I have a complex app structure with lot of tables and one database. I am able to perform CRUD operations on all of the tables except for one. I am not able to identify what I am doing wrong.
Code -->
Entity
#Entity(tableName = "add_time")
data class TimeEntity(
#PrimaryKey(autoGenerate = true)
var id: Int? = null,
#ColumnInfo(name = "start_time")
var startTime: String? = "",
#ColumnInfo(name = "end_time")
var endTime: String? = "",
#ColumnInfo(name = "running_time")
var runningTime: String? = "",
)
DAO
#Dao
interface FloorTimeDao {
//Insert time
#Insert(onConflict = OnConflictStrategy.IGNORE)
suspend fun insertTimeForScheduling(timeEntity: TimeEntity) {
}
}
REPOSITORY
class AddTimeRepository(private val timeDao: FloorTimeDao) {
//insert
#WorkerThread
suspend fun insertTime(time: TimeEntity) = timeDao.insertTimeForScheduling(time)
}
VIEWMODEL
class AddTimeViewModel(private val repository: AddTimeRepository) : ViewModel() {
//insert
fun insertTime(timeEntity: TimeEntity) = viewModelScope.launch {
repository.insertTime(timeEntity)
}
}
VIEWMODEL FACTORY
#Suppress("UNCHECKED_CAST")
class AddTimeViewModelFactory(private val repository: AddTimeRepository) :
ViewModelProvider.Factory {
override fun <T : ViewModel> create(modelClass: Class<T>, extras: CreationExtras): T {
if (modelClass.isAssignableFrom(AddTimeViewModel::class.java)) {
return AddTimeViewModel(repository) as T
}
throw IllegalArgumentException("Unknown ViewModel Class")
}
}
CODE INSIDE FRAGMENT FOR ADDING VALUE
//Inserting new time in the table
binding.btnAddTime.setOnClickListener {
try {
timeRunning = timeEnd - timeStart
timeRunning = abs(timeRunning)
Timber.d("Final Time start : $timeStart")
Timber.d("Final Time end : $timeEnd")
Timber.d("Final Time running : $timeRunning")
val timeEntity = TimeEntity(
startTime = timeStart.toString(),
endTime = timeEnd.toString(),
runningTime = timeRunning.toString()
)
Timber.d("Time Entity: $timeEntity")
addTimeViewModel.insertTime(timeEntity)
} catch (e: Exception) {
Timber.d("Exception : $e")
}
}
Everything seems to be added correctly and I still cannot seem to find out what I am doing wrong here. Any help would be appreciated. If you need more code I can provide just let me know.
NOTE: With this code basically I am trying to store the time in room database which is taken from the user using Time Picker Dialog.
Edit 1: Found something which I don't know if it is related to the issue or anything. But for the tables in which I am able to insert and read the data the function says something like this :
and for the table (TimeEntity) the function says this :
The difference is that (for the one in which it is working) the functions says Choose Implementation and have a green symbol on left side. But for the table for which it is not working the function says Choose overridden method.
Update: I was able to fix the issue by creating a new Dao Interface for the table. I am still not very sure what might have been the issue as both DAO files looks same. But I have some doubt that it might be due to the room automatic implementations when we create new DAO's or table.
#PrimaryKey(autoGenerate = true)
var id: Int? = null,
Try to change id Int? - To non nullable type Int or Long.
Why do you need to have predefined (initialized) values in entity constructor?
I was able to fix the issue by creating a new Dao file with same functions. The issue may have been due to the automatic implementation of DAO's provided by Room Library.

Parsing tiny JSON from android assets is incredibly slow

I am writing a simple Android app in Kotlin, which will show prayers divided into categories to user. There are 5 JSON files in assets folder, each of them has just around 10 KiB.
I use Klaxon for parsing the JSON files into those two data classes:
data class Prayer(val prayerName: String, val verseTitle: String, val verseBody: String,
val prayerLine: String, val prayerBody: String, val prayerEnding: String)
data class PrayerCategory(val title: String, val bgImage: String, val headerImage: String,
val prayers : List<Prayer>)
Here is the code I use for parsing the prayers:
private fun loadPrayerNames(jsonFile: String) {
val millis = measureTimeMillis {
val input = assets.open("${jsonFile}.json")
val prayerCategory = Klaxon().parse<PrayerCategory>(input)
if (prayerCategory != null) {
for (prayer in prayerCategory.prayers) {
val prayerName = prayer.prayerName
prayersMap[prayerName] = prayer
}
}
}
println("Loading prayer category took $millis ms.")
}
As you can see, there is just one access to assets. No assets.list(), no bullshit.
And as you noticed, I have measured the time.. make your guesses.. Here is the debug output:
Loading prayer category took 3427 ms.
Yup, that's right. Loading and parsing 10KiB big JSON took 3.5 SECONDS! I repeat. No rocket science involved. Just parsing 10 KiB JSON. 3.5 seconds..... Hmm..
Btw, I am testing it on Nokia 6.1, which is a pretty snappy phone.
So.. my questions:
What causes this behaviour?
Is there any way how to speed this up other than building a database for storing about 50 prayers?
I will be very thankful for your help!
Android assets seem to have bad reputation when it comes to performance. However, my testing proved that in this situation, it was Klaxon library who was responsible.
After finding major performance problem in Klaxon (see https://github.com/cbeust/klaxon/issues/154), which is still not fixed, I have tried recommended alternative: Moshi (https://github.com/square/moshi).
Moshi did improve performance, but parsing my JSON still took about 1 second.
After these experiments, I have resorted to the good old fashioned parsing using JSONObject:
data class Prayer(val prayerName: String, val verseTitle: String, val verseBody: String,
val prayerLine: String, val prayerBody: String, val prayerEnding: String) {
companion object {
fun parseJson(json: JSONObject) : Prayer = Prayer(json.getString("prayerName"),
json.getString("verseTitle"), json.getString("verseBody"),
json.getString("prayerLine"), json.getString("prayerBody"),
json.getString("prayerEnding"))
}
}
data class PrayerCategory(val title: String, val bgImage: String, val headerImage: String,
val prayers : List<Prayer>) {
companion object {
fun parseJson(json: JSONObject): PrayerCategory {
val prayers = ArrayList<Prayer>()
val prayersArray = json.getJSONArray("prayers")
for(i in 0 until prayersArray.length()) {
prayers.add(Prayer.parseJson(prayersArray.getJSONObject(i)))
}
return PrayerCategory(json.getString("title"), json.getString("bgImage"),
json.getString("headerImage"), prayers)
}
}
}
This has reduced parsing time from 3427 ms to 13ms. Case closed. ;-)

Fold with Kotlin in a property of a data class model in an Android app

I'm not sure the title is clarifying enough, so I'll explain here:
I have four models:
This model is the one I use for parsing the JSON from a file
#JsonClass(generateAdapter = true)
data class Account(
val account: String,
val balance: String,
val transactions: List<Transaction>
)
This Account has a list of Transactions:
#JsonClass(generateAdapter = true)
data class Transaction(
val id: String,
val amount: String,
val description: String,
val otherAccount: String,
val date: String
)
For the UI layer, I have some equivalent models:
data class AccountEntity(
val account: String,
val balance: String,
val transactions: List<TransactionEntity>
)
and
data class TransactionEntity(
var amount: BigDecimal,
var balanceBefore: BigDecimal,
var balanceAfter: BigDecimal,
val description: String,
val otherAccount: String,
val date: Date
)
The tricky thing I want to do is, that an Account has a balance and I need to show in every TransactionData the balanceBefore and the balanceAfter which are respectively, the balance of the account BEFORE this transaction was made, and the balance of the account AFTER this transaction was made.
For me this is a bit tricky not only to understand but also to implement.
I thought of maybe implementing some sort of fold operation in every transaction, but I'm not sure how to apply it only in the balanceBefore and the balanceAfter properties.
Any other idea?
Thanks a lot in advance!
Well, it's some sort of reduce/fold function that also allows you to map your Transaction objects.
What this function do is to track the current balance in an accumulator and transform each Transaction object:
inline fun <R> List<Transaction>.map(
initial: Int,
transform: (transaction: Transaction, beforeBalance: Int, afterBalance: Int) -> R
): List<R> {
var accumulator = initial
val destination = ArrayList<R>()
for (transaction in this) {
val after = accumulator - transaction.amount
destination.add(transform(transaction, accumulator, after))
accumulator = after
}
return destination
}
And you can use it like:
account.transactions.map(account.balance, {t, b, a-> TransactionEntity(...)})
To be more generic and functional, you can move this line accumulator - transaction.amount to the function parameter as a fold input, and call the function foldMap or whatever which reduce your current transaction and also map it.

How to get a specific Entity in Room

So, I'm using Room database to store courses and I'm stuck on the method that returns the course with the name(course) that I want because it's always returning null. I have diminished my database to have 2 courses with the course variable as:
As you can see in the picture above, when I try to get the CourseEnt in the Repository with course = fun, which I can see below that it exists, it returns a LiveData with a null value instead of the CourseEnt that I wanted.
Any idea on what I'm doing wrong or on what should I look into with debugger?
Here's the code:
Entity:
#Entity(tableName = "courses_table")
data class CoursesEnt (#PrimaryKey val course: String,
val location: String,
val description: String,
val difficulty: Double,
val distance: Double,
val photos: ListInt,
val category: String,
val activities: ListString)//ListString is a type converter that converts a String into a List<String> and vice-versa
DAO:
#Dao
interface CoursesDao {
#Query("SELECT * from courses_table ORDER BY course ASC")
fun getAllCourses(): LiveData<List<CoursesEnt>>
#Query("SELECT * FROM courses_table WHERE course LIKE :str")
fun getCourse(str: String):LiveData<CoursesEnt>
...
}
Repository:
class CoursesRepository(private val coursesDao: CoursesDao){
val allCourses: LiveData<List<CoursesEnt>> = coursesDao.getAllCourses()
var singleCourse: LiveData<CoursesEnt> = coursesDao.getCourse("")
#WorkerThread
fun getCourse(str: String) {
singleCourse = coursesDao.getCourse(str)
}
...
}
Read documentation, liveData will always return null for straight call, you have to observe LiveData, the result value from Room will be in block. This is some example of usage
mViewModel.getAllUsers().observe( this#YourActivity, Observer {
// it - is all users from DB
})
but if you will call
val users = mViewModel.getAllUsers()
the result will be null

Categories

Resources