Android Paging3 Room performance - android

I'm playing with Paging3 with Room. I wrote a little test app that populates a Room DB with a single table with 5 string fields (plus its id). I populate the table with 100,000 items.
The code for test project is here: https://github.com/johngray1965/wordlePeople
The list is a recyclerview with a PagingDataAdapter.
There are options in the UI to select filters for the color and gender.
The initial loading works fine. Filtering down the data works fine. Clearing the filter (going back to full list) sends the app into a dizzy for a number of seconds. I've looked at the profiling data in Android Studio (which I'm finding to be a source of frustration). I can see where I did the touch for the clear, I can see that resulted in garbage collection. Then the UI seems to be moderately busy for a long time (the UI is unresponsive for 5-7 seconds when not running the profiler).
It doesn't look like the query is what's slow. But I don't see the problem if there are only 2,000 items in the table.
I've turned the debug logging in Room, and I can see the queries always get all the items in a subquery, and then apply the limit and offset:
SELECT * FROM ( SELECT * FROM wordle_people ) LIMIT 90 OFFSET 0
BTW, I see the problem without debug logging in Room, on an emulator, or on a real device.
Here's the entity:
#Entity(
tableName = "wordle_people",
indices = [
Index(value = ["gender"], unique = false),
Index(value = ["color"], unique = false),
]
)
data class WordlePerson(
#PrimaryKey(autoGenerate = true)
override var id: Long? = null,
var firstName: String = "",
var middleName: String = "",
var lastName: String = "",
#TypeConverters(Gender::class)
var gender: Gender,
#TypeConverters(Color::class)
var color: Color
): BaseEntity()
The Dao:
#Dao
abstract class WordlePersonDao : BaseDao<WordlePerson>("wordle_people") {
#Query("SELECT * FROM wordle_people")
abstract fun pagingSource(): PagingSource<Int, WordlePerson>
#Query("SELECT * FROM wordle_people where gender IN (:genderFilter)")
abstract fun pagingSourceFilterGender(genderFilter: List<Gender>): PagingSource<Int, WordlePerson>
#Query("SELECT * FROM wordle_people where color IN (:colorFilter)")
abstract fun pagingSourceFilterColor(colorFilter: List<Color>): PagingSource<Int, WordlePerson>
#Query("SELECT * FROM wordle_people where color IN (:colorFilter) AND gender IN (:genderFilter)")
abstract fun pagingSourceFilterGenderAndColor(genderFilter: List<Gender>, colorFilter: List<Color>): PagingSource<Int, WordlePerson>
}
Relevant parts of the ViewModel:
private val stateFlow = MutableStateFlow(FilterState(mutableSetOf(), mutableSetOf()))
#OptIn(ExperimentalCoroutinesApi::class)
val wordlePeopleFlow = stateFlow.flatMapLatest {
Pager(
config = PagingConfig(
pageSize = 30,
),
pagingSourceFactory = {
when {
it.colorSet.isEmpty() && it.genderSet.isEmpty() ->
wordlePeopleDao.pagingSource()
it.colorSet.isNotEmpty() && it.genderSet.isNotEmpty() ->
wordlePeopleDao.pagingSourceFilterGenderAndColor(it.genderSet.toList(), it.colorSet.toList())
it.colorSet.isNotEmpty() ->
wordlePeopleDao.pagingSourceFilterColor(it.colorSet.toList())
else ->
wordlePeopleDao.pagingSourceFilterGender(it.genderSet.toList())
}
}
).flow
}
And finally in the fragment, we get the data and pass to the adapter:
lifecycleScope.launchWhenCreated {
viewModel.wordlePeopleFlow.collectLatest { pagingData ->
adapter.submitData(pagingData)
}
}

The smaller red circle is the db lookup, the larger red circle is all the DiffUtil. Its selected, you can see FlameChart for it. I might be wrong, but this looks like a bug in the PagingDataAdapter

Related

How to create Entity and data classes by ROOM in android

How to create Entity and data classes by ROOM in android?
I have JSON structure:
data class ListResponse(val item: ListItem)
data class ListItem(
#SerializedName("id")
val id: List<CheckUnCheckItem>
)
data class CheckUnCheckItem(
#SerializedName("check")
val check: CheckItem,
#SerializedName("unCheck")
val UnCheck: UnCheckItem
)
data class CheckItem(
#SerializedName("url")
val url: String,
#SerializedName("text")
val text: String,
#SerializedName("color")
val color: String
)
data class UnCheckItem(
#SerializedName("url")
val urlUnCheck: String,
#SerializedName("text")
val textUnCheck: String,
#SerializedName("color")
val colorUnCheck: String
)
But How can I create such ROOM Entity?
Do I need to use #TypeConverter?
#Entity(tableName = TABLE_NAME)
data class ListEntity(
#PrimaryKey #SerializedName("id")
val id: CheckUnCheckItem,
#SerializedName("check")
val check: CheckItem,
#SerializedName("unCheck")
val unCheck: UnCheckItem,
#SerializedName("url")
val url: String,
#SerializedName("text")
val text: String,
#SerializedName("size")
val size: String
){
companion object{
const val TABLE_NAME = "db_table"
}
class RoomTypeConverters{
#TypeConverter
fun convertCheckItemListToJSONString(checkList: CheckItem): String = Gson().toJson(checkList)
#TypeConverter
fun convertJSONStringToCheckItemList(jsonString: String): CheckItem = Gson().fromJson(jsonString,CheckItem::class.java)
}
}
is my data and entity classes are correct?
Do I need class witch extends RoomDatabase?
Or better I need to separate db and create for check and uncheck another db?
Or better I need to separate db and create for check and uncheck another db?
As database implies it is able to store data not just one but many. As such a single database is all that would be required. SQLite is a relational database and is designed to store related data. Related data is typically stored in multiple tables. So again a single database will very likely be sufficient.
Do I need to use #TypeConverter?
You never actually need Type Converters. However, for any Object, other than those directly handled (e.g. String, Int, Long, Double, Float, ByteArray) then you either need to break these down into such handled objects or have a types converter that will convert the object to and from such an object.
For example, based upon your #Entity annotated ListEntity class then:-
field id would need a TypeConverter as the type CheckUnCheckItem is not an object type that can be directly handled by Room. So you would need two TypeConverters that could convert from a CheckUncheckItem to and from a type that can be handled by Room.
fields check and uncheck would need two TypeConverters (and it looks as though the Type Converters you have coded will handle the conversion).
fields url,text and size, as they are all String types do not need Type Converters as Room handles strings.
Room has to know about the Type Converters. So you need an #TypeConverters annotation. It's placement defines the scope. Using the annotation to preced the #Database annotation has the most far reaching scope.
Do I need class witch extends RoomDatabase?
Yes. However it has to be an abstract class and should have an abstract function to retrieve an instance of each #Dao annotated interface (or abstract class, in which case the functions have to be abstract, there is no need for an abstract class with Kotlin as functions in an interface can have bodies)).
This class should be annotated with the #Database annotation, the entities parameter of the annotation should include the list of classes for each each table (#Entity annotated class). e.g.
#TypeConverters(value = [ListEntity.RoomTypeConverters::class])
#Database(entities = [ListEntity::class], exportSchema = false, version = 1)
abstract class TheDatabase: RoomDatabase(){
}
However, using the above along with your classes results in a build error as per:-
ListEntity.java:11: error: Cannot figure out how to save this field into database. You can consider adding a type converter for it. private final a.a.so74708202kotlinroomentitydesign.CheckUnCheckItem id = null;
as explained CheckUnCheckItem cannot be handled by Room.
Amending the RoomTypeConverters class to be:-
class RoomTypeConverters{
#TypeConverter
fun convertItemListToJSONString(invoiceList: Item): String = Gson().toJson(invoiceList)
#TypeConverter
fun convertJSONStringToItemList(jsonString: String): Item = Gson().fromJson(jsonString,Item::class.java)
#TypeConverter
fun convertCheckUnCheckItemToJSONString(cuc: CheckUnCheckItem): String = Gson().toJson(cuc)
#TypeConverter
fun convertJSONStringToCheckUnCheckItem(jsonString: String): CheckUnCheckItem = Gson().fromJson(jsonString,CheckUnCheckItem::class.java)
}
Resolves the build issue and in theory you have a potentially usable database.
However, you obviously need code to access the database. As such you would very likely want to have. as previously mentioned, an #Dao annotated interface e.g
#Dao
interface TheDAOs {
#Insert
fun insert(listEntity: ListEntity): Long
#Query("SELECT * FROM ${TABLE_NAME}")
fun getAll(): List<ListEntity>
}
This will suffice to allow rows to be inserted into the database and for all the rows to be extracted from the database into a List<ListEntity).
From the built database you need to get an instance of the TheDAOs and thus the #Database annotated class could then be
:-
#TypeConverters(value = [ListEntity.RoomTypeConverters::class])
#Database(entities = [ListEntity::class], exportSchema = false, version = 1)
abstract class TheDatabase: RoomDatabase(){
abstract fun getTheDAOsInstance(): TheDAOs
}
To demonstrate actual use of the above then consider the following code in an activity:-
class MainActivity : AppCompatActivity() {
lateinit var roomDBInstance: TheDatabase
lateinit var theDAOs: TheDAOs
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
roomDBInstance = Room.databaseBuilder(this,TheDatabase::class.java,"The_database_name.db")
.allowMainThreadQueries() /* NOTE ADDED FOR CONVENIENCE AND BREVITY */
.build()
/* Note the database itself does not yet exist, it's creation is delayed until an attempt is made to access it. So:- */
theDAOs = roomDBInstance.getTheDAOsInstance() /* Still the database is not created/accessed */
showData(theDAOs.getAll()) /* No data has been added BUT the database will now exist */
theDAOs.insert(
ListEntity(
id = CheckUnCheckItem(
check = Item (
url ="URL001",
text = "TEXT001",
color = "RED"
),
unCheck = Item(
url ="URL002",
text = "TEXT002",
color = "BLUE"
)
),
check = Item(url = "URL003", text ="TEXT003", color ="WHITE"),
unCheck = Item(url = "URL004", text = "TEXT004", color = "BLACK"),
url = "URL005", text = "TEXT005", size = "BIG"
)
)
showData(theDAOs.getAll())
}
fun showData(listEntities: List<ListEntity>) {
for (li in listEntities) {
Log.d(
"DBINFO",
"id is $li.id.check.url${li.id.check.text}.... " +
"\n\tcheck is ${li.check.url} .... " +
"\n\tuncheck is ${li.unCheck.url} ...." +
"\n\turl is ${li.url} text is ${li.text} size is ${li.size}"
)
}
}
}
The output to the log being:-
D/DBINFO: id is ListEntity(id=CheckUnCheckItem(check=Item(url=URL001, text=TEXT001, color=RED), unCheck=Item(url=URL002, text=TEXT002, color=BLUE)), check=Item(url=URL003, text=TEXT003, color=WHITE), unCheck=Item(url=URL004, text=TEXT004, color=BLACK), url=URL005, text=TEXT005, size=BIG).id.check.urlTEXT001....
check is URL003 ....
uncheck is URL004 ....
url is URL005 text is TEXT005 size is BIG
The Database via App Inspection being"-
So finally
is my data and entity classes are correct?
From a database aspect yes, they work after a few amendments. However, I suspect that your classes are probably not what you intended.
An Alternative Approach
If this were to be approached from a database perspective and normalised and without bloat and without the need for type converters then consider the following:-
The embedded Item's (uncheck and check) are basically repetition, so could probably be a table (related to the db_table). Hence 2 tables. One for the ListEntity (Alternative) and another for the Items (AlternativeItem) so the 2 #Entity annotated classes could be:-
/* Alternative Approach */
#Entity(
/* Foreign Keys NOT REQUIRED, they enforce Referential Integrity */
foreignKeys = [
ForeignKey(
entity = AlternativeItem::class,
parentColumns = ["alternativeItemId"],
childColumns = ["unCheckIdMap"]
/* OPTIONAL within a Foreign Key, they help automatically maintain Referential Integrity*/,
onDelete = ForeignKey.CASCADE,
onUpdate = ForeignKey.CASCADE
),
ForeignKey(
entity = AlternativeItem::class,
parentColumns = ["alternativeItemId"],
childColumns = ["checkIdMap"],
onDelete = ForeignKey.CASCADE,
onUpdate = ForeignKey.CASCADE
)
]
)
data class Alternative(
#PrimaryKey
val id: Long?=null,
#ColumnInfo(index = true)
val unCheckIdMap: Long, /* map to the id of the related Item (AlternativeItem) for the uncheck */
#ColumnInfo(index = true)
val checkIdMap: Long, /* map to the id of the related Item (AlternativeItem) for the uncheck */
val url: String,
val text: String,
val size: String
)
#Entity
data class AlternativeItem(
#PrimaryKey
val alternativeItemId: Long?=null,
val alternativeItemUrl: String,
val alternativeItemText: String,
val alternativeItemColor: String
)
As you would typically want the Alternative along with it's related AlternativeItems then a POJO that caters for the togetherness :-
data class AlternativeWithUncheckAndCheck(
#Embedded
val alternative: Alternative,
#Relation(entity = AlternativeItem::class, parentColumn = "unCheckIdMap", entityColumn = "alternativeItemId")
val unCheck: AlternativeItem,
#Relation(entity = AlternativeItem::class, parentColumn = "checkIdMap", entityColumn = "alternativeItemId")
val check: AlternativeItem
)
There would be a need for some extra functions in the #Dao annotated interface, so :-
#Insert
fun insert(alternative: Alternative): Long
#Insert
fun insert(alternativeItem: AlternativeItem): Long
#Transaction
#Query("")
fun insertAlternativeAndUncheckAndCheck(alternative: Alternative, uncheck: AlternativeItem, check: AlternativeItem): Long {
var uncheckId = insert(uncheck)
var checkId = insert(check)
return insert(Alternative(null,url = alternative.url, text = alternative.text, size = alternative.size, unCheckIdMap = uncheckId, checkIdMap = checkId ))
}
#Transaction
#Query("SELECT * FROM alternative")
fun getAllAlternativesWithRelatedUnCheckAndCheck(): List<AlternativeWithUncheckAndCheck>
note that the insertAlternativeAndUncheckAndCheck does what it says (note that it is overly simple and could need some enhancements to expand upon the principle)
To demonstrate this, all that is then required is to add the new entities to the entities parameter and to then add some code to the activity.
The amended #Database annotation:-
#Database(entities = [ListEntity::class, /* for the alternative approach */ Alternative::class, AlternativeItem::class], exportSchema = false, version = 1)
The activity code (that caters for both approaches in a similar/equivalanet way of storing and retrieving the data) :-
class MainActivity : AppCompatActivity() {
lateinit var roomDBInstance: TheDatabase
lateinit var theDAOs: TheDAOs
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
roomDBInstance = Room.databaseBuilder(this,TheDatabase::class.java,"The_database_name.db")
.allowMainThreadQueries() /* NOTE ADDED FOR CONVENIENCE AND BREVITY */
.build()
/* Note the database itself does not yet exist, it's creation is delayed until an attempt is made to access it. So:- */
theDAOs = roomDBInstance.getTheDAOsInstance() /* Still the database is not created/accessed */
showData(theDAOs.getAll()) /* No data has been added BUT the database will now exist */
theDAOs.insert(
ListEntity(
id = CheckUnCheckItem(
check = Item (
url ="URL001",
text = "TEXT001",
color = "RED"
),
unCheck = Item(
url ="URL002",
text = "TEXT002",
color = "BLUE"
)
),
check = Item(url = "URL003", text ="TEXT003", color ="WHITE"),
unCheck = Item(url = "URL004", text = "TEXT004", color = "BLACK"),
url = "URL005", text = "TEXT005", size = "BIG"
)
)
showData(theDAOs.getAll())
/* Alternative equivalent */
theDAOs.insertAlternativeAndUncheckAndCheck(
Alternative(url = "URL005", size = "BIG", text = "TEXT005", checkIdMap = -1, unCheckIdMap = -1),
check = AlternativeItem(alternativeItemUrl = "URL001", alternativeItemText = "TEXT001", alternativeItemColor = "RED"),
uncheck = AlternativeItem(alternativeItemUrl = "URL002", alternativeItemText = "TEXT002", alternativeItemColor = "BLUE" )
)
showAlternativeData(theDAOs.getAllAlternativesWithRelatedUnCheckAndCheck())
}
fun showData(listEntities: List<ListEntity>) {
for (li in listEntities) {
Log.d(
"DBINFO",
"id is $li.id.check.url${li.id.check.text}.... " +
"\n\tcheck is ${li.check.url} .... " +
"\n\tuncheck is ${li.unCheck.url} ...." +
"\n\turl is ${li.url} text is ${li.text} size is ${li.size}"
)
}
}
fun showAlternativeData(listAlternatives: List<AlternativeWithUncheckAndCheck>) {
for (la in listAlternatives) {
Log.d("DBALTINFO",
"id is ${la.alternative.id} URL is ${la.alternative.url} TEXT is ${la.alternative.text} SIZE is ${la.alternative.size} " +
"\n\t UNCHECK id is ${la.unCheck.alternativeItemId} url is ${la.unCheck.alternativeItemUrl} text is ${la.unCheck.alternativeItemText} color is ${la.unCheck.alternativeItemColor}" +
"\n\t CHECK id is ${la.check.alternativeItemId} url is ${la.check.alternativeItemUrl} text is ${la.check.alternativeItemText} color is ${la.check.alternativeItemColor}")
}
}
}
Note that the Alternative code is probably more along the lines of what you probably want according to the interpretation of the shown JSON.
When run then the result is now:-
D/DBINFO: id is ListEntity(id=CheckUnCheckItem(check=Item(url=URL001, text=TEXT001, color=RED), unCheck=Item(url=URL002, text=TEXT002, color=BLUE)), check=Item(url=URL003, text=TEXT003, color=WHITE), unCheck=Item(url=URL004, text=TEXT004, color=BLACK), url=URL005, text=TEXT005, size=BIG).id.check.urlTEXT001....
check is URL003 ....
uncheck is URL004 ....
url is URL005 text is TEXT005 size is BIG
D/DBALTINFO: id is 1 URL is URL005 TEXT is TEXT005 SIZE is BIG
UNCHECK id is 1 url is URL002 text is TEXT002 color is BLUE
CHECK id is 2 url is URL001 text is TEXT001 color is RED
it is suspected that BLACK/WHITE or RED/BLUE is superfluous in your interpretationof the JSON to data classes (and hence excluded in the alternative).
The database, via App Inspection (in regards to the alternative approach) is:-
and :-
i.e. only the actual data is store the BLOAT (field/type descriptions, separators, enclosing data) is not stored thus
the database will hold more data in less space.
the handling of the data will thus be more efficient (e.g. a buffer can hold more actual data instead of BLOAT).
querying the data such as for example searching for all BLUE's is directly a search for that, whilst with converted data you may have issues distinguishing between BLOAT and actual data
However, the negative, is that more code and thought is required.
Note this answer is intended to deal with the basic principles and is most certainly not fully comprehensive.

Android Paging v3 with just local Room database: how do I scroll to specific position in a paged list?

I'm not sure if this is the correct use case for Paging v3 but here goes.
All my data is in a local Room database (no network calls, no APIs). I have a potentially large data set of todo-lists (one for each date). I have a RecyclerView that displays these todo-lists to the user. Obviously, I don't want to load all the todo-lists for the user so I chose to go with a paged data source and decided to use Paging v3.
I followed this youtube tutorial for my paging implementation.
I got everything working for a basic use case, but now I'm stuck. I want to let the user jump to a specific date in the list. I also want to listen if they scroll to a new date and create a new todo-list for them.
I'm not sure how to scroll my RecyclerView or paged source to a specific point in the list.
My Dao:
#Transaction
#Query("SELECT * FROM todo_list ORDER BY date DESC")
fun loadTodoListsPaged(): PagingSource<Int, TodoList>
My View Model:
class TodoListViewModel(application: Application) : AndroidViewModel(application) {
val todoListsPaged = Pager(PagingConfig(
pageSize = 10,
enablePlaceholders = false,
maxSize = 30
)) {
myTodoListDao.loadTodoListsPaged()
}.flow.cachedIn(viewModelScope)
}
My Adapter:
class TodoListPagingDataAdapter(
val context: Context
): PagingDataAdapter<TodoList, TodoListViewHolder>(TodoListDiffUtilCallback()) {
...
}
I'm not sure if this is possible because Room only supports PagingSource with Key of type Int.
Using Paging v3, how do I jump or scroll to a specific point in the list?
Ideally, I'd just pass something the date and it would tell me the page I need. Then I pass the page to the adapter/recyclerview/pager and it loads it for me.
Update
I just ended up storing a mutable list of dates as LiveData and used a Transformation Switch Map to map that to my todo lists (passing in the list of dates to the database and getting associated lists back). As the user scrolls, I add/remove dates from this mutable list and it automatically updates. It worked a lot better than the paging and was much more simple. If anyone wants more details, just ask.
I don't know if there is a way to jump based on a specific value in the adapted data.
If an answer is posted saying otherwise more power to you.
If you can't find away to use the PagingSource I think I may have a solution.
When the user enters a date programmatically scroll through the list until you find it.
fun scrollTilFindDate(date : Date, offset : Int){
var dateFound : Boolean = false;
var todos: List<Todo> = adapter.snapshot().items;
for(todo : Todo in todos){
if(todo.date == date){
dateFound = true;
break;
}
}
if(!recyclerView.canScrollVertically(1) || dateFound){
return;
}else{
var newOffset : Int = offset+pageSize;
recyclerView.post(()->{
recyclerView.scrollToPosition(newOffset);
scrollTilFindDate(date,newOffset);
});
}
}
Someone asked for more details about my alternative solution. This solution was easy and worked well for my needs. I could jump to any date by changing the ViewModel's current date and it would automatically load other dates around that date (according to the paging config settings).
My RecyclerView only showed 1 item at a time. I used the SnapOnScrollListener's onSnapPositionReadyToSnap to update the ViewModel's current date and trigger the dates to load.
Depending on how you order the dates (ASC vs DESC) and depending on if your RecyclerView has reverseLayout set to true, the loadNextPage / loadPrevPage may be backwards for you. So keep that in mind.
ViewModel
This contains most of the logic & probably could be split up. It does the heavy lifting to facilitate loading items by date.
// NOTE: taskListRepository is just a wrapper for my Room DAOs.
class MyViewModel(application: Application) : AndroidViewModel(application) {
val selectedDateLive = MutableLiveData(LocalDate.now())
var selectedDate get() = selectedDateLive.value
set(value) { selectedDateLive.value = value }
// I used Android's paging config as my own config, but I did NOT use paging itself
private val pageConfigForDates = PagingConfig(
pageSize = 10, // how many to load in each page
prefetchDistance = 3, // how far from the end before we should load more; defaults to page size
initialLoadSize = 10, // how many items should we initially load; defaults to 3x page size
maxSize = 16 // how many items do we keep in memory; defaults to unlimited (must be > pageSize + 2*prefetchDistance)
)
private var datesLoaded: List<LocalDate> = emptyList()
private val datesToLoadLive = Transformations.switchMap(selectedDateLive) { currentDate ->
val indexOfCurrent = datesLoaded.indexOf(currentDate)
// initial dates
if (datesLoaded.isEmpty() || indexOfCurrent == -1) {
val initialDates = getDatesAround(currentDate, pageConfigForDates.initialLoadSize)
datesLoaded = initialDates
ensureListsExistForDates(initialDates)
return#switchMap MutableLiveData(datesLoaded)
}
// previous dates
val shouldLoadPreviousPage = indexOfCurrent < pageConfigForDates.prefetchDistance
if (shouldLoadPreviousPage) {
val pastDates = getPastDates(datesLoaded.first(), pageConfigForDates.pageSize)
datesLoaded = pastDates.plus(datesLoaded).take(pageConfigForDates.maxSize)
ensureListsExistForDates(pastDates)
return#switchMap MutableLiveData(datesLoaded)
}
// next dates
val shouldLoadNextPage = (datesLoaded.count() - indexOfCurrent) < pageConfigForDates.prefetchDistance
if (shouldLoadNextPage) {
val futureDates = getFutureDates(datesLoaded.last(), pageConfigForDates.pageSize)
datesLoaded = datesLoaded.plus(futureDates).takeLast(pageConfigForDates.maxSize)
ensureListsExistForDates(futureDates)
return#switchMap MutableLiveData(datesLoaded)
}
return#switchMap MutableLiveData(datesLoaded)
}
private fun getPastDates(date: LocalDate, numberOfDays: Int): List<LocalDate> {
return (numberOfDays downTo 1L).map { date.minusDays(it) }
}
private fun getDatesAround(date: LocalDate, numberOfDays: Int): List<LocalDate> {
val halfNumberOfDays = numberOfDays / 2
val counter = ((-halfNumberOfDays..0L) + (1L..halfNumberOfDays)).takeLast(numberOfDays)
return counter.map { date.plusDays(it) }.toList()
}
private fun getFutureDates(date: LocalDate, numberOfDays: Int): List<LocalDate> {
return (1L..numberOfDays).map { date.plusDays(it) }
}
private fun ensureListsExistForDates(dates: List<LocalDate>) {
viewModelScope.launch(Dispatchers.IO) {
val items = taskListRepository.getTaskListsWithTasksForDates(dates)
if (items.count() < pageConfigForDates.pageSize) {
val missingDates = dates.minus(items.map { it.taskList.date })
taskListRepository.createTaskLists(missingDates)
}
}
}
private val taskListsFromDatesLive = Transformations.distinctUntilChanged(datesToLoadLive).switchMap { dates ->
taskListRepository.getTaskListsWithTasksForDatesLive(dates!!)
}
}
DAO for accessing DB
#Dao
interface TaskListWithTasksDao {
#Transaction
#Query("SELECT * FROM task_list WHERE date IN (:dates) ORDER BY date DESC")
suspend fun getTaskListWithTasksForDates(dates: List<LocalDate>): List<TaskListWithTasks>
#Transaction
#Query("SELECT * FROM task_list WHERE date IN (:dates) ORDER BY date DESC")
fun getTaskListWithTasksForDatesLive(dates: List<LocalDate>): LiveData<List<TaskListWithTasks>>
}
Data class (for the foreign key relationship)
data class TaskListWithTasks(
#Embedded
var taskList: TaskList = TaskList(),
#Relation(parentColumn = "id", entityColumn = "task_list_id", entity = TaskListItem::class)
var tasks: List<TaskListItem> = emptyList()
)

Why would this query work in the Database Inspector section of the App Inspection in Android Studio, but not within a Room Query?

I have a table within my local SQLite database, the Class for it is as follows:
#Entity(tableName = "table_bp_reading")
class BPReading(
var systolicValue: Int = 120,
var diastolicValue: Int = 80,
var pulseValue: Int = 72,
var timeStamp: String = getDateTimeStamp(),
#PrimaryKey(autoGenerate = true) var pId: Int = 0
) {
...
The relevant part of the Dao looks as such:
...
#Query("SELECT * FROM table_bp_reading WHERE timeStamp BETWEEN :startDate AND :endDate")
fun getReadingsByDateRange(startDate: String, endDate: String): Flow<List<BPReading>>
...
The database inspector displays a table with
the following entries.
Running the following query within the Database Inspector:
SELECT * FROM table_bp_reading WHERE timeStamp BETWEEN '2021-09-18 00:00:00' AND '2021-09-18 23:59:59'
Gives the following result.
However, when I try to observe the data using the following:
bpReadingViewModel.bpReadingsByDate("2021-09-18 00:00:00", "2021-09-18 23:59:59").observe(viewLifecycleOwner, {
bpReading ->
bpReading.let {
bpReadingsByDate = it
if(it.isNotEmpty()) bindDBDataToScatterChart()
}
})
No data is displayed on my end, the query results in an empty list.
Note, however, that if instead of using the full timestamp with time, I just use a date string such as "2021-09-18":
bpReadingViewModel.bpReadingsByDate("2021-09-18", "2021-09-18").observe(viewLifecycleOwner, {
bpReading ->
bpReading.let {
bpReadingsByDate = it
if(it.isNotEmpty()) bindDBDataToScatterChart()
}
})
And I also change the query from the Dao given previously to the following:
#Query("SELECT * FROM table_bp_reading WHERE DATE(timeStamp) BETWEEN :startDate AND :endDate")
fun getReadingsByDateRange(startDate: String, endDate: String): Flow<List<BPReading>>
It works and it returns the 3 entries for that day as it is meant to. However, once I change the date range in the above, to filter between 2021-09-18 and 2021-09-19, it stops working again.
Please help, I am so confused by what is going on here.
EDIT:
Here's the relevant call from Repository:
fun readingsByDateRange(s: String, e: String): Flow<List<BPReading>> =
bpReadingDao.getReadingsByDateRange(e, s)
And the relevant call from the ViewModel for the class:
fun bpReadingsByDate(s: String, e: String): LiveData<List<BPReading>> =
repository.readingsByDateRange(s, e).asLiveData()
Which is why it is indeed bpReadingsByDate, and not getReadingsByDateRane. Sorry, this is my mistake for not being consistent with naming the function across files.
The query is fine (tested), from your comments and edited question, you have swapped the start and end timestamps which will normally result in nothing being selected.
So you should change bpReadingDao.getReadingsByDateRange(e, s)to be bpReadingDao.getReadingsByDateRange(s, e).

Access Room DB inside Transformation.map android

In my Database i have a table called Account which looks kinda like this
#Entity(tableName = "accounts", primaryKeys = ["server_id", "account_id"])
data class Account(
#ColumnInfo(name = "server_id")
val serverId: Long,
#ColumnInfo(name = "account_id")
val accountId: Int,
#ColumnInfo(name = "first_name", defaultValue = "")
var firstname: String
)
So lets say that we have the following Database snapshot
server_id account_id first_name
1 10 Zak
1 11 Tom
1 12 Bob
1 13 Jim
1 14 Mike
Now i also have the following POJO which represents an available video room inside a chatRoom
data class RoomInfo(
#SerializedName("m")
val participantIntList: List<Int>,
#SerializedName("o")
val roomId: String,
#SerializedName("s")
val status: Int
)
So i get an incoming response from my Socket which is like the following
[
{"m": [10, 11, 12], "o": "room_technical", "s": 1},
{"m": [13, 14], "o": "room_operation", "s": 1}
]
which i map it in a List so i have
val roomInfo: LiveData<List<RoomInfo>> = socketManager.roomInfo
// So the value is basically the json converted to a list of RoomInfos using Gson
In order to display this available list of Rooms to the User i need to convert the m (which is the members that are inside the room right now) from accountIds to account.firstnames.
So what i want to have finally is a List of a new object called RoomInfoItem which will hold the list of the rooms with the accountIds converted to firstNames from the Account table of the Database.
data class RoomInfoItem(
val roomInfo: RoomInfo,
val participantNames: List<String>
)
So if we make the transformation we need to have the following result
RoomInfo (
// RoomInfo
{"m": [10, 11, 12], "o": "room_technical", "s": 1},
// Participant names
["Zak", "Tom", "Bob"]
)
RoomInfo (
// RoomInfo
{"m": [13, 14], "o": "room_operation", "s": 1},
// Participant names
["Jim", "Mike"]
)
My Activity needs to observe a LiveData with the RoomInfoItems so what i want is given the LiveData<List> to transform it to LiveData<List>. How can i do that?
Well, finally i could not find a solution but i think that what i am trying to achieve, cannot be done using the Transformation.switchMap or Transformation.map
As I understand you want get LiveData<List<RoomInfoItem>> by analogy LiveData<List<ResultData>> in my sample. And you have next condition: you want to observe list of RoomInfo and for each RoomInfo in this list you want to observe participantNames. (Each pair of RoomInfo and participantNames you map to RoomInfoItem). I think you can achive this behaviour by using MediatorLiveData. I show sample how you can do this bellow:
// For example we have method which returns liveData of List<String> - analogy to your List<RoomInfo>
fun firstLifeData(): LiveData<List<String>> {
//TODO
}
// and we have method which returns liveData of List<Int> - analogy to your participantNames(List<String>)
fun secondLifeData(param: String): LiveData<List<Int>> {
//TODO
}
//and analogy of your RoomInfoItem
data class ResultData(
val param: String,
val additionalData: List<Int>
)
Then I will show my idea of implementation of combined liveDatas:
#MainThread
fun <T> combinedLiveData(liveDatas: List<LiveData<T>>): LiveData<List<T>> {
val mediatorLiveData = MediatorLiveData<List<T>>()
// cache for values which emit each liveData, where key is an index of liveData from input [liveDatas] list
val liveDataIndexToValue: MutableMap<Int, T> = HashMap()
// when [countOfNotEmittedLifeDatas] is 0 then each liveData from [liveDatas] emited value
var countOfNotEmittedLifeDatas = liveDatas.size
liveDatas.forEachIndexed { index, liveData ->
mediatorLiveData.addSource(liveData) { value ->
// when liveData emits first value then mack it by decrementing of countOfNotEmittedLifeDatas
if (!liveDataIndexToValue.containsKey(index)) {
countOfNotEmittedLifeDatas--
}
liveDataIndexToValue[index] = value
// when countOfNotEmittedLifeDatas is 0 then all liveDatas emits at least one value
if (countOfNotEmittedLifeDatas == 0) {
// then we can push list of values next to client-side observer
mediatorLiveData.value = liveDataIndexToValue.toListWithoutSavingOrder()
}
}
}
return mediatorLiveData
}
fun <V> Map<Int, V>.toListWithoutSavingOrder(): List<V> = this.values.toList()
/**
* Key should be an order
*/
fun <V> Map<Int, V>.toListWithSavingOrder(): List<V> = this.entries.sortedBy { it.key }.map { it.value }
/*
or you can run [for] cycle by liveDataIndexToValue in [combinedLiveData] method or apply [mapIndexed] like:
liveDatas.mapIndexed{ index, _ ->
liveDataIndexToValue[index]
}
to receive ordered list.
*/
And how to use all of that together:
fun resultSample(): LiveData<List<ResultData>> {
return firstLifeData().switchMap { listOfParams ->
val liveDatas = listOfParams.map { param -> secondLifeData(param).map { ResultData(param, it) } }
combinedLiveData(liveDatas)
}
}
// u can add extension function like:
fun <T> List<LiveData<T>>.combined(): LiveData<List<T>> = combinedLiveData(this)
// and then use it in this way
fun resultSample_2(): LiveData<List<ResultData>> = firstLifeData().switchMap { listOfParams ->
listOfParams.map { param -> secondLifeData(param).map { ResultData(param, it) } }.combined()
}
I suggest you to consider using room's Relations. I think by room's Relations you can get LiveData<RoomInfoItem> . I cant get you more details about this approach because I don't know details about your data scheme and domain, at the moment.

Room Dao Returns Insert IDs but Data is Missing from Database

When saving a list of objects in my room database using a Dao
#Insert()
fun saveCharmRankMaterialCosts(materialCosts: List<CharmRankCraftingCost>) : List<Long>
And this is used from my repository class to save results from an API call:
val charmRankCosts = CharmRankCraftingCost.fromJsonCraftingCost(
charmRankId.toInt(),
jsonCharmRank.crafting
)
// save crafting/upgrade costs for the rank
val results = charmDao.saveCharmRankMaterialCosts(charmRankCosts)
Log.d("CharmRepository", "Saved charm material costs: ${results.toString()}");
assert(!results.contains(-1))
When running this code, insert ID's are returned and the assertion is never triggered (i.e. no inserts fail).
But when I inspect the data base on the device, most of the supposedly inserted IDs are missing from the table. I'm very confused as to what is going on here. I've debugged this issue for many hours and have been unsuccessful in getting this to work. Is there something obvious I'm missing?
The issue seems to have been related to foreign key constraints. I had a CharmRank data class with multiple related data objects. See below:
/**
* Copyright Paul, 2020
* Part of the MHW Database project.
*
* Licensed under the MIT License
*/
#Entity(tableName = "charm_ranks")
data class CharmRank(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "charm_rank_id")
var id: Int = 0,
#ColumnInfo(name = "charm_id")
var charmId : Int,
#ColumnInfo(name = "charm_rank_level")
var level: Int = 0, // 3
#ColumnInfo(name = "charm_rank_rarity")
var rarity: Int = 0, // 6
#ColumnInfo(name = "charm_rank_name")
var name: String = "",
#ColumnInfo(name = "craftable")
var craftable: Boolean
)
Each charm rank has associated skills and items to craft said rank. These objects are simply relational objects in that they hold the ID of the CharmRank and a SkillRank in the case of the skills object, or the ID of the CharmRank and the ID of the Item object.
data class CharmRankSkill(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "charm_rank_skill_id")
var id: Int,
var charmRankId : Int,
var skillRankId: Int
)
data class CharmRankCraftingCost(
#PrimaryKey(autoGenerate = true)
#ColumnInfo(name = "charm_rank_crafting_cost_id")
var id: Int,
#ColumnInfo(name = "charm_rank_id")
var charmRankId: Int,
#ColumnInfo(name = "charm_rank_crafting_cost_item_quantity")
val quantity: Int,
val itemId: Int
)
Originally in CharmRankCraftingCost, I had a foreign key constraint on the Item object and the CharmRank object. Below is the foreign key constraint on the Item object:
ForeignKey(
entity = Item::class,
parentColumns = ["item_id"],
childColumns = ["itemId"],
onDelete = ForeignKey.CASCADE
)
The Item data object has IDs provided by the remote data source, so when I insert items into it's respective table, the conflict resolution is set to Replace. During the process of saving the relational items to the data base for the CharmRanks, I also have to save the Item objects prior to saving CharmRankCraftingCosts. It seems that what was happening is that when the Item objects are inserted, sometimes the items would get replaced, which would trigger the cascade action of the foreign key resulting in the CharmRankCraftingCosts items I just saved for the CharmRank to be deleted due to the cascading effect.
Removing the foreign key constraint on the Item table solved my issue.
As I understood from the comments, you make a delete before inserts. The problem is that it happens that the insert gets completed before the delete since you do them in separate threads. What you need is to do both in one transaction. Create a method in the the DAO class with #Transaction annotation (Make sure your dao is an abstract class so you can implement the body of this method):
#Dao
public abstract class YourDao{
#Insert(onConflict = OnConflictStrategy.IGNORE)
public abstract List<Long> insertData(List<Data> list);
#Query("DELETE FROM your_table")
public abstract void deleteData();
#Transaction
public void insertAndDeleteInTransaction(List<Data> list) {
// Anything inside this method runs in a single transaction.
deleteData();
insertData(list);
}
}
Read this for Kotlin Version of the code.

Categories

Resources