I'm trying to get all my posts which time is newer ones first, I have made this query that should bring to me all the newest posts that I uploaded or other people did.
Thing is that I'm getting posts shuffled with timestamps , there are all mixed up instead of ordered by new ones
If I do the same query from the firebase console they are ordered the way it should
My recyclerview does NOT have any reverselayout or stackfromend attributes and I'm not expecting to use them, instead I just want my list to come from firebase ordered
#ExperimentalCoroutinesApi
suspend fun getLatestPosts(): Flow<Result<List<Post>>> = callbackFlow {
val postList = mutableListOf<Post>()
// Reference to use in Firestore
var eventsCollection: CollectionReference? = null
try {
eventsCollection = firestore.collection("posts")
eventsCollection.orderBy("created_at", Query.Direction.DESCENDING)
} catch (e: Throwable) {
// If Firebase cannot be initialized, close the stream of data
// flow consumers will stop collecting and the coroutine will resume
close(e)
}
val suscription = eventsCollection?.addSnapshotListener { value, error ->
if (value == null) {
return#addSnapshotListener
}
try {
postList.clear()
for (post in value.documents) {
post.toObject(Post::class.java)?.let { fbPost ->
fbPost.apply {
created_at = post.getTimestamp(
"created_at",
DocumentSnapshot.ServerTimestampBehavior.ESTIMATE
)?.toDate()
}
postList.add(fbPost)
}
}
offer(Result.Success(postList))
} catch (e: Exception) {
close(e)
}
}
awaitClose { suscription?.remove() }
}
Now if I sort the list locally after getting the data that works, but I don't want it to be client side, I want to have an ordered list from the server.
What I'm doing wrong ?
Posts timestamp are saved with #ServerTimestamp and Date format into Firestore
CollectionReference (Query) objects are immutable - they can't be changed once created. They use a builder type pattern to construct new Query objects by adding constraints. The original Query remains unmodified.
If you want to compose a set of operations to perform on a Query, you would have to remember the new Query returned by each operation. An easy way to do this is by reassigning the prior Query if you no longer need it:
var eventsQuery: Query = firestore.collection("posts")
eventsQuery = eventsQuery.orderBy("created_at", Query.Direction.DESCENDING)
Or you can do it as a chained sequence of operations, which is more idiomatic in the case that you know all the operations ahead of time:
val eventsQuery = firestore
.collection("posts")
.orderBy("created_at", Query.Direction.DESCENDING)
If you need to conditionally apply operations, however, you will need to take the first approach. A more detailed explanation can be found here: Firestore: Multiple conditional where clauses
Problem was at this line
eventsCollection = firestore.collection("posts")
eventsCollection.orderBy("created_at", Query.Direction.DESCENDING)
Seems like I need to wrap it up all into 1 line and make it a Query, it does not apply any changes to the eventsCollection, this can be done changing CollectionReference to Query
Related
I am writing code to get the newsfeed from the database and to show this feed in UI. This is my Firestore database structure:
users->uniqueUserId |-->UsersProfileInfo--->Profile(document)
|-->FeedNewsFeed |--->unique documents for each newsfeed
|--->unique documents for each newsfeed
|--->unique documents for each newsfeed
to get each news feed from every user in my user collection I have to write nested for loops which takes some extra time and getting data from Firestore also takes some times, so is there any nice way to optimize this problem.
Function for getting newsfeed
suspend fun getAllNewsFeeds(): ArrayList<NewsFeedClass> {
list.clear()
val querySnapshot = collectionRef.get().await()
//Traversing through each document in collection
for (document in querySnapshot) {
val currDocRef = document.reference
//Getting user name
val userName =
currDocRef
.collection(Constants.UserProfileInfo)
.document(Constants.Profile)
.get()
.await()
.toObject(FeedUserName::class.java)?.userName
// adding username in newsFeedClass for displaying on newsfeed
val newsFeedClass = NewsFeedClass()
if (userName != null) {
newsFeedClass.username = userName
}
//getting QuerySnapshot from FeedNewsFeed collection
val newsFeedQuerySnapshot =
currDocRef
.collection(Constants.FeedNewsFeeds).get().await()
//Traversing through each document in
// collecting and respectively adding feed on newsFeedClass
//adding newsFeedClass to list
for (documentOfFeed in newsFeedQuerySnapshot) {
val thisDocRef = documentOfFeed.reference
val feed =
thisDocRef
.get().await().toObject<FeedNewsFeed>()
if (feed != null) {
newsFeedClass.content = feed.newsfeed
list.add(NewsFeedClass(newsFeedClass.username, newsFeedClass.content))
}
}
}
list.shuffle()
return list
}
What you're experiencing in your code is the expected behavior. Why? Because at every iteration of your loop, you are reading data from Firestore using get() and right after that you call await(). This means that all the operations run one after another. It basically means, that at each iteration of your loop, you wait until you get the data from the database. The more iterations you have, the longer it will take. So these operations run sequentially, and not in parallel as you probably might think.
If you need to get the data in parallel you can add the kotlinx-coroutines-play-services library to your project and use the asDeferred extension function that converts a Task into a Deferred object. In the end, you can call the awaitAll() extension function that will wait while all Firestore read operations are loaded in parallel. So in code, it should look like this:
val tasks: MutableList<Deferred<DocumentSnapshot>> = mutableListOf()
for (document in querySnapshot) {
val currDocRef = document.reference
val deferredTask = currDocRef
.collection(Constants.UserProfileInfo)
.document(Constants.Profile)
.get()
.asDeferred()
tasks.add(deferredTask)
}
tasks.awaitAll().forEach { document ->
//Do what you need to do with your documents.
}
My use case is as follows:
Imagine that there is an Android Fragment that allows users to search for Grocery items in a store. There's a Search View, and as they type, new queries are sent to the Grocery item network service to ask for which items match the query. When successful, the query returns a list of Grocery items that includes the name, price, and nutritional information about the product.
Locally on the Android device, there is a list of known for "items for sale" stored in a raw file. It's in the raw resources directory and is simply a list of grocery item names and nothing else.
The behavior we wish to achieve is that as the user searches for items, they are presented with a list of items matching their query and a visual badge on the items that are "For Sale"
The constraints I am trying to satisfy are the following:
When the user loads the Android Fragment, I want to parse the raw text file asynchronously using a Kotlin coroutine using the IO Dispatcher. Once parsed, the items are inserted into the Room database table for "For Sale Items" which is just a list of names where the name is the primary key. This list could be empty, it could be large (i.e. >10,0000).
Parallel, and independent of #1, as the user types and makes different queries, I want to be sending out network requests to the server to retrieve the Grocery Items that match their query. When the query comes back successfully, these items are inserted into a different table in the Room database for Grocery Items
Finally, I only want to render the list returned from #2 once I know that the text file from #1 has been successfully parsed. Once I know that #1 has been successfully parsed I want to join the tables in the database on name and give that LiveData to my ViewModel to render the list. If either #1 or #2 fail, I want the user to be given an "Error occurred, Retry" button
Where I am struggling right now:
Seems achievable by simply kicking off a coroutine in ViewModel init that uses the IO Dispatcher. This way I only attempt to parse the file once per ViewModel creation (I'm okay with reparsing it if the user kills and reopens the app)
Seems achievable by using another IO Dispatcher coroutine + Retrofit + Room.
Satisfying the "Only give data to ViewModel when both #1 and #2 are complete else show error button" is the tricky part here. How do I expose a LiveData/Flow/something else? from my Repository that satisfies these constraints?
When you launch coroutines, they return a Job object that you can wait for in another coroutine. So you can launch a Job for 1, and 3 can await it before starting its flow that joins tables.
When working with Retrofit and Room, you can define your Room and Retrofit DAOs/interfaces with suspend functions. This causes them to generate implementations that internally use an appropriate thread and suspend (don't return) until the work of inserting/updating/fetching is complete. This means you know that when your coroutine is finished, the data has been written to the database. It also means it doesn't matter which dispatcher you use for 2, because you won't be calling any blocking functions.
For 1, if parsing is a heavy operation, Dispatchers.Default is more appropriate than Dispatchers.IO, because the work will truly be tying up a CPU core.
If you want to be able to see if the Job from 1 had an error, then you actually need to use async instead of launch so any thrown exception is rethrown when you wait for it in a coroutine.
3 can be a Flow from Room (so you'd define the query with the join in your DAO), but you can wrap it in a flow builder that awaits 1. It can return a Result, which contains data or an error, so the UI can show an error state.
2 can operate independently, simply writing to the Room database by having user input call a ViewModel function to do that. The repository flow used by 3 will automatically pick up changes when the database changes.
Here's an example of ViewModel code to achieve this task.
private val parsedTextJob = viewModelScope.async(Dispatchers.Default) {
// read file, parse it and write to a database table
}
val theRenderableList: SharedFlow<Result<List<SomeDataType>>> = flow {
try {
parsedTextJob.await()
} catch (e: Exception) {
emit(Result.failure(e)
return#flow
}
emitAll(
repository.getTheJoinedTableFlowFromDao()
.map { Result.success(it) }
)
}.shareIn(viewModelScope, SharingStarted.WhileSubscribed(5000), replay = 1)
fun onNewUserInput(someTextFromUser: String) {
viewModelScope.launch {
// Do query from Retrofit.
// Parse results and write to database.
}
}
If you prefer LiveData to SharedFlow, you can replace theRenderableList above with:
val theRenderableList: LiveData<Result<List<SomeDataType>>> = liveData {
try {
parsedTextJob.await()
} catch (e: Exception) {
emit(Result.failure(e)
return#liveData
}
emitSource(
repository.getTheJoinedTableFlowFromDao()
.map { Result.success(it) }
.asLiveData()
)
}
You could do this by having the ViewModel monitor when the two tasks are complete and set loading state LiveData variable to indicate that the UI should only update once both tasks are complete. For example:
class MainViewModel : ViewModel() {
private var completedA = false
private var completedB = false
private val dataALiveData = MutableLiveData("")
val dataA: LiveData<String>
get() = dataALiveData
private val dataBLiveData = MutableLiveData("")
val dataB: LiveData<String>
get() = dataBLiveData
private val dataIsReadyLiveData = MutableLiveData(false)
val dataIsReady: LiveData<Boolean>
get() = dataIsReadyLiveData
// You can trigger a reload of some of this data without having to reset
// any flags - the UI will be updated when the task is complete
fun reloadB() {
viewModelScope.launch { doTaskB() }
}
private suspend fun doTaskA() {
// Fake task A - once it's done post relevant data
// (if applicable), indicate that it is completed, and
// check if the app is ready
delay(3200)
dataALiveData.postValue("Data A")
completedA = true
checkForLoaded()
}
private suspend fun doTaskB() {
// Fake task B - once it's done post relevant data
// (if applicable), indicate that it is completed, and
// check if the app is ready
delay(2100)
dataBLiveData.postValue("Data B")
completedB = true
checkForLoaded()
}
private fun checkForLoaded() {
if( completedA && completedB ) {
dataIsReadyLiveData.postValue(true)
}
}
// Launch both coroutines upon creation to start loading
// the two data streams
init {
viewModelScope.launch { doTaskA() }
viewModelScope.launch { doTaskB() }
}
}
The activity or fragment could observe these three sets of LiveData to determine what to show and when, for example to hide the displayed elements and show a progress bar or loading indicator until it is done loading both.
If you wanted to handle error states, you could have the dataIsReady LiveData hold an enum or string to indicate "Loading", "Loaded", or "Error".
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = ActivityMainBinding.inflate(layoutInflater)
setContentView(binding.root)
val model: MainViewModel by viewModels()
binding.textA.visibility = View.INVISIBLE
binding.textB.visibility = View.INVISIBLE
binding.progressBar.visibility = View.VISIBLE
model.dataA.observe(this) { data ->
binding.textA.text = data
}
model.dataB.observe(this) { data ->
binding.textB.text = data
}
// Once the data is ready - change the view visibility state
model.dataIsReady.observe(this) { isReady ->
if( isReady ) {
binding.textA.visibility = View.VISIBLE
binding.textB.visibility = View.VISIBLE
binding.progressBar.visibility = View.INVISIBLE
// alternately you could read the data to display here
// by calling methods on the ViewModel directly instead of
// having separate observers for them
}
}
}
I am performing this query to get posts of user whom the current user follows [like in social media]. but this query is taking hell lot of a time.As you can see I also convert them to post entity and set their corresponding properties for each , I think that is the main culprit. How can i optimise it?
i have different current collection for both users and posts in firestore
suspend fun getPostsForProfile(uid: String) = withContext(Dispatchers.IO) {
safeCall {
Log.d(" basePostRepository ", " getPostsForProfile is called ")
val Currentuid = FirebaseAuth.getInstance().uid!!
// get Posts where authorUid is equal to uid
val profilePosts = posts.whereEqualTo("authorUid", uid)
.orderBy("date", Query.Direction.DESCENDING)
.get()
.await()
.toObjects(Post::class.java)
.onEach { post ->
Log.d(" basepostRepository : getPostforProfile ", post.authorUid)
val user = getUser(post.authorUid).data!!
post.authorUsername = user.type
val isLiked_init = post.likedBy.find { item -> item == Currentuid}
post.isLiked = when (isLiked_init) {
null -> false
else -> true
}
}
Resource.Success(profilePosts)
}
}
suspend fun getUser(uid: String) = withContext(Dispatchers.IO) {
safeCall {
val currentUid = FirebaseAuth.getInstance().uid!!
val user = users.document(uid).get().await().toObject(User::class.java)
?: throw IllegalStateException()
val currentUser = users.document(currentUid).get().await().toObject(User::class.java)
?: throw IllegalStateException()
val isfollowed_init = currentUser.follows.find { item -> item == uid}
user.isfollowing = when (isfollowed_init) {
null -> false
else -> true
}
Resource.Success(user)
}
}
There are multiple things you can do:
1 - when getting the data use the firestore limit to get onl a specific amount of posts. Idealy a little bit more than the user can se on a usual user device scrren size. Probably 15 to 20 and load more if the users scrolls. This is a good aproach for most queries where you know that there is a huge amount of data in it.
2 - save the author data to the post to avoid getting it by a separate call (even if it is a single user wou would make the same call again and again) also save the athor uid. That way you can query by the author uid the current user is following and awoid getting all posts and filtering them on the client side.
3 - don't call the users the current user is following each time you try to get the post author data/user. Save the followers in a collection and call them once with a realtime listener in your app.
I'm pretty new in the world of MVI pattern. So I'm trying to understand how fit together all the pieces.
I have an app that I structured using MVI pattern (or at least it was what I was meant to do). I have my fragment (I used navigation component but at the moment focus just on one fragment), which is supported by its own ViewModel. Then I have a repository class where all viewmodels retrieve data. Repository has 2 source of data, a web API and a local DB used as cache of data, I used Room for DB management.
I tried different approaches to the problem. At the moment I have done in this way:
In the DAO I used this instruction to retrieve data from the DB:
#Query("SELECT * FROM Users WHERE idTool=:idTool AND nickname LIKE '%' || :query || '%'")
fun users(idTool: Int, query: String) : Flow<List<User>>
Then in my repository I simple get this query to forward to ViewModels:
fun usersFlow(idTool: Int, query: String) = userDao.users(idTool, query)
In the ViewModel I created two MutableLiveData, coordinated by a MediatorLiveData:
val nicknameQuery = MutableStateFlow("")
private val nicknameQueryFlow = nicknameQuery.flatMapLatest {
repository.usersFlow(idToolQuery.value, it)
}
val idToolQuery = MutableStateFlow(DEFAULT_TOOL_ID)
private val idToolQueryFlow = idToolQuery.flatMapLatest {
repository.usersFlow(it, nicknameQuery.value)
}
val users = MediatorLiveData<List<User>>()
init {
users.addSource(nicknameQueryFlow.asLiveData()) {
users.value = it
}
users.addSource(idToolQueryFlow.asLiveData()) {
users.value = it
}
fetchUsers()
}
In this way, from my fragment, I can simply update nicknameQuery or idToolQuery to have an updated list in my RecyclerView. My first doubt is that in this way the fetch of data from my DB is done 2 times, one time for each mutable, but I'd like to retrieve data just one on the app opening (maybe the solution fro this is just check in the nicknameQuery that current query is different from the passed one, in this way since at the beginning current query is empty and it pass an empty query, it is bypassed).
In the Init method of ViewModel, I also call fetchUsers():
private fun fetchUsers() {
viewModelScope.launch {
repository.fetchUsers(DEFAULT_TOOL_ID).collect {
_dataState.value = it
}
}
}
This method checks into the database if there are already cached users with this specific idTool, if not it fetches them from the web and it stores retrieved data into the DB. This is the method inside my repository class:
suspend fun fetchUsers(
idTool: Int,
forceRefetch: Boolean = false
): Flow<DataState<List<User>>> = flow {
try {
var cachedUser = userDao.users(idTool, "").first()
val users: List<User>
if(cachedUser.isEmpty() || forceRefetch) {
Log.d(TAG, "Retrieve users: from web")
emit(DataState.Loading)
withContext(Dispatchers.IO) {
appJustOpen = false
val networkUsers =
api.getUsers(
idTool,
"Bearer ${sessionClient.tokens.accessToken.toString()}"
)
users = entityMapper.mapFromEntitiesList(networkUsers)
userDao.insertList(users)
}
} else {
users = cachedUser
}
emit(DataState.Success(users))
} catch (ex: Exception) {
emit(DataState.Error(ex))
}
}
This method checks if I have already users inside the DB with this specific idTool, if not it fetches them from API. It uses a DataState to update the UI, based on the result of the call. During the fetch of data, it emits a Loading state, this shows a progress bar in my fragment. If data is correctly fetched it emits a Success, and the fragment hides the progress bar to shows the recycler view. This is done in the following way. In my ViewModel I have this mutable state
private val _dataState = MutableLiveData<DataState<List<User>>>()
val dataState: LiveData<DataState<List<User>>> get() = _dataState
As you saw above, my fetch method is
private fun fetchUsers() {
viewModelScope.launch {
repository.fetchUsers(DEFAULT_TOOL_ID).collect {
_dataState.value = it
}
}
}
And finally in my fragment I have:
userListViewModel.dataState.observe(viewLifecycleOwner, { dataState ->
when (dataState) {
is DataState.Success -> {
showUserList()
}
is DataState.Error -> {
Log.e("TEST", dataState.exception.toString())
hideLoader()
Toast.makeText(activity, "Error retrieving data: ${dataState.exception}", Toast.LENGTH_LONG).show()
}
is DataState.Loading -> {
showLoader()
}
else -> {
// Do Nothing in any other case
}
}
})
At this moment Success state takes a list of users, but this list is there from a previous approach, at the moment it is useless since after data is fetched list is inserted into the DB, and I have a Flow to the DB which takes care to update the UI. In this way when I change idTool, when I change query, when I remove a user, the view is always notified
Is this approach correct?
Before this, I used another approach. I returned not a flow from my DB but just a List. Then my fetchUsers always returned a DataState<List>, it checked in the DB and if didn't found anything it fetched data from the web and returned that list. This approach caused me some problems, since every time I changed idTool or query, I always had to call fetchUsers method. Even if a user was removed from database, views didn't get notified since I didn't have a direct flow with the DB.
my app architecture, quite common:
Please explain me if I have list of Entities, for example
#Entity(tableName = TABLE_NAME)
class Item constructor(
#PrimaryKey(autoGenerate = false)
var id: Long = 0L,
#TypeConverters(ListToStringConverter::class)
var eventDescription: List<String> = emptyList(),
#TypeConverters(DateConverter::class)
var date: Date = Date(),
var rate: Int ?= null)
Picture explanation:
Currently I do (according to picture above):
mLiveData getLiveData from Repository
callbackrefreshFromDataBase()
mLiveData.addSource from LiveData of DataBase - what causes that Android View is updated quickly
callback refreshFromNetwork()
Rest updates DatabaseTable
Database insert causes that LiveData add pushes changes to the View
Formulation of the problem
What is the best practice for 5 step - a moment when new data comes and I have to replace old data with the freshest?
Currently I'm using RxJava, Room, Kotlin and I'm using in step 3 nested Rx.Single which is very ugly spagetti code.
like
disposable = usecase.getDataFromNetwork()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy(onSuccess = {
val itemsList = it
dataBase.deleteAllItems()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy (onComplete = { dataBase.insertNewItems(itemsList)
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribeBy(onComplete = {
// Process complete
}, onError = {
...
})
}
}, onError = {
...
})
}, onError = {
...
})
Ugly spagetti code.
QUESTIONS
Should I wipe all Table before inserting new Items? Livedata listen to the changes of List, so whenever anything changes (some item updated, some inserted, some removed) view will be updated
Under Circumstances where all my Streams are Single (reason: they dispose by itself when onComplete), how to chain them sequentially in oder to restrict one stream rule?
This depends if your fetched data will be similar or not. If you fresh data is normally completly different I would delete all elements and then insert new elements with only one method like following:
#Insert
public void insertItems(Item... items);
If your new data is similar and you don't care about their order I would compare all new items to the old ones to know for each item if you have to update, insert or delete item. At the end you call deleteItems(itemsToDelete), updateItems(itemsToUpdate) and insertItems(itemsToInsert). This three methods are similar as the one written above but with #Delete, #Update and #Insert.
If you care about the order I recommend you take a look at Android Paging Library and create a custom DataSource where you can tell to your live data when to update its values. Some links to start: this, this, this and this