I am performing this query to get posts of user whom the current user follows [like in social media]. but this query is taking hell lot of a time.As you can see I also convert them to post entity and set their corresponding properties for each , I think that is the main culprit. How can i optimise it?
i have different current collection for both users and posts in firestore
suspend fun getPostsForProfile(uid: String) = withContext(Dispatchers.IO) {
safeCall {
Log.d(" basePostRepository ", " getPostsForProfile is called ")
val Currentuid = FirebaseAuth.getInstance().uid!!
// get Posts where authorUid is equal to uid
val profilePosts = posts.whereEqualTo("authorUid", uid)
.orderBy("date", Query.Direction.DESCENDING)
.get()
.await()
.toObjects(Post::class.java)
.onEach { post ->
Log.d(" basepostRepository : getPostforProfile ", post.authorUid)
val user = getUser(post.authorUid).data!!
post.authorUsername = user.type
val isLiked_init = post.likedBy.find { item -> item == Currentuid}
post.isLiked = when (isLiked_init) {
null -> false
else -> true
}
}
Resource.Success(profilePosts)
}
}
suspend fun getUser(uid: String) = withContext(Dispatchers.IO) {
safeCall {
val currentUid = FirebaseAuth.getInstance().uid!!
val user = users.document(uid).get().await().toObject(User::class.java)
?: throw IllegalStateException()
val currentUser = users.document(currentUid).get().await().toObject(User::class.java)
?: throw IllegalStateException()
val isfollowed_init = currentUser.follows.find { item -> item == uid}
user.isfollowing = when (isfollowed_init) {
null -> false
else -> true
}
Resource.Success(user)
}
}
There are multiple things you can do:
1 - when getting the data use the firestore limit to get onl a specific amount of posts. Idealy a little bit more than the user can se on a usual user device scrren size. Probably 15 to 20 and load more if the users scrolls. This is a good aproach for most queries where you know that there is a huge amount of data in it.
2 - save the author data to the post to avoid getting it by a separate call (even if it is a single user wou would make the same call again and again) also save the athor uid. That way you can query by the author uid the current user is following and awoid getting all posts and filtering them on the client side.
3 - don't call the users the current user is following each time you try to get the post author data/user. Save the followers in a collection and call them once with a realtime listener in your app.
Related
I am writing code to get the newsfeed from the database and to show this feed in UI. This is my Firestore database structure:
users->uniqueUserId |-->UsersProfileInfo--->Profile(document)
|-->FeedNewsFeed |--->unique documents for each newsfeed
|--->unique documents for each newsfeed
|--->unique documents for each newsfeed
to get each news feed from every user in my user collection I have to write nested for loops which takes some extra time and getting data from Firestore also takes some times, so is there any nice way to optimize this problem.
Function for getting newsfeed
suspend fun getAllNewsFeeds(): ArrayList<NewsFeedClass> {
list.clear()
val querySnapshot = collectionRef.get().await()
//Traversing through each document in collection
for (document in querySnapshot) {
val currDocRef = document.reference
//Getting user name
val userName =
currDocRef
.collection(Constants.UserProfileInfo)
.document(Constants.Profile)
.get()
.await()
.toObject(FeedUserName::class.java)?.userName
// adding username in newsFeedClass for displaying on newsfeed
val newsFeedClass = NewsFeedClass()
if (userName != null) {
newsFeedClass.username = userName
}
//getting QuerySnapshot from FeedNewsFeed collection
val newsFeedQuerySnapshot =
currDocRef
.collection(Constants.FeedNewsFeeds).get().await()
//Traversing through each document in
// collecting and respectively adding feed on newsFeedClass
//adding newsFeedClass to list
for (documentOfFeed in newsFeedQuerySnapshot) {
val thisDocRef = documentOfFeed.reference
val feed =
thisDocRef
.get().await().toObject<FeedNewsFeed>()
if (feed != null) {
newsFeedClass.content = feed.newsfeed
list.add(NewsFeedClass(newsFeedClass.username, newsFeedClass.content))
}
}
}
list.shuffle()
return list
}
What you're experiencing in your code is the expected behavior. Why? Because at every iteration of your loop, you are reading data from Firestore using get() and right after that you call await(). This means that all the operations run one after another. It basically means, that at each iteration of your loop, you wait until you get the data from the database. The more iterations you have, the longer it will take. So these operations run sequentially, and not in parallel as you probably might think.
If you need to get the data in parallel you can add the kotlinx-coroutines-play-services library to your project and use the asDeferred extension function that converts a Task into a Deferred object. In the end, you can call the awaitAll() extension function that will wait while all Firestore read operations are loaded in parallel. So in code, it should look like this:
val tasks: MutableList<Deferred<DocumentSnapshot>> = mutableListOf()
for (document in querySnapshot) {
val currDocRef = document.reference
val deferredTask = currDocRef
.collection(Constants.UserProfileInfo)
.document(Constants.Profile)
.get()
.asDeferred()
tasks.add(deferredTask)
}
tasks.awaitAll().forEach { document ->
//Do what you need to do with your documents.
}
The main feature of the application is as follows
Allow the user to find whether any of his friend has read a particular book or not.
Allow the user to find if any of his friend's friend has read a particular book or not e.g. if person A has a friend B who happens to have a friend C who has read the book that person A is interested in we should be able to make a link between the two.
I am using Kotlin and Firestore.
My database structure is as follows
There is a user collection that contains all the details about the user and array list of the IDs of all the books that he has read.
There is a friends collection that contains the root user ID and all his friends user IDs.
data class friends(
val userID: String = "",
val friendsList: ArrayList = ArrayList()
)
The function to find the desired link between users based on the books they have read is as follows.
fun findLink(mBookID: String) {
mFireStore.collection(Constants.FRIENDS)
.whereEqualTo(Constants.USER_ID, getCurrentUserID())
.get()
.addOnSuccessListener { document ->
for (i in document.documents) {
val rootUser = i.toObject(friends::class.java)!!
val rootUserFriendList = rootUser.friendsList
for (j in rootUserFriendList) {
mFireStore.collection(Constants.FRIENDS)
.whereEqualTo(Constants.USER_ID, j)
.get()
.addOnSuccessListener { document ->
for (k in document.documents) {
val branchUser = k.toObject(friends::class.java)!!
val branchUserArray = branchUser.friendsList
for (l in branchUserArray) {
if (userReadBook(l)) {
val linkArray: ArrayList<String> = ArrayList()
linkArray.add(l)
linkArray.add(j)
}
}
}
}
}
}
}
.addOnFailureListener { e ->
Log.e(
activity.javaClass.simpleName,
"Error while finding a link.",
e
)
}
}
The idea is to make an array of list of users leading to the actual person who has read the book. As there could be several search links I need a method to store all those arrays.
Any help in either achieving the given task or a lead to resources that could help me in achieving the task would be greatly appreciated.
I'm trying to get all my posts which time is newer ones first, I have made this query that should bring to me all the newest posts that I uploaded or other people did.
Thing is that I'm getting posts shuffled with timestamps , there are all mixed up instead of ordered by new ones
If I do the same query from the firebase console they are ordered the way it should
My recyclerview does NOT have any reverselayout or stackfromend attributes and I'm not expecting to use them, instead I just want my list to come from firebase ordered
#ExperimentalCoroutinesApi
suspend fun getLatestPosts(): Flow<Result<List<Post>>> = callbackFlow {
val postList = mutableListOf<Post>()
// Reference to use in Firestore
var eventsCollection: CollectionReference? = null
try {
eventsCollection = firestore.collection("posts")
eventsCollection.orderBy("created_at", Query.Direction.DESCENDING)
} catch (e: Throwable) {
// If Firebase cannot be initialized, close the stream of data
// flow consumers will stop collecting and the coroutine will resume
close(e)
}
val suscription = eventsCollection?.addSnapshotListener { value, error ->
if (value == null) {
return#addSnapshotListener
}
try {
postList.clear()
for (post in value.documents) {
post.toObject(Post::class.java)?.let { fbPost ->
fbPost.apply {
created_at = post.getTimestamp(
"created_at",
DocumentSnapshot.ServerTimestampBehavior.ESTIMATE
)?.toDate()
}
postList.add(fbPost)
}
}
offer(Result.Success(postList))
} catch (e: Exception) {
close(e)
}
}
awaitClose { suscription?.remove() }
}
Now if I sort the list locally after getting the data that works, but I don't want it to be client side, I want to have an ordered list from the server.
What I'm doing wrong ?
Posts timestamp are saved with #ServerTimestamp and Date format into Firestore
CollectionReference (Query) objects are immutable - they can't be changed once created. They use a builder type pattern to construct new Query objects by adding constraints. The original Query remains unmodified.
If you want to compose a set of operations to perform on a Query, you would have to remember the new Query returned by each operation. An easy way to do this is by reassigning the prior Query if you no longer need it:
var eventsQuery: Query = firestore.collection("posts")
eventsQuery = eventsQuery.orderBy("created_at", Query.Direction.DESCENDING)
Or you can do it as a chained sequence of operations, which is more idiomatic in the case that you know all the operations ahead of time:
val eventsQuery = firestore
.collection("posts")
.orderBy("created_at", Query.Direction.DESCENDING)
If you need to conditionally apply operations, however, you will need to take the first approach. A more detailed explanation can be found here: Firestore: Multiple conditional where clauses
Problem was at this line
eventsCollection = firestore.collection("posts")
eventsCollection.orderBy("created_at", Query.Direction.DESCENDING)
Seems like I need to wrap it up all into 1 line and make it a Query, it does not apply any changes to the eventsCollection, this can be done changing CollectionReference to Query
I'm pretty new in the world of MVI pattern. So I'm trying to understand how fit together all the pieces.
I have an app that I structured using MVI pattern (or at least it was what I was meant to do). I have my fragment (I used navigation component but at the moment focus just on one fragment), which is supported by its own ViewModel. Then I have a repository class where all viewmodels retrieve data. Repository has 2 source of data, a web API and a local DB used as cache of data, I used Room for DB management.
I tried different approaches to the problem. At the moment I have done in this way:
In the DAO I used this instruction to retrieve data from the DB:
#Query("SELECT * FROM Users WHERE idTool=:idTool AND nickname LIKE '%' || :query || '%'")
fun users(idTool: Int, query: String) : Flow<List<User>>
Then in my repository I simple get this query to forward to ViewModels:
fun usersFlow(idTool: Int, query: String) = userDao.users(idTool, query)
In the ViewModel I created two MutableLiveData, coordinated by a MediatorLiveData:
val nicknameQuery = MutableStateFlow("")
private val nicknameQueryFlow = nicknameQuery.flatMapLatest {
repository.usersFlow(idToolQuery.value, it)
}
val idToolQuery = MutableStateFlow(DEFAULT_TOOL_ID)
private val idToolQueryFlow = idToolQuery.flatMapLatest {
repository.usersFlow(it, nicknameQuery.value)
}
val users = MediatorLiveData<List<User>>()
init {
users.addSource(nicknameQueryFlow.asLiveData()) {
users.value = it
}
users.addSource(idToolQueryFlow.asLiveData()) {
users.value = it
}
fetchUsers()
}
In this way, from my fragment, I can simply update nicknameQuery or idToolQuery to have an updated list in my RecyclerView. My first doubt is that in this way the fetch of data from my DB is done 2 times, one time for each mutable, but I'd like to retrieve data just one on the app opening (maybe the solution fro this is just check in the nicknameQuery that current query is different from the passed one, in this way since at the beginning current query is empty and it pass an empty query, it is bypassed).
In the Init method of ViewModel, I also call fetchUsers():
private fun fetchUsers() {
viewModelScope.launch {
repository.fetchUsers(DEFAULT_TOOL_ID).collect {
_dataState.value = it
}
}
}
This method checks into the database if there are already cached users with this specific idTool, if not it fetches them from the web and it stores retrieved data into the DB. This is the method inside my repository class:
suspend fun fetchUsers(
idTool: Int,
forceRefetch: Boolean = false
): Flow<DataState<List<User>>> = flow {
try {
var cachedUser = userDao.users(idTool, "").first()
val users: List<User>
if(cachedUser.isEmpty() || forceRefetch) {
Log.d(TAG, "Retrieve users: from web")
emit(DataState.Loading)
withContext(Dispatchers.IO) {
appJustOpen = false
val networkUsers =
api.getUsers(
idTool,
"Bearer ${sessionClient.tokens.accessToken.toString()}"
)
users = entityMapper.mapFromEntitiesList(networkUsers)
userDao.insertList(users)
}
} else {
users = cachedUser
}
emit(DataState.Success(users))
} catch (ex: Exception) {
emit(DataState.Error(ex))
}
}
This method checks if I have already users inside the DB with this specific idTool, if not it fetches them from API. It uses a DataState to update the UI, based on the result of the call. During the fetch of data, it emits a Loading state, this shows a progress bar in my fragment. If data is correctly fetched it emits a Success, and the fragment hides the progress bar to shows the recycler view. This is done in the following way. In my ViewModel I have this mutable state
private val _dataState = MutableLiveData<DataState<List<User>>>()
val dataState: LiveData<DataState<List<User>>> get() = _dataState
As you saw above, my fetch method is
private fun fetchUsers() {
viewModelScope.launch {
repository.fetchUsers(DEFAULT_TOOL_ID).collect {
_dataState.value = it
}
}
}
And finally in my fragment I have:
userListViewModel.dataState.observe(viewLifecycleOwner, { dataState ->
when (dataState) {
is DataState.Success -> {
showUserList()
}
is DataState.Error -> {
Log.e("TEST", dataState.exception.toString())
hideLoader()
Toast.makeText(activity, "Error retrieving data: ${dataState.exception}", Toast.LENGTH_LONG).show()
}
is DataState.Loading -> {
showLoader()
}
else -> {
// Do Nothing in any other case
}
}
})
At this moment Success state takes a list of users, but this list is there from a previous approach, at the moment it is useless since after data is fetched list is inserted into the DB, and I have a Flow to the DB which takes care to update the UI. In this way when I change idTool, when I change query, when I remove a user, the view is always notified
Is this approach correct?
Before this, I used another approach. I returned not a flow from my DB but just a List. Then my fetchUsers always returned a DataState<List>, it checked in the DB and if didn't found anything it fetched data from the web and returned that list. This approach caused me some problems, since every time I changed idTool or query, I always had to call fetchUsers method. Even if a user was removed from database, views didn't get notified since I didn't have a direct flow with the DB.
I am learning to use Firebase Firestore and I have created a list of items that I want to display in my app. When trying to load the list, I don't receive the data but I also don't get any error. I cannot display the data in the Log. What could be happening?
fun getDriverData(): LiveData<MutableList<Driver>> {
val mutableData = MutableLiveData<MutableList<Driver>>()
FirebaseFirestore.getInstance().collection("drivers").get().addOnSuccessListener { result ->
val listData = mutableListOf<Driver>()
Log.i("repo","called")
for (document in result) {
val photoUrl = document.getString("photoUrl")!!
val name = document.getString("name")!!
val team = document.getString("team")!!
Log.i("repo", "${document.id}} => ${document.data}")
val driver = Driver(name,team,photoUrl)
listData.add(driver)
}
mutableData.value = listData
}.addOnFailureListener {
Log.i("repo", "getDriverData: ${it.message}")
}
return mutableData
}
Your collection is actually called "drivers" - WITH the quotation marks. Whatever is generating the documents here is using extra quota when building the name of the collection.
You could read them back by adding those quotes:
FirebaseFirestore.getInstance().collection("\"drivers\"").get()
But you probably want to fix the code that generates the document to not add those quotes.