I am using firebase Cloud Firestore for my Android and iOS apps. I am stuck with how to structure my database.
Basically I have an "organization" collection that contains many users, and for every user, I want to save attendance time and leave time for every day.
The problem is I want to generate reports that allow me to get every day-attendance for each user, single-day attendance for all users, and all days attendance for all users.
so I tried this: inside the user I would have an "attendance" collection then each document is Unix timestamp of that day (to make sure that it's unique). then save fields like attend_time and leave_time...etc. the path is like that "/organization/android/users/3PRs42gRFzZQhLKcUhpf4wPMRV43/attendance/1590271200"
then I needed to get attendance for a single day for all users, so I did this: now I have another path "/organization/android/attendance" and inside the attendance, I store the Unix timestamp of the day, then the user ID then his attendance. and now I am saving attendance twice.
but I still can't get attendance for all days for all users!
this would be easy in Relational Database like SQL. any idea how to do it in firebase?
If you want to track attendance across all users, you're looking for a collection group query. This allows you to query documents from all collections named attendance.
Since a query in Firestore can only see data in the collection(s) it queries, you may have to duplicate some data from parent collections (your users and organizations) into each attendance document to allow the query. This type of data duplication is quite normal in NoSQL databases too.
Finally: if you need to perform many ad-hoc queries, you might want to consider using a database for those that is more suited to that use-case. For example, it is quite common to use Firestore to handle the direct-to-client interactions that require scaling to massive number of users, but then use BigQuery for the ad-hoc querying of that data. There is even a Firebase Extension that automatically exports to BigQuery to make this easier.
Related
I am a novice android programmer, and I am developing my first android application. I'm trying to figure out if firebase is suitable for my needs.
My app looks like this:
users add the product to the database by filling in the product
fields
products are stored in the database as objects with simple fields (numbers and strings)
users search for a product in the database to compare its fields according to different criteria
users edit (update) product fields
users have their own accounts
My priority is the ease of use.
My question is:
Is Firebase suitable for my needs or is there a more suitable solution?
Do I understand correctly that I am interested in the Cloud Storage product?
users add the product to the database by filling in the product fields
Yes, you can achieve this using either Cloud Firestore or the Realtime Database. In both cases, you can add the data as a Map object or as a custom object of your choice.
products are stored in the database as objects with simple fields (numbers and strings)
Yes, you can achieve that. Here are Cloud Firestore supported data types, as well as Firebase Realtime Database data types.
users search for a product in the database to compare its fields according to different criteria.
None of the above databases supports native indexing or search for text fields, so you can use third-party libraries like Algolia or Elasticsearch.
users edit (update) product fields.
These are basic CRUD operations that are supported by both databases.
users have their own accounts
In this case, you should use Firebase Authentication, a service that works perfectly with both databases.
Is firebase suitable for my needs or is there a more suitable solution.
As I see in your requirements, yes, it suits your needs.
Do I understand correctly that I am interested in the Cloud Storage product?
For storing data (objects) you need to choose one of the above-mentioned databases, or why not even both, and for storing files, you should indeed use Cloud Storage for Firebase.
You could use firestore for this. Although full-text search won't work, you can use algoia/elastic to do full-text search.
In my chat application, I store the participants of a chat as their UIDs in a Map so I can so I can do queries like this:
.whereEqualTo("participantUIDs.$currentUserUid", true)
.whereEqualTo("participantUIDs.$partnerUid", true)
The problem is when I try to use this with orderBy
.whereEqualTo("participantUIDs.$currentUserUid", true)
.orderBy("lastMessageSentTimestamp")
I have to create a custom index. But this index will contain that specific user UID and I can't create an index for every user in my app. How can I circumvent this problem?
You can order the documents on the client after an unordered query. This should not be very taxing on the client app when the number of documents is less than 10,000.
Regarding:
I can't create an index for every user in my app.
That's definitely not an option, as there are some limitations when it comes to Cloud Firestore indexes:
https://firebase.google.com/docs/firestore/quotas#indexes
However, even if you manage to stay below these limits, that's not an option to manually create an index for each and every user that joins your app.
In my opinion, for your particular use-case, you should consider augmenting your data structure to allow a reverse lookup. Meaning that you should create a participantUIDs collection where you should keep the lists for each user. This technique is called denormalization and is a common practice when it comes to NoSQL databases like Cloud Firestore or Firebase Realtime Database.
But remember, there is "no perfect database structure":
What is the correct way to structure this kind of data in Firestore?
It's a little old, but I think this video might also help:
https://www.youtube.com/watch?v=u3KwKQddPoo
More info regarding why you need an index:
Why does this firestore query require an index?
P.S. You can also rely on Firebase Realtime Database when Cloud Firestore may become a little expensive. Both work really well together.
Info:
Array or Subcollection for storing events user uploaded
I need to perform two different queries in my app.
Display a global leaderboard (rank all users based on points)
Display a leaderboard only among friends of the currently logged-in user.
Currently, I have structured my database as follows.
Users (collection):
user_id (document):
...
points: number
...
Friends (sub-collection):
user_id (document):
name: string
points: number
This approach works for populating the global leaderboard as it's just:
db.collection("Users")
.orderBy("points", Query.Direction.DESCENDING)
.limit(10)
.get()
Displaying a leaderboard among friends of a user also works in this case, but updating points of a user becomes an expensive operation because of multiple duplication.
Is there a way to structure my database so I can efficiently query and update my users collection?
I think what you're doing now is probably the best way to go. Data duplication is normal for NoSQL type databases, and it's expected that you'll do multiple updates to keep everything in sync if something changes. That's totally normal.
The alternative is to keep all relevant data in a single collection somehow (don't use a subcollection to store friends), but then you'll end up with a different problem of eventually exceeding the capacity of the document that contains all the data for each user.
The bottom line here is that NoSQL databases like Firestore give you better scalability and faster queries at massive scale, at the expense of less flexible querying and more work keeping duplicated data up to date. If you would prefer more flexible querying and easier updates, then Firestore might not be the best database for your application.
I have a database in Firestore that is available for all my users. This way updates made by me to this database is instant updated for my users.
My challenge is that my users can edit some of the data in the database - user preferred values. These changes can of course not be written to the public database, but must either be written to a local SQLite-database on the phone or somehow written to each users private collection of documents in firestore.
A SQLite solution means that each time a document from Firestore is displayed, I need to read the local SQLite database to check for changes.
Using private collection of documents in Firestore means that I have to read two documents for each item that I want to display.
I struggle to find the "perfect" solution for this situation.
What is the best approach? Is there a solution I haven't thought of?
I believe the best approach is Firestore Security Rules.
Any data for a user should go in a special document: /users/userId
Set up the firestore rules so that this document is only writable by the user with ID === userId. You can make the document readable by all if you need or limit it to only the user who created it.
I currently have an Android app which uses 3 SQLite database tables and I want to store this data on the cloud in my Java-based GAE app. It will be used as backup and also, the user will be able to view it in their browser upon logging in. The user is entering data into the Android app so all the data in the 3 tables belongs to that user. Is there a recommended way of storing this type of user-specific data? Should I store user email with each entity in order to identify it or have a User entity as the parent and all the entities belonging to this user as the children? Are there any advantages of using a parent in this case?
It all depends on how many records you have for a single user, how frequently these records are updated, and how you access this data (what kind of queries you need, etc.) So there is no simple and definitive answer to your question.
Most likely, you will be fine with either approach unless you have thousands of records per user and they update them every few minutes, at which point you may run into some limitations.
Note that you don't need to include an email address to identify each record. Typically, you create a user entity first, and then you use an id of this user entity (a Long) to identify all other entities that are related to this user.
My two cents.Unlike Sqlite,Google App Engine is not a relational database so saving your SQlite data to GAE won't be a straightforward task.However, you could create an app on GAE where you use the useremail from ur app as the Entity key.You can then retrieve the user specific info based on this key.All(well,the most important thing)you need to do in this case is find a way to send that data from your app to GAE.