I am starting with Firebase and want to know what are the most effective ways to structure data
Let's take the example of a simple social media app where only photos can be shared (like the beginnings of Instagram).
user upload a photo with some meta data (Description)
(Home Feed) the users (followers) will see the post in a chronological way
and offcource there will be other functionality like (liking the post , saving it , commenting)
searching and following users
notification about likes and comments
search in comments
what could be a good structure for storing the data and good effcient way to get data ASAP
I'll go ahead and leave an answer for how I would approach this. My answer will be geared more towards Firestore even though the question is marked as Realtime Database. There are multiple ways to structure the data. This is the general structure I would use given your example:
users
- name
- timestamp
posts
- imageURL
- description
- timestamp
- likeCount
- commentCount
posts/comments //subcollection
- userID
- comment
- timestamp
posts/likes //subcollection
- userID
- timestamp
savedposts
- postID
- userID
followers
- userID
- followedID
Some additional notes:
Image Upload
The best option here is to upload the images to cloud storage and utilize a cloud function to generate a public URL and save it to the post document.
Comment / User Search
As stated in my comment, Firebase does not have a great solution for text based searches. The solution I utilized in my project was to utilize a cloud function to keep an Algolia index in sync with my users collection. I then offload the user search to them through a callable cloud function - though you could utilize the Algolia client SDK directly in your app if you wanted. In your scenario, you would also have to keep all of your comments in sync as well. Algolia isn't a cheap service, so I would look into the pros / cons of using the other options listed in the docs.
Document IDs
I generally let Firestore auto ID the documents, but here I would make some exceptions. For the savedposts and followers collections I would utilize a (manual) compound ID of {userID}{postID} and {userID}{followedID} respectively. It allows you to perform simple actions of unliking and unfollowing without querying for the document first. Ex) firestore().collection('postsaves').doc(`${userID}${postID}`).delete()
Final Thoughts
You mention maybe moving to AWS. I have worked much more in Firebase than in AWS, but I have done both. In my opinion, Firebase is unmatched in both usability and documentation. There are some compromises in terms of functionality and fine tuning but I recommend sticking with Firebase if the lack of text searching is the only hurdle.
Related
Working with Firebase for the first time and looking for advice of setting up the right structure for my project which is basically an "offers/coupon" type starter project.
The scenario is this:
I have a node containing a list of all offers available to users
This list of offers is displayed to users after successful Firebase authentication
When a user redeems an offer, I want to be able to count/record that activity in their child node under user and hide that offer so that they cannot see it again once used.
My question is what would be the best way to do this given that offers may be added, may expire, or may change at some point in the future. So, in effect, the user should receive the list of most updated offers, minus the ones he/she have used in the past.
a) would it be more effective to have a master list of offers, and then run a cloud/server function to clone this list for each new user an track that way
Firebase Structure 1
or
b) Keep a master list of offers in one node, then track user specific offer usage
Firebase Structure 2
Appreciate your guidance
The second solution is better because you'll save bandwith. This practice is called denormalization and is a common practice when it comes to Firebase. The first solution is not good becase every time you want to display the users you donwload unnecessary data. If you want to read more details about how you can structure a Firebase database in a efficient way, please read this post, Structuring your Firebase Data correctly for a Complex App. Also, you can take a look a this tutorial, Denormalization is normal with the Firebase Database, for a better understanding.
Second solution is much good. Because in first one we are having redundancy of data in our database. And second one obviously removing that cause.
But instead of using true or false because it is only showing you, "it's available or not", so you can use a string type parameter as "expired", "going to expire" and "updated" or whatever sooo. So it. Will be able to trace all you information related to offer for particular user. I think this is your requirement also.
Happy coding.
I have only found code snippets to load the currently logged in user:
FirebaseUser user = FirebaseAuth.getInstance().getCurrentUser();
But what if I need to load another user's data, like the image path and name? I want to load them by their UID.
I could of course save the data myself like this:
-> users
-> UID
-> imagePath
-> name
But I thought there might be a way to use the FirebaserUser class.
I actually think there has to be some way, it's just not written in the docs.
There's no way to do this directly using FirebaseUser class. The old docs of Firebase 1.0 (Before it was acquired by Google) mentioned that exact scenario and the possible solution. The current doc is not completed yet.
The solution was to save the user's data in the Real-time database keeping the uid as the key. So when you need the data of another user, you'll just have to remember their UID and can search for their data (Exactly like how you mentioned in the question description).
I searched the other way to do that without using Real-time database when I started working on my own app, however after finding no other method and reading the old doc, I gave up and used the database way. Trust me it's way easier and flexible to use the database, instead of hacking your way to avoid using it altogether.
I understand that one cannot do aggregation queries on firebase to compute statistics and/or return bucketed data similar to a relational db. I wondered if denormalising data could some how replicate aggregation?
I have the following specific problem:
I have a set of data with id, subject, score, author, date, tags, locations.
I would like to bucket the data in each of these dimensions according to some "similarity" rules e.g. all data on the same day bucketed together, all data located within 10 metres etc., each bucket showing me how many pieces of data in that bucket and the average score.
Once I choose a bucket, the underlying ids are retrieved.
Given a set of retrieved Ids, I download the data or go back to step 1 to refine using these ids.
I have gone through the Android documentation but cannot work out a way of doing this. Is something like this possible in Firebase? Is it possible via denormalisation or by uploading server code?
I am loathe to give up Firebase as it is so easy and simple to use but aggregation is a core part of my app so hope it is...
Thanks,
Riz
Edit: Edited to make problem clearer.
I am looking into options how to realize the following use case. A iOS/Android user is using my app which gets its table view data populated by a cloud database solution. The user must be able to send back information (e.g. name + date) which needs to be send back to the database and gets stored there in a/different table/s. Moreover, I would need the db-solution to run automated reports based on the information sent back by the users (e.g. in an excel file).
So far I only found ragic.com might suit my needs. But what other options are out there, which might not be as fancy looking but will get the job done. Thanks guys!
A WebService is considered best practice regarding these matters. Have a look at this guys tutorial:
http://android.programmerguru.com/android-webservice-example/
I already published an android app where you can see a list of specific objects and detailed informations about them. The list changes every day but some of the objects can appear again.
The application is communicating with a PHP server over HTTP and periodically pulls the list of objects.
I now plan to extend the app to make it possible to rate the objects and add a comment similar to how it is done in the android market. I'd like to avoid forcing the user to sign up for an account for being able to comment.
I see two problems:
The comment-system could be abused by spammers
A comment could be added from another system
So my questions are:
How to protect the system from spam?
How to authenticate the application with the server?
How do I limit the number of comments to one per user and object?
What about the androids device id? Is it unique enough to use it as identifier for the user?
Which other problems do you see?
2020 Commenting/Rating/Reviews Options
Since Socialize is out, here are a few options you can explore:
Build your own comment/rating implementation. Personally I love reddit and how it handles nested comments and ratings. Here's a library I found that implements it beautifully. Please note you'll need to tie this with a cloud-database. This is based on groupie. Article & implementation. Many ways to do this - https://stackoverflow.com/a/59472206/668240
Disqus - SDK's coming soon to iOS and Android.
BazaarVoice - commercial
Social Networks SDKs like Facebook, Twitter, etc. Personally I dislike this as we'll need to authenticate users with respective networks to use the APIs. It's like we are shipping off users of our apps to social networks. If you don't have a problem with that - then it might be for you
Legacy Option in 2014:
You can try out Socialize SDK which is open-source and a really good SDK for the rating and commenting you are looking for. It already has a well-functioning Commenting system built-in along with a 'like/love' facility and sharing to FB and Twitter. Each 'entity' (object in your case) can have metadata associated with it. So all you have to do is construct/use a rating widget, then send that rating with the entity attached to your object. To display your rating/comment is as simple as retrieving them from Socialize.
Each object (element from your app) should be associated with an entity which has a unique key in the form of a URL - sort of like a primary key to recognize your items. This entity can have meta-data - any data that you can insert on behalf of your object. Once you do that, you can retrieve that metadata any time you want.
I've been using Socialize for around a year now. They've matured over this period and are always aspiring to be the best at what they do.
Look at the Socialize Bar at the bottom. Its can be customized to your needs.
What's more - Socialize is free.
As for your questions:
There is comment moderation built into the Socialize Web Component
where you can filter out anything you feel is out of place.
Socialize allows you to authenticate through Facebook and Twitter.
Limiting to one comment per user can be achieved by using their User
and Comments API.
Socialize has both Anonymous authentication as well as Social A/c
authentication. I believe you can remove anonymous auth. So that
ensure that every user is authenticated before rating/commenting.
For authentication, you could use OpenID like StackOverflow does or Facebook authentication. Once you have them authentication, it shoud be easy to limit the number of comments to one per user per object. As far as spam, you could follow StackOverflow's model and allow users to vote comments up or down or flag as spam. Perhaps users with comments that have been voted up would have more power and be able to flag comments as spam.
You'll need some sort of rate limiting. I've used this one in this example before.
So you need a table with the user's ID and how many api calls they have left, and then when their last api call was. Then use the algorithm to update the values in the table every time a method is called.
Read through this, I think it should be possible to create an UUID for every case:
http://android-developers.blogspot.de/2011/03/identifying-app-installations.html
And then keep a hidden api key which is hard coded, or at least get's everytime calculated the same or in enigma style influenced by the time it is used. But you will be never be sure, that it won't be find out by crackers/hackers and maybe abused, you will always have this Problem.
Authenticate with the UUID of the user + api-key.