I am currently getting a single dataSnapshot from Firebase like so:
public Task<DataSnapshot> getLatestMessage(#NonNull String roomId) {
final TaskCompletionSource<DataSnapshot> source = new TaskCompletionSource<>();
DatabaseReference dbRef = mDatabase.getReference(NODE_MESSAGES).child(roomId);
dbRef.keepSynced(true);
ValueEventListener listener = new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
source.setResult(dataSnapshot);
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
source.setException(databaseError.toException());
}
};
Query query = dbRef.orderByKey().limitToLast(1);
query.addListenerForSingleValueEvent(listener);
return source.getTask();
}
Notice that I called keepSynced() on dbRef object already.
Here's the sample data structure:
/root
/messages
/$roomId
/$messageId
/content
/timestamp
/etc...
I am able to get the most recent single snapshot data as expected, but I was wondering, does it make any difference if I move the keepSynced() call in the Query object instead of the DatabaseReference? i.e.
// dbRef.keepSynced(true); >> REMOVE THIS <<
ValueEventListener listener = new ValueEventListener() {...};
Query query = dbRef.orderByKey().limitToLast(1);
query.keepSynced(true); // >> ADD THIS <<
query.addListenerForSingleValueEvent(listener);
We're currently averaging 50% load (per day) on Firebase right now and with the steady inflow of users, I was wondering if it could improve anything in the app somehow, specially with the load. I even tried something as silly as this:
dbRef.keepSynced(true);
ValueEventListener listener = new ValueEventListener() {...};
Query query = dbRef.orderByKey().limitToLast(1);
query.addListenerForSingleValueEvent(listener);
dbRef.keepSynced(false);
-- enabling keepSynced() at the start to make sure that the reference is pointing to the most recent version, then disabling it after querying and adding the listener. Unfortunately, this doesn't provide the fresh data, not like when keeping it enabled.
I've already gone through the Optimization DB Performance documentation and believe that I followed the suggested practices as needed.
Putting the keepSynced() call on a limitToLast() query will not make any difference on the load on the database server. The server needs to load the exact same data and monitor it, it just only returns the last item to the client.
I recommend using keepSynced sparingly in your app. Each call to keepSynced keeps an empty listener to the reference/query you attach it to. That means that each client has an active listener to each chat room you call keepSynced on, even when the user is not looking at that room. While that may be precisely the right call for the use-cases of your app, it will limit the scalability of your app.
If you're worried about reaching peak load, you might want to consider looking into how to shard your data over multiple databases. Chat apps are typically relatively easy to shard, since each chat room is already isolated.
Related
I'm trying to build a simple shopping list android app with a firebase database.
Quick context:
User registers with email and should be able to create a group to share a shopping list that belongs to the group (and all the members of the group). And then later on shopping list items are added to the shopping list.
I'm a bit stuck on how to handle the many-to-many relationship between the users and the groups.
I read this post: Many to Many relationship in Firebase
And based on that I created this data structure in Firebase:
What I don't really understand that how I will query this exactly. As we use value event listeners in Firebase, my guess was that I would need to attach the listener to the database reference and save the related data locally in the app to populate the screens.
For example, I would like a recycler view with all the groups that belong to the user, where the cards would display the name of the group, the creation date of the group, and the group members. Based on the above data structure, that's 4 separate value event listeners nested into each other, is this correct?
This is part of the code that I have written and this is only saving the userIDs for each group so it would require another nested query:
//get group ids - there is at least one as own_group is set up with the user details
Utils.userGroupReference.child(Utils.firebaseUserID).addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot snapshot) {
for (DataSnapshot groupIDSnapshot : snapshot.getChildren()) {
Utils.myGroupIDs.add(groupIDSnapshot.getKey());
}
for (int i = 0; i < Utils.myGroupIDs.size(); i++) {
//get groups based on group ids
Utils.groupReference.child(Utils.myGroupIDs.get(i)).addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot snapshot) {
Group group = (snapshot.getValue(Group.class));
GroupUser groupUser = new GroupUser(group);
Utils.myGroups.add(group);
Utils.groupUserReference.child(snapshot.getKey()).addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot groupUserSnapshot) {
ArrayList<String> userIDs = new ArrayList<>();
for (DataSnapshot userIDSnapshot: groupUserSnapshot.getChildren()) {
userIDs.add(userIDSnapshot.getKey());
}
Utils.myGroupUsers.add(new GroupUser(group, userIDs));
}
#Override
public void onCancelled(#NonNull DatabaseError error) {
}
});
Utils.myGroupUsers.add(groupUser);
}
#Override
public void onCancelled(#NonNull DatabaseError error) {
}
});
I would appreciate it if someone could shed some light on this for me.
Another question is where do we normally call the value event listeners? This snippet is from the onCreate() method in the MainScreen which eventually would contain all the shopping lists. I was thinking to put it in the splash screen but my experience is that whenever a node is updated, the application will get thrown back to the screen that contains the value event listener, so I'm not sure which is a better option
Thank you very much!
I would typically use addListenerForSingleValueEvent listeners or getData calls for some of the inner data loading here, but it is correct that you'll need a separate call for each piece of data you load.
Since Firebase pipelines all these requests over a single connection, the data is loaded really quickly - so it's really just the code that gets a bit convoluted.
If that is something you'd like to prevent, you could consider duplicating some of the data from the User into each GroupUser and/or UserGroup entry where that user is present too. This will make the code that writes the data more complex, but simplifies the data loading.
This sort of trade-off is quite common when dealing with NoSQL databases, and is one of the reasons they can scale to so many users. To learn more about this, I recommend checking out NoSQL data modeling, Firebase for SQL developers, and Getting to know Cloud Firestore (the latter is for Cloud Firestore, but many of the lessons also apply here).
First time listener firing takes so much time (approx. 40s), subsequent loads takes lesser time (approx. 1s), how can speed up the first time load as well?
I have given,
FirebaseDatabase.getInstance().setPersistenceEnabled(true);
on start of the activity.
DatabaseReference ref = FirebaseDatabase.getInstance().getReference("");
ref.keepSynced(true);
ref.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
});
If you are using a listener, you need to know that you are trying to read data over the internet. You cannot compare this with an atempt to read a SQLite database, which is stored locally on disk. The speed of getting the data from Firebase servers, depends on the speed of your internet connection and on the ammount of data that you are trying to get. So most likely the reason for waiting so much is one of this. If the reason is the ammount of data, try to optimize your queries or try to get the data in small parts.
If we are speaking abot the first atempt to read a record, it might be slower than the subsequent ones, because it has to initiate the internet connection. I know that Firebase team is trying to improve the performance, but you can't expect 0ms when retrieving data over a network.
According to your comment, I need to tell you a few more things. There is no way to force the retrieval of the data from the cache while you're connected to the server, as you cannot to stop the retrieval of the data from the cache while you are not connected to the server.
Firebase is desinged to retrieve data from the chache when the device is permanently offline or while your application temporarily loses its network connection and you cannot change this behaviour.
Edit:
So to get the FirebaseDatabase object you need to use the following line of code once:
FirebaseDatabase firebaseDatabase = FirebaseDatabase.getInstance();
Then to get a DatabaseReference, you need to use the following line.
DatabaseReference rootRef = firebaseDatabase.getReference();
I'm sure you'll need to in your activity more then one reference. Let's say you'll use two:
DatabaseReference usersRef = rootRef.child("users");
DatabaseReference postRef = rootRef.child("post");
You can now add a listener for on each one of these references like this:
ValueEventListener eventListener = new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
//code to get the data
}
#Override
public void onCancelled(DatabaseError databaseError) {}
};
usersRef.addListenerForSingleValueEvent(eventListener);
And the code the remove the listener is as explained in my answer from this post.
Don't forget, onDestroy() is not always called.
As a conclusion, you create a single database connection, use as many references you need, add the listeners accordingly, remove then according to the life-cycle of you activity.
I'm trying out Firebase as my backend for a prototype app im creating.
As a mockup, im creating a fake game.
The data is pretty simple:
There are 3 lists of 'levels' common to all users, organized by difficulty ( easy, medium hard ). ( These are in a Fragment inside a ViewPager
Each level has mini-games inside of them than the users complete.
On the Android app, the user sees that list but also sees a counter of how many mini-games of that level he/she has completed.
If the user clicks on a level, he/she sees the list of mini-games and sees which of those are completed.
Im currently structuring my data as follows:
levels:{
easy:{
easy-level-1-id:{
total-mini-games:10,
easy-level-1-id:String
}
},
medium:{....},
hard:{.....}
}
user-progress:{
user-id:{
levels:{
easy-level-1-id:{
user-completed:int
}
},
mini-games:{
mini-game-1:true
mini-game-2:true
}
}
}
I have to access many places in order to check if a game has been completed, or how many games have been completed per level.
All of this is in order to avoid nesting data, as the docs recommend.
Is it better to do it like this or is it better to store every available level under every user id in order to do less calls, but having much more repeated data ?
I know about the FirebaseRecyclerViewAdapter class provided by the Firebase team, but because it takes a DatabaseReference object as an argument for knowing where the data, it doesn't really apply here, because i fetch data from different structures when building the model for the lists.
mRef1.child(some_child).addValueEventListener(new ValueEventListener(){
#Override
public void onDataChange(DataSnapshot dataSnapshot){
final Model model = dataSnapshot.getValue(Model.class);
mRef2.child(some_other_child).addValueEventListener(new ValueEventListener(){
#Override
public void onDataChange(DataSnapshot dataSnapshot){
//add more data into the model class
.....
}
#Override
public void onCancelled(DatabaseError databaseError){}
});
#Override
public void onCancelled(DatabaseError databaseError){}
});
Any pointers as to how to work with more structured data ?
FirebaseUI includes support for working with indexed data. It can load linked data from an index, such as in your mini-games node.
In your example that could work as:
keyRef = ref.child("user-progress").child(currentUser.getUid()).child("mini-games");
dataRef = ref.child("mini-games");
adapter = new FirebaseIndexRecyclerAdapter<Chat, MiniGameHolder>(
MiniGame.class,
android.R.layout.two_line_list_item,
MiniGameHolder.class,
keyRef, // The Firebase location containing the list of keys to be found in dataRef.
dataRef) //The Firebase location to watch for data changes. Each key key found at keyRef's location represents a list item in the RecyclerView.
I'm keeping track of a count that users update on the Firebase database through an Android app. The way it works right now is that upon interaction the user's app looks up the current count on the database (using a addListenerForSingleValueEvent() and onDataChange() method defined within the new ValueEventListener) and adds one to it and then sets the count to this new value using mRef.setValue() where mRef is the reference to the database.
The issue I'm worried about is what would happen if a large number of users interacted with the database together at the same time; does Firebase take care of making sure that the value is read and incremented properly or is there a lot of overlap and potentially a loss of data because of that.
When working with complex data that could be corrupted by concurrent modifications, such as incremental counters, Firebase provides a transaction operation.
You give this operation two arguments: an update function and an optional completion callback. The update function takes the current state of the data as an argument and will return the new desired state you would like to write.
For example, if we wanted to increment the number of upvotes on a specific blog post, we would write a transaction like the following (Legacy code):
Firebase upvotesRef = new Firebase("https://docs-examples.firebaseio.com/android/saving-data/fireblog/posts/-JRHTHaIs-jNPLXOQivY/upvotes");
upvotesRef.runTransaction(new Transaction.Handler() {
#Override
public Transaction.Result doTransaction(MutableData currentData) {
if(currentData.getValue() == null) {
currentData.setValue(1);
} else {
currentData.setValue((Long) currentData.getValue() + 1);
}
return Transaction.success(currentData); //we can also abort by calling Transaction.abort()
}
#Override
public void onComplete(FirebaseError firebaseError, boolean committed, DataSnapshot currentData) {
//This method will be called once with the results of the transaction.
}
});
Legacy source
New firebase version source
Firebase database handles up to 100 simultaneous real time connections to your database if your are using their free plan but once the 101st users connects to your database the database would stop responding and would display the values that were last edited. Firebase is really good at handling real time connections simultaneously so it depends on your pricing plans. If you want to use the database for free, there will be no issues handling 100 connections but if you want to handle more users use their generous pricing plans.
I am new to Firebase and need some help with a query to retrieve data from a table. I am currently able to access and retrieve the data that I need from firebase, however, the timing is the problem I am having an issue with.
From everything I've seen, the firebase database requires me to add event listeners to the Query or DatabaseReference objects. I am trying to download the contents of a node called "questions" before a method to display the question contents is called, however, I cannot control the timing of the firing of the event which downloads the data, and as a result my display method is always called before the firebase event fires.
How can I execute a query when I want, and be sure it will be completed before a certain section of my code executes? I am used to traditional RDBs where you execute a query and get its results and then move forward with your logic. The need to use an event handler with firebase is what I am having a hard time with. I have even tried moving the definition of the firebase reference object and the event handler into onCreate() and moved the code that calls my display method into onStart() without any success - same problem. The data I am trying to get does not change so I only need to download it once at the beginning to have available for the display method.
Here is an image of my "questions" node which is a child of the root.
image of the child "questions" node on my firebase DB
Here is my code:
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// Get Firebase DB reference
firebase = FirebaseDatabase.getInstance();
fdbRef = firebase.getReference("questions");
// [START Question_event_listener]
fdbRef.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
// Get Questions object and use the values to update the UI
objQuestions = dataSnapshot.getValue();
Log.w("Firebase:", "In Firebase ValueEventListener");
}
#Override
public void onCancelled(DatabaseError databaseError) {
// Getting Questions failed, log a message
Log.w("Firebase Error:", "onCancelled:", databaseError.toException());
Toast.makeText(ReviewActivity.this, "Failed to load question!", Toast.LENGTH_SHORT).show();
}
});
//. . . remaining onCreate logic removed for simplicity
} //end of onCreate
#Override
public void onStart() {
super.onStart();
// I moved this logic from onCreate to onStart but did not help...
// Firebase retrieve must execute before I call any of these
if (list_type == MainActivity.LIST_UNREVIEWED_DOCS)
displayNewReviewForm();
else if (list_type == MainActivity.LIST_REVIEWS)
displayCompletedReview();
else // (list_type == MainActivity.LIST_DRAFTS)
displayDraftReview();
}
Other alternatives if I can't get this resolved may be to move this retrieve logic to the prior Activity in my sequence and pass the retrieved data as an extra to this activity - but that seems really silly to have to do such a thing. I would think I should be able to get data from a DB when I need it... not when it feels like giving it to me.
I appreciate any help getting me past this issue.
Your code is downloading the snapshot data containing all the data at the first go only, and with Firebase, you cannot download data timely, you can only do it through different references.
What I would suggest you to do is, to have a DatabaseReference of q01, q02 respectively and then call data as in when required.
If your Keys "q01", "q02" are static, which they are looking at the scenario. I would suggest you to have their DatabaseReferences:
question_one = firebase.getReference("q01");
question_two = firebase.getReference("q02");
question_three = firebase.getReference("q03");
//Once you have the reference, you can call their ValueListeners respectively
question_one.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
// Get Questions object and use the values to update the UI
objQuestions = dataSnapshot.getValue();
Log.w("Firebase:", "In Firebase ValueEventListener");
}
#Override
public void onCancelled(DatabaseError databaseError) {
// Getting Questions failed, log a message
Log.w("Firebase Error:", "onCancelled:", databaseError.toException());
Toast.makeText(ReviewActivity.this, "Failed to load question!", Toast.LENGTH_SHORT).show();
}
});
After looking at this a bit more, I came up with 2 possible solutions to the problem I had.
The first one I sort of mentioned already in my original question post, however it's not ideal in my opinion. It basically involves relocating the firebase retrieve logic to the prior Android Activity and passing the retrieved data to the Activity I need it in as an Extra. In my case the data is a HashMap so I would need to use the serialize versions of the methods to pass the serialized content to the desired Activity.
The best solution, is much simpler. I basically relocated the logic that I had in the onStart() function (which is calling my custom display methods) and moved it inside of the Firebase Event Listener's onDataChange() method, right after the call to dataSnapshot.getValue(). This ensures that I get the data before I call my display methods. This seems to be working well now.