I'm keeping track of a count that users update on the Firebase database through an Android app. The way it works right now is that upon interaction the user's app looks up the current count on the database (using a addListenerForSingleValueEvent() and onDataChange() method defined within the new ValueEventListener) and adds one to it and then sets the count to this new value using mRef.setValue() where mRef is the reference to the database.
The issue I'm worried about is what would happen if a large number of users interacted with the database together at the same time; does Firebase take care of making sure that the value is read and incremented properly or is there a lot of overlap and potentially a loss of data because of that.
When working with complex data that could be corrupted by concurrent modifications, such as incremental counters, Firebase provides a transaction operation.
You give this operation two arguments: an update function and an optional completion callback. The update function takes the current state of the data as an argument and will return the new desired state you would like to write.
For example, if we wanted to increment the number of upvotes on a specific blog post, we would write a transaction like the following (Legacy code):
Firebase upvotesRef = new Firebase("https://docs-examples.firebaseio.com/android/saving-data/fireblog/posts/-JRHTHaIs-jNPLXOQivY/upvotes");
upvotesRef.runTransaction(new Transaction.Handler() {
#Override
public Transaction.Result doTransaction(MutableData currentData) {
if(currentData.getValue() == null) {
currentData.setValue(1);
} else {
currentData.setValue((Long) currentData.getValue() + 1);
}
return Transaction.success(currentData); //we can also abort by calling Transaction.abort()
}
#Override
public void onComplete(FirebaseError firebaseError, boolean committed, DataSnapshot currentData) {
//This method will be called once with the results of the transaction.
}
});
Legacy source
New firebase version source
Firebase database handles up to 100 simultaneous real time connections to your database if your are using their free plan but once the 101st users connects to your database the database would stop responding and would display the values that were last edited. Firebase is really good at handling real time connections simultaneously so it depends on your pricing plans. If you want to use the database for free, there will be no issues handling 100 connections but if you want to handle more users use their generous pricing plans.
Related
I have 2 questions related to Firebase's transaction in the real-time database. It will be easier to explain with an example. This is just an example, not my real code. So, do not worry if there are some compile errors.
Let say I have a building. In this building, there are have some data. I have an array of floors. Each floor can have a counter of how many chairs there are on this floor. A client can have a lot of floors, so I do not want to load all of them. I just load the ones I need for this client. I need to know how many chairs there are in total even if I do not load them all. The rules can look like this:
"...":
{
"building":
{
"...":
{
},
"totalNbChairs":
{
".validate": "newData.isNumber()"
},
"floors":
{
"$floorId":
{
"nbChairs":
{
".validate": "newData.isNumber()"
},
"...":
{
},
},
},
},
},
As I said, this is just an example, not my actual code. DO not worry about code issues.
My clients can connect on multiple devices, so I need to use transactions to adjust the "totalNbChairs" when a floor changes his "nbChairs". Important, I need to set the actual number of chairs on the floor, not just decrease a value. If the "nbChairs" is 10 for a floor and the client set "8" on 2 devices at the same time, I can not do "-2" on both devices at the same time.
The transaction will look like this
void SetNbChair(String floorId, long nbChairsToSet){
FirebaseDatabase.getInstance().getReference()
.child("...")
.child("building")
.runTransaction(new Transaction.Handler() {
#Override
public Transaction.Result doTransaction(MutableData mutableData) {
//first I need to know how many chairs I have right now on the floor
MutableData nbChairMutableData = mutableData.child("floors").child(floorId).child("nbChairs");
Long nbChairLong = (Long)nbChairMutableData.getValue();
long nbChair = 0;
if(nbChairLong != null){
nbChair = nbChairLong;
}
long diff = nbChairsToSet - nbChair;
//now I can update the number of chair in the floor
nbChairMutableData.setValue(nbChairsToSet);
//Update the totalNbChairs
MutableData totalNbCHairsMutableData = mutableData.child("totalNbChairs");
Long previousTotalChairLong = (Long)totalNbCHairsMutableData.getValue();
long totalChair = 0;
if(previousTotalChairLong != null){
totalChair = previousTotalChairLong;
}
totalChair += diff;
//update the value
totalNbCHairsMutableData.setValue(totalChair);
}
#Override
public void onComplete(DatabaseError databaseError, boolean committed,
DataSnapshot currentData) {
...
}
});
}
My first question is: When are downloaded the data I need to get on the client side? Because for what I see, 2 things can happen.
First, when I do this
FirebaseDatabase.getInstance().getReference()
.child("...")
.child("building")
.runTransaction
It's possible Firebase downloads everything in (".../building"). If this is the case, this is pretty bad for me. I do not want to download everything. So if all the data is downloaded for this transaction, this is really bad. If this is the case, does anyone have an idea how I can do this transaction without download all the floors?
But, it's also possible Firebase downloads the data when I do this
Long nbChairLong = (Long)nbChairMutableData.getValue();
In this case, it's far better but not perfect. In this case, I need to download "nbChairs". Wait to get it. After, I need to download "totalNbChairs" and wait to get it. If firebase downloads the data when we do a getValue(). Can we batch all the getValue I need in a single call to avoid waiting to download twice?
But I may be wrong and firebase does something else. Can someone explain to me when and what firebase downloads to the client so I will not have a huge surprise?
Now the second question. I implemented the version I show. But I can not use it before I know the answer to my first question. But, I still did some tests. Pretty fast, I found out my "transaction" callback got some "null" event if there is data in the database. Ok, the documentation said it was expected behavior. Ok, no problem with that. I protected my code and I have something like this
if(myMandatoryData == null){
return Transaction.success(mutableData);
}
and, yes, the first time, my early return is called and the function is recalled. The data is valid the second time. Ok, seems fine, but... and a BIG BUT! I noticed something pretty bad. Something to mention, I have some ValueEventListener active to know when the data changed in the database, So I have some stuff like this
databaseReference.addValueEventListener( new ValueEventListener(){
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
...
}
#Override
public void onCancelled(DatabaseError databaseError) {
...
}
} );
After my early return, every "onDataChange" on every ValueEventListener is called with "null". So, my code in the callback handles this like if the data was deleted. So, I got an unexpected result in my Ui, it's like someone deleted all my data. When the transaction retries and has the data, the "onDataChange" is recalled with the valid data. But until it does, my UI just shows like there is nothing in the database. Am I supposed to cancel every ValueEventListener when I start a simple transaction? This seems pretty bad. I do not want to cancel them all. Also, I do not want to redownload all the data after I restart them when the transaction is done. I do not want to add a hack to ignore deleted data while a transaction is running in every ValueEventListener. I can miss if some data is really deleted from another device when the transaction is running. What Am I supposed to do at this point?
Thanks
When you execute this code:
FirebaseDatabase.getInstance().getReference()
.child("...")
.child("building")
.runTransaction
Firebase will read all data under the .../building path. For this reason it is recommended to run transactions as low in the JSON tree as possible. If this is not an option for you, consider running the transaction in something like Cloud Functions (maybe even with maxInstances set to 1) to reduce the contention on the transaction.
To you second question: this is the expected behavior. Your transaction handler is immediately called with the client's best guess to the current value of the node, which in most cases will be null. While this may never be possible in your use-case, you will still have to handle the null by returning a value.
For a longer explanation of this, see:
Firebase realtime database transaction handler gets called twice most of the time
Firebase runTransaction not working - MutableData is null
Strange behaviour of firebase transaction
I'm new in Android and Firestore. Im currently making an app that uses Cloud Firestore. In my transaction I have one read (transaction.get()) and one write (transaction.set()) operation. I've noticed that in usage tab in Firebase Console, this transaction increments read counter by 1, but write counter increments by 2. I have removed transaction.set operation for testing, and with transaction.get operation only, this whole transaction still increments write counter by 1. Is it normal? Are those normal rules for billing transactions in firestore? I don't know if it matters that reading and writing is done to different files in Cloud Firestore.
db.runTransaction(new Transaction.Function<Object>() {
#Nullable
#Override
public Object apply(#NonNull Transaction transaction) throws FirebaseFirestoreException {
DocumentSnapshot snapshot = transaction.get(carReference);
.
.
.
transaction.set(pointReference, point);
return null;
}
});
According to the Official Documentation, You are charged for each document read, write, and delete that you perform with Cloud Firestore.
In your case, I am not able to see why the write is incremented by 2 the first time you write. Maybe you are writing something else in the code.
But regarding the reads, it's an expected behavior because when you listen to the results of a query, you are charged for a read each time a document in the result set is added or updated. And in your case, as you are setting, so the second time, the read is incremented by 1.
I created something similar to a subscription/like counter using Firebase Real-time Database and Firebase's cloud functions (using a transaction):
// This is a cloud function that increases subs by 1
export const onSubscriberCreate = functions.database
.ref('/channels/{$ch_id}/subscribers/{$uid}')
.onCreate((snapshot, context) => {
const countRef = snapshot.ref.parent.parent.child('subs_count')
return countRef.transaction(count => {
return count + 1
})
})
Then, I used FirebaseUI (FirebaseRecyclerAdapter) for Android to populate a RecyclerView of channels. When the user presses a channel's "Subscribe" button, his id is being sent to /channels/{$ch_id}/subscribers/ which triggers the cloud function.
However, the cloud function is really slow (about 5 secs), so I want to "fake" the update of the counter displayed to the user even before the cloud function is executed (I tried it by changing the TextView):
channelRef.child("subscribers")
.child(user.getUid()).setValue(true)
.addOnSuccessListener(new OnSuccessListener<Void>() {
#Override
public void onSuccess(Void aVoid) {
subsInfo.setText(channel.getSubs_count() + 1) + " SUBSCRIBERS");
}
});
The problem is that the channel object is being updated twice (the user id on the subscribers' list and the counter increased) so that the information of the channel is being downloaded again and binded to the ViewHolder even before the server updates the counter, so that's not a good solution.
I thought about moving the transaction code into the client, is it really necessary? Is there a better solution?
The best thing I feel you should do is to move the subscriber's list from inside the channel node to somewhere out. This will also make your channel object weigh lesser and you can store/update number of subscribers easily inside the channel node. Now for every user, you are downloading the entire list of users everytime you want to download a channel's information. And you don't need a cloud function to update the number of subscribers. You can do that totally on the client side using Transactions.
root/channels/{$channel}/{channelName,numberOfSubscribers,etc}
root/subscribers/{&channel}/{$userId}
This is probably how you want your data structure should be unless you really want to get all the users list. If that's the case, you can just show the size of the list of subscribers inside the TextView where you are showing the number of subscribers.
I'm trying out Firebase as my backend for a prototype app im creating.
As a mockup, im creating a fake game.
The data is pretty simple:
There are 3 lists of 'levels' common to all users, organized by difficulty ( easy, medium hard ). ( These are in a Fragment inside a ViewPager
Each level has mini-games inside of them than the users complete.
On the Android app, the user sees that list but also sees a counter of how many mini-games of that level he/she has completed.
If the user clicks on a level, he/she sees the list of mini-games and sees which of those are completed.
Im currently structuring my data as follows:
levels:{
easy:{
easy-level-1-id:{
total-mini-games:10,
easy-level-1-id:String
}
},
medium:{....},
hard:{.....}
}
user-progress:{
user-id:{
levels:{
easy-level-1-id:{
user-completed:int
}
},
mini-games:{
mini-game-1:true
mini-game-2:true
}
}
}
I have to access many places in order to check if a game has been completed, or how many games have been completed per level.
All of this is in order to avoid nesting data, as the docs recommend.
Is it better to do it like this or is it better to store every available level under every user id in order to do less calls, but having much more repeated data ?
I know about the FirebaseRecyclerViewAdapter class provided by the Firebase team, but because it takes a DatabaseReference object as an argument for knowing where the data, it doesn't really apply here, because i fetch data from different structures when building the model for the lists.
mRef1.child(some_child).addValueEventListener(new ValueEventListener(){
#Override
public void onDataChange(DataSnapshot dataSnapshot){
final Model model = dataSnapshot.getValue(Model.class);
mRef2.child(some_other_child).addValueEventListener(new ValueEventListener(){
#Override
public void onDataChange(DataSnapshot dataSnapshot){
//add more data into the model class
.....
}
#Override
public void onCancelled(DatabaseError databaseError){}
});
#Override
public void onCancelled(DatabaseError databaseError){}
});
Any pointers as to how to work with more structured data ?
FirebaseUI includes support for working with indexed data. It can load linked data from an index, such as in your mini-games node.
In your example that could work as:
keyRef = ref.child("user-progress").child(currentUser.getUid()).child("mini-games");
dataRef = ref.child("mini-games");
adapter = new FirebaseIndexRecyclerAdapter<Chat, MiniGameHolder>(
MiniGame.class,
android.R.layout.two_line_list_item,
MiniGameHolder.class,
keyRef, // The Firebase location containing the list of keys to be found in dataRef.
dataRef) //The Firebase location to watch for data changes. Each key key found at keyRef's location represents a list item in the RecyclerView.
I am using firebase geofire library to fetch key's based on location but because of thousands of keys are returned in onKeyEntered() event every time I have to take the key refer firebase and get the object back and set it to listview becoming very slow. I tried commenting all other work in onKeyEntered() to see how fast geofire is then I was surprised withing 900 milli's all callbacks received.
So now what is the best optimized way to get the data from firebase using the key passed in onKeyEntered() callback and set it to listview so that even for thousands of entries listview should load fast
I thought of AsyncTask in every callback give the fetching data work to AsyncTask and proceed with next callback key and do same, but not sure thats correct.
Or load only few and then load as scroll's is also good idea but geofire returns keys from all over the database so there is no option to get only few latest one so not sure how to implement it.
This is what I am trying but list view loads very slow.
#Override
public void onKeyEntered(String key, GeoLocation location) {
Log.d("geoevent", key);
mDatabaseTemp = FirebaseDatabase.getInstance().getReference("/posts/" + key);
mDatabaseTemp.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
Post post = new Post();
post = dataSnapshot.getValue(Post.class);
mPosts.add(post);
mPostAdapter.notifyDataSetChanged();
mRecycler.smoothScrollToPosition(mPosts.size() - 1);
}
#Override
public void onCancelled(DatabaseError databaseError) {
Toast.makeText(getActivity(), "error" + databaseError, Toast.LENGTH_LONG).show();
}
});
}
Question of the type "how to best" are notoriously difficult to answer. But I'll give a few hints about the general behavior of the system that you may not be aware of.
the Firebase Database client interacts with the network and disk on a separate thread. When data is available for your client, it calls your handlers on the main/UI thread. This means that putting your calls in an AsyncTask is only useful if you'll be doing significant work in the callback.
the Firebase Database client pipelines the calls to the server over a single connection. To read more about why that affects performance, see Speed up fetching posts for my social network app by using query instead of observing a single event repeatedly.
loading thousands of items into a mobile device over the network is in general not a good idea. You should only load data that you'll show to the user and thousands of items is well beyond what can be reasonably presented on a mobile screen. When developing a location based app, you could show a map where the user indicates a hotspot and you use Geoqueries to only request items around that spot.