I can't find documentation on best practices for the maximum number of messages one should send to the Firebase database (or one like it) over a period of time, like one second, and also what rate an app could handle receiving without slowing down significantly. For example:
//send updated location of user character in MMORG
MyDatabaseReference.child(LOCATIONS).child(charid).setValue . . .
//recieive locations of other characters in a MMORG
MyDatabaseReference.child(LOCATIONS).addValueEventListener(new
ValueEventListener() { . . .
In testing, 3 devices each sending 20 messages per second to the database, and each receiving 60 messages per second, appears to work OK (S8 used, a fast device). I was wondering what would happen with, say, 100 devices, in which case each user app would be getting 2000 messages per second theoretically. I imagine there is some automatic throttling of this.
As mentioned in Firebase officil documentation regarding Firebase database limits, there is a maximum of 1000 write operations/second for the free plan.
If you want to stay on the free plan, remember that when you'll reach the maximum number of writes per second, it doesn't mean that you'll not be able to use Firebase database anymore. When 1001th simultaneous connection occurs, a queue of operations is created and Firebase will wait until one connection is closed, and than it uses your new connection.
Related
I Will describe my use case with figures attached and everything i can do to be clear so we may get up with a better idea ...
General idea:
Whatsapp chat application with firebase
Use case:
As we know one of the features of whatsapp is the last seen, when the user did (exited the app, log out, lost wifi connection, etc ..)
I tried to use:
onDisconnect, but onDisconnect gives bad result when losing wifi connection (because of the socket latency to be timed out)
the one I am using now, is every user updates its timestamp every 3 seconds (update document every 3 seconds), when the user loses connection,
he won't be able to update his timestamp, right? So, if another user wants to chat with this offline user, I can show him the user's last seen. Hope this
this is clear...
Developed using Flutter framework
Redux to manage app state
Firebase, cloud firestore
The code below is dispatching an action every three seconds, this dispatched action will update the last seen in firebase...
timer = Timer.periodic(Duration(seconds: 3), (Timer t) {
// store.dispatch(updateUserOnline());
});
As you can see in the figure below my data structure of how I am updating last seen for this user every 3 seconds ...
This implementation is very expensive to get satisfactory results for a last seen for a user, if we have million users and these million users are updating their
last seen every 3 seconds it will cost a lot $ per month, as we are doing a write operation, no?
So, my other solution is to implement a socket connection to my own server and let all the users listen to the onDisconnect socket event on my server instead of Firebase server, is this doable to avoid the huge cost of writing operations?
image attached: here
Firebase writes would indeed be a bit costlier since you would be sending in a lot of writes, which, apparently are just for the job of "last seen".
Instead, as you mentioned, having a socket connection with your own server will help reduce the number of queries you make. i.e. As soon as the socket disconnects from the server, you can send a write operation to Firebase. "Every 3 seconds vs Only when the user disconnects".
Plus (not something that you asked for), if you would be setting up a socket server of your own, then it shall help in the following scenarios as well:
Typing events (The indication we get when the other person is typing a message)
Quicker way to know if the person at the other end is online/offline (because of sockets)
I'm developing a Chat style application that could potentially have tons of users chatting at the same time. I'm retrieving a list of chat-rooms when I enter on my Home screen and then adding an SnapshotListenerto each one of the chat-rooms to listen the messages in real time. This can grow exponentially and I can end up having hundred or even thousands of SnapshotListener active at the same time on the client side.
My question is, do this affect performance? I mean, regarding Firebase library, it has any kind of limit or performance issue when having that many listeners active at once? Of course not everyone is gonna be chatting at the same time, so even if there are 1000 open listeners, just 5-10 would be chatting and sending callbacks at once.
According to the Best Practices section of the official Firestore documentation:
You should always try to keep the number of snapshot listeners per
client under 100.
Also you should keep in mind some more generic limits and quotas such as:
The maximum concurrent connections for mobile/web clients per database
= 1,000,000
I am planning to use Google's FCM messaging for sending Real Time Alerts to my Tsunami Alerter app.
Wanted to know whether if there are 100,000 users of my app they would be able to receive instant alerts within say 3-5 minutes? (I heard FCM has limits) but I am still not able to comprehend what are those limits are!
Can anyone please explain me further? (100,000 messages to 100,000 different users within 5 minutes)
Whether Parallel connections limit like 1000 users will come into picture if I use a Free Plan?
Can anyone please explain me further? (100,000 messages to 100,000 different users within 5 minutes)
Typically you get a push message within seconds but of course there are other factors such as internet connection and device power profiles.
Whether Parallel connections limit like 1000 users will come into picture if I use a Free Plan?
There is no such thing as a "Free Plan" since FCM is free. Other Firebase services you have to pay for.
All the 1000 connections thing means is that you can send a max of 1000 messages in a single request so you have to split them up
I am working on a restaurant project which sends the order of the customer from a android tablet to the kitchen, I want my application to wait for a specific time before sending the order to the kitchen if the server already containing more than 10 orders. In short I want my application to read the number of orders and wait, Kindly guide me if it is possible.
To wait big intervals of time I recommend you to use AlarmManager
http://developer.android.com/reference/android/app/AlarmManager.html
The pseudocode for the problem would be something like this:
1 Request to check number of orders
2a If <= 10, send the order
2b If not create an Alarma to wait X minutes and return to 1
Anyway, this has a lot of problems like what happends if a lot of tablet are waiting to send the order, how can the tablets sort the orders? One order can be waiting infinite time because other orders are scheluded before always....
In my opinion you should save all the orders in the server and manage the orders from there. That is the best and simple solution. Then in your server you can always show to the kitchen only 10 orders and manage what order show when one is finished and given to the client. If not, you will need a very complex system to comunicate between the tablets, but for me has no sense the cost of do that
We are building an application which requires good amount of data exchanges between different users. We are using SQLite to store the info and Rest api to exchange data with server.
To ensure high performance and less CPU /memory hogging but to also maintain good user experience we need following suggestions:
1 We tried running sync at frequency of 30 seconds but it hogs resources.Is there any client side framework which can be used to sync sqlite with MySQL or we have to only plan all possible events for same
2 How does applications like Gmail /twitter work- do they sync only on demand or keep on syncing in background. I feel it is on demand but not sure.
3 Notifications should be server side or client side (based on updates in sqlite). In whatsapp I observed it is client side only. If I do not click a received message I keep on getting the notification about same
4 IF we keep notifications server side and sync on demand basis. then on clicking a new notification when app will open up at that time should we make a sync call
Need an expert opinion that such applications should be designed to manage sync and notifications in such a way that it does not hogs resources and also gives online kind of experience to customer
Your question is pretty broad, but I'll at least give you a direction to start.
I've run local databases in iOS and Android that are over 100 MB without incident. SQLite should never be your problem, if you use it correctly. Even with 100,000 rows of data, it is fast and efficient. Where people get into trouble is by not properly indexing the data or over-normalizing the data. A quick search can tell you how to use indexes to optimize your queries, so I won't go into that any further. Over-normalization seems to be not fully understood, so I'll go into a bit more depth on it.
When designing a server database, it is important to minimize the amount of duplicate data. This often is done by breaking up a single record into multiple tables and using foreign keys. On a 1 GB database, this type of data normalization may save 20%, which is fairly significant. This gain in storage comes at the cost of perform. Sequential lookups and joins are frequently necessary to get complete data. On a server, there are plenty of CPU cycles and memory, and no one really notices if a request takes an extra millisecond or two.
A mobile app is not a full database server. The user is actively staring at the screen waiting for the app to respond. Additionally, the CPU and memory available are minimal, which makes that delay take even longer. To add insult to injury, mobile databases are only a small portion the size of a server database and duplicate data is already pretty minimal. The same data normalization that may have saved 200 MB (20% of 1 GB) server size, may now only save 5% of 10 MB, or 500 KB. That minor gain is not worth the effort or performance hit.
When you sync, you do not need a full data set each time. You only need to get data that has changed since the last sync. In many cases, that will be no change at all. You should find a way to identify what the device has on it now and only get the changes.
It is important that the UI does not stall waiting for the network request. All network activity should be done on a background thread and notify the UI to refresh once the sync completes.
Lastly, I'll mention that SQLite is NOT thread safe. It is important to limit concurrency with your database access.