I am an android user and of course I use whatsapp, twitter for android, facebook and many other apps that notify me of events.
As a proogramer whats keeps me wondering is how fast notifications or whatsapp messages arrive.
My intuition tells me that is not possible for the whatsapp or twitter server to open a TCP connection with my cellphone by a given port to deliver a new message. If i am in wifi mode the router would block that connection.
And if my whatsapp client is pooling the server every second.... Poor server if it has 1000 clients making request every second.
What is the approach to face this issue?.
Is there some other protocol involved?.
Those apps use services that utilize "long polling" - primarily based on XMPP or some variation of XMPP (like jabber - http://www.jabber.org/). The client does not poll often. A quote for the Wiki page:
The original and "native" transport protocol for XMPP is Transmission
Control Protocol (TCP), using open-ended XML streams over long-lived
TCP connections.
It sends a message to the server that basically is a mechanism for the server to send a message back at any time (as long as the client is available). It's like sending a request to an HTTP server and the server "time-out" does not occur for a very long time (hours), so the client just waits. If the server receives a message destined for the client, it sends a "response" to that request. After the time out does occur, the client sends another request and waits.
GCM does the same thing - but does not require you to setup servers for all portions of the connection. It's easy to search for GCM, AWS, etc. to see examples.
Typically GCM should be used if you dont want to guarantee immediate delivery and it is okay for your app to miss out on certain messages.
This is because GCM tries to optimize by bundling several messages (even from other apps) into a single package. And it has a limited buffer to maintain the messages per device (in case the device is not reachable).
Here is just one way to do the job.
Related
Let's say have an app that has 10s of millions of installs and 10s of thousands of active users at a given point of time. I need to log my users' activity data to my servers. Currently, I make HTTP requests from the device to my servers. I have a bunch of machines running a web server, sitting behind amazon's ELB. They parse the data coming from the devices and put it in mongodb.
Now, I would like to capture device data by using upstream CCS provided by Google' GCM (so that I can piggyback on GCM for more reliable delivery of data) I have written a prototype XMPP server and I can make whole thing work, but I am worried about scaling it up. What will happen if Google starts sending me messages at a rate faster than I can consume? Earlier, I was able to use multiple servers behind load balancer to tackle high request rate. Is there a concept of load balancing here?
If I open multiple connections from my server to Google's server (Google says I can have till 1000 connections for a given sender id), will the incoming requests be load balanced between these connections?
Finally, is there recommended solution which takes care of solving most of the problems above? Will using ejabberd solve some of the problems above?
Thanks a bunch.
What will happen if Google starts sending me messages at a rate faster than I can consume?
At the end https://developers.google.com/cloud-messaging/ccs you may read
Conversely, to avoid overloading the app server, CCS stops sending if there are too many unacknowledged messages. Therefore, the app server should "ACK" upstream messages, received from the client application via CCS, as soon as possible to maintain a constant flow of incoming messages. The aforementioned pending message limit doesn't apply to these ACKs. Even if the pending message count reaches 100, the app server should continue sending ACKs for messages received from CCS to avoid blocking delivery of new upstream messages.
In the same document, you find partial answer to your second and third questions
If at any point the connection fails, you should immediately reconnect. There is no need to back off after a disconnect that happens after authentication.
For me it means, that Google implemented a simple redundancy logic and probably not a fair load balancing system (anyway I hope so). If you have that high volumes, I suggest you to contact them directly.
For the last ones, ejabberd is a good product, there are a lot of deployed systems with a clustered infrastructure and a plenty of documents on how do taht. I suggest you to start from here http://docs.ejabberd.im/admin/guide/clustering/ .
Anyway, for your high volumes I would evaluate RabbitMQ which is another Erlang jewel.
ejabberd can be clustered and placed behind a load balancer to distribute connections. A 3 or 4 server cluster should be able to handle that load fine and give you fail over protection. You can add servers if needed. Once you get close to 10 servers you may want to consider using Redis for the in memory DB rather than mnesia.
I want to develop an app that requires near real-time communication. For efficiency and feasibility , I found that these technologies are usually used :
GCM (google keeps a socket open for messaging, and the developer's server sends messages to google's server, which then 'pushes' that message to the device)
Persistent socket- A socket, connected to developer's server is connected forever. whenever there is anything to be pushed, the server pushes it to the device.
Long polling- usually an HTTP request, but the server waits for any new message. If there is something new arrives, the server responds.
I would like to know what the battery cost for a send-to-sync message. In this architecture, a persistent socket ( usually GCM) , tells the device to 'check' for anything new from the server, ONLY when there IS something new to be checked. Then the device just polls for the new content.
My app will not have more than a 2to3 (maximum case) messages delivered to the client. So, is it a feasible thing to do ?
Could anybody tell me the approximate 'cost' of one http request ?
I'm working on app the share tasks between users. By Google Cloud Messaging, I can notify the user target that he has a new task shared. The problem is : GCM does not offer a delivery guarantee. Would someone use an app like WhatsApp if he took minutes to deliver a message or not come to hand? That's my problem with GCM.
So I've got a solution : Use Socket!!!
Using Socket.oi and Node.js, my dream came become really works like magic !!!
But as nothing is free, keep a socket connection has a very high cost for the battery. Some people argue that the use of Sockets when there is no communication, nothing in or out, no cycles, so there is no consumption.
My friends, I've read a lot of text and do not know what approach should I follow. I ask your help. Soket.oi? WebSocket ???
How to maintain a connection to my server permante preserving the most of the battery?
I appreciate everyone's help!
You need to use mix of a socket connection and GCM. Both connection types do no guarantee delivery so you need to implement mechanism which checks consistency of messages history.
Simplified scenario could look like this:
a user launches your client
the client app registers at GCM and sends google id to your server.
the client app establishes socket connection
your server sends messages to a client through GCM and socket connection (if it is establish with particular connection)
each message has unique id, therefore the client could just ignore second identical message from Google of a socket connection
About not delivered messages:
When client connects to the server through socket connection it should receive response where history of messages should be put. It shouldn't be full history, it could be just last message (in case you develop chat app). Then a client just checks if he has notified user about last message or not. If not then your client makes request(http or through socket) to your server and receives undelivered messages.
Battery consumption:
Do not acquire wake lock to maintain socket connection! A device must go sleep. GCM will wake up handset.
Socket.io is good, and certainly useful in many real-time applications, but what happens when the app is terminated by the user? Or if the user restarts their phone? How would you receive notifications then?
For all purposes, GCM is good enough.
I have a question regarding a simple messaging application which I want to write.
Lets say we have 1000 clients. Now client A writes a message to client B.
Of course, client A sends the message to a server which distributes the messages. But the next step is unclear for me:
Does the server send the message to the specific device of client B? If so, how does it address explicitly this device and not send it to the other devices as well?
Or does client B continuously check for new messages on the server which are addressed to client B?
I am not sure if idea 1 is even possible, and what is the best way. But I just want to ask for how it is normally done.
Those are both valid ways to create a chat platform. The problem with option 2, which is generally referred to as a "polling" model (where the client "polls" the server for new messages on a regular basis) is scaleability. In your example alone if you have 1000 clients you would have 1000 requests to the server for new messages every interval. To give the appearance of real time messaging this interval needs to be pretty short (i.e. a few minutes at most). If you do the math you can see that the volume of requests can balloon very quickly.
The better approach is option 1. Using something like OpenFire and the Smack API what you're describing - the server sending messages to client B - is possible. You can read more about the APIs here. The idea is the same as push notifications. Client A sends a message to the server which then will "push" that message to Client B. This is scalable and eliminates the need for polling (which kills server resources and the phone' battery with constant HTTP requests).
This is not a question for a ready-to-use solution including sources but for getting ideas/hints/tips for a solution.
Assumed I have a messenger. User A types some text that has to be sent to user B. This text is sent to a central server first where it is stored when user B is not online or where it has to be transmitted to user B immediately when he is available.
For second case, what mechanism should be used here on a mobile device?
1.) Let the messenger of user B open a client connection to the server and to permanently receive data from there does not sound good to me. When the connection is interrupted it has to be re-established - possible until next interruption. So establishing of such a connection may cause traffic and consume power without transporting any payload in between.
2.) Let the messenger use a ServerSocket and let the central "server" connect to the device has the same problems: the connection may be interrupted.
So my question: is there a mechanism available for mobile devices that transmits such messages only in case they are available and establishes a connection only when it is needed? Some kind of automated push-notification without permanent connection between client and server?
It is recomended to use GCM for Android. Here You have nice tutorial. You could also use frameworks like Parse.
Either
1. Have a persistent connection between client and the server. Client can poll at a predifined interval to check incoming payload. You may have to optimize the 'poll' logic to avoid frequent 'poll' payload
Or
Server side may push a WAP push to the client when messages are available, then the client wakes up and retreive the payload.
don't use Polling. Use Google Cloud Messaging. suseba answer references to gcm deprecated in the "Here" link.
Use GoogleCloudMessaging. comes with GooglePlayServices Library
Documentation : http://developer.android.com/google/gcm/client.html
and the source is: https://code.google.com/p/gcm/source/browse/#git%2Fgcm-client
you just need to import libraries