I have a question regarding a simple messaging application which I want to write.
Lets say we have 1000 clients. Now client A writes a message to client B.
Of course, client A sends the message to a server which distributes the messages. But the next step is unclear for me:
Does the server send the message to the specific device of client B? If so, how does it address explicitly this device and not send it to the other devices as well?
Or does client B continuously check for new messages on the server which are addressed to client B?
I am not sure if idea 1 is even possible, and what is the best way. But I just want to ask for how it is normally done.
Those are both valid ways to create a chat platform. The problem with option 2, which is generally referred to as a "polling" model (where the client "polls" the server for new messages on a regular basis) is scaleability. In your example alone if you have 1000 clients you would have 1000 requests to the server for new messages every interval. To give the appearance of real time messaging this interval needs to be pretty short (i.e. a few minutes at most). If you do the math you can see that the volume of requests can balloon very quickly.
The better approach is option 1. Using something like OpenFire and the Smack API what you're describing - the server sending messages to client B - is possible. You can read more about the APIs here. The idea is the same as push notifications. Client A sends a message to the server which then will "push" that message to Client B. This is scalable and eliminates the need for polling (which kills server resources and the phone' battery with constant HTTP requests).
Related
Let's say have an app that has 10s of millions of installs and 10s of thousands of active users at a given point of time. I need to log my users' activity data to my servers. Currently, I make HTTP requests from the device to my servers. I have a bunch of machines running a web server, sitting behind amazon's ELB. They parse the data coming from the devices and put it in mongodb.
Now, I would like to capture device data by using upstream CCS provided by Google' GCM (so that I can piggyback on GCM for more reliable delivery of data) I have written a prototype XMPP server and I can make whole thing work, but I am worried about scaling it up. What will happen if Google starts sending me messages at a rate faster than I can consume? Earlier, I was able to use multiple servers behind load balancer to tackle high request rate. Is there a concept of load balancing here?
If I open multiple connections from my server to Google's server (Google says I can have till 1000 connections for a given sender id), will the incoming requests be load balanced between these connections?
Finally, is there recommended solution which takes care of solving most of the problems above? Will using ejabberd solve some of the problems above?
Thanks a bunch.
What will happen if Google starts sending me messages at a rate faster than I can consume?
At the end https://developers.google.com/cloud-messaging/ccs you may read
Conversely, to avoid overloading the app server, CCS stops sending if there are too many unacknowledged messages. Therefore, the app server should "ACK" upstream messages, received from the client application via CCS, as soon as possible to maintain a constant flow of incoming messages. The aforementioned pending message limit doesn't apply to these ACKs. Even if the pending message count reaches 100, the app server should continue sending ACKs for messages received from CCS to avoid blocking delivery of new upstream messages.
In the same document, you find partial answer to your second and third questions
If at any point the connection fails, you should immediately reconnect. There is no need to back off after a disconnect that happens after authentication.
For me it means, that Google implemented a simple redundancy logic and probably not a fair load balancing system (anyway I hope so). If you have that high volumes, I suggest you to contact them directly.
For the last ones, ejabberd is a good product, there are a lot of deployed systems with a clustered infrastructure and a plenty of documents on how do taht. I suggest you to start from here http://docs.ejabberd.im/admin/guide/clustering/ .
Anyway, for your high volumes I would evaluate RabbitMQ which is another Erlang jewel.
ejabberd can be clustered and placed behind a load balancer to distribute connections. A 3 or 4 server cluster should be able to handle that load fine and give you fail over protection. You can add servers if needed. Once you get close to 10 servers you may want to consider using Redis for the in memory DB rather than mnesia.
I am an android user and of course I use whatsapp, twitter for android, facebook and many other apps that notify me of events.
As a proogramer whats keeps me wondering is how fast notifications or whatsapp messages arrive.
My intuition tells me that is not possible for the whatsapp or twitter server to open a TCP connection with my cellphone by a given port to deliver a new message. If i am in wifi mode the router would block that connection.
And if my whatsapp client is pooling the server every second.... Poor server if it has 1000 clients making request every second.
What is the approach to face this issue?.
Is there some other protocol involved?.
Those apps use services that utilize "long polling" - primarily based on XMPP or some variation of XMPP (like jabber - http://www.jabber.org/). The client does not poll often. A quote for the Wiki page:
The original and "native" transport protocol for XMPP is Transmission
Control Protocol (TCP), using open-ended XML streams over long-lived
TCP connections.
It sends a message to the server that basically is a mechanism for the server to send a message back at any time (as long as the client is available). It's like sending a request to an HTTP server and the server "time-out" does not occur for a very long time (hours), so the client just waits. If the server receives a message destined for the client, it sends a "response" to that request. After the time out does occur, the client sends another request and waits.
GCM does the same thing - but does not require you to setup servers for all portions of the connection. It's easy to search for GCM, AWS, etc. to see examples.
Typically GCM should be used if you dont want to guarantee immediate delivery and it is okay for your app to miss out on certain messages.
This is because GCM tries to optimize by bundling several messages (even from other apps) into a single package. And it has a limited buffer to maintain the messages per device (in case the device is not reachable).
Here is just one way to do the job.
I have a server with sql database.
Also have about 100k users on android application.
What I need now is to send immediately notifications from the server to all devices.
Im researching the GCM system but as I see there`s a huge delay on the receiving side.
What I need is when I click the send button on my server,everyone device to receive it in a few seconds.
Is the delay only happening when using the HTTP connection?
Is it going to be different with the XMPP connection ?
You are trying to broadcast a message to nearly 100k users and currently xmpp downstream messaging does not support broadcasting. Use http server to send message to 1000 devices at a time. This can be improved by using multi curl. see this https://github.com/mseshachalam/GCMMessage-MultiCURL
In general the GCM is the right choice for massive broadcasting.
On the other hand the messages are not guaranteed to be delivered immediately, the delay might be up to 25(!) minutes given, that all devices have your app up and running.
See Google Cloud Messaging - messages either received instantly or with long delay for explanations why
This is not a question for a ready-to-use solution including sources but for getting ideas/hints/tips for a solution.
Assumed I have a messenger. User A types some text that has to be sent to user B. This text is sent to a central server first where it is stored when user B is not online or where it has to be transmitted to user B immediately when he is available.
For second case, what mechanism should be used here on a mobile device?
1.) Let the messenger of user B open a client connection to the server and to permanently receive data from there does not sound good to me. When the connection is interrupted it has to be re-established - possible until next interruption. So establishing of such a connection may cause traffic and consume power without transporting any payload in between.
2.) Let the messenger use a ServerSocket and let the central "server" connect to the device has the same problems: the connection may be interrupted.
So my question: is there a mechanism available for mobile devices that transmits such messages only in case they are available and establishes a connection only when it is needed? Some kind of automated push-notification without permanent connection between client and server?
It is recomended to use GCM for Android. Here You have nice tutorial. You could also use frameworks like Parse.
Either
1. Have a persistent connection between client and the server. Client can poll at a predifined interval to check incoming payload. You may have to optimize the 'poll' logic to avoid frequent 'poll' payload
Or
Server side may push a WAP push to the client when messages are available, then the client wakes up and retreive the payload.
don't use Polling. Use Google Cloud Messaging. suseba answer references to gcm deprecated in the "Here" link.
Use GoogleCloudMessaging. comes with GooglePlayServices Library
Documentation : http://developer.android.com/google/gcm/client.html
and the source is: https://code.google.com/p/gcm/source/browse/#git%2Fgcm-client
you just need to import libraries
i want a two communication between server and Android. From Android want to send my current location coordinates to server after each 10 minutes. i can send message or images to server at any time. Similarly from server i want to send data to Android whenever needed.
What should be the architecture of my application.
Communication from Android to server and Server to Android is independent of each other i should not ask it as a two way communication but infact i want communication from both sides to each other any time.
Should i use a Web service of any kind or just network sockets or Something else.
You can use websockets or...
Simply send data from phone to the server using normal request. When you want send something from server to telephone you can use push notifications (C2DM) and if you receive such push message you know that server has data for you and telephone can download it using normal request;-). It depends what data you want to send. Sometimes this approach will be good, sometimes it's better to use websockets, TCP sockets or even XMPP protocol ;-)
You can Use Acknowledge for the same. You need to Implement ACK/NACK Logic in your code. When you are sending Data just wait for ACK for a particular time period. If ACK is not received the you need to send NACK for the same. Here you need to use a session-id ( a kind of id for communication at both side, which can be any random number ).
You can use this Logic at both the side.