I've currently setup an Android and iOS app to use ActiveMQ MQTT for my message relays. It all works fine but I have the following questions that I would be grateful if anyone could help with:
1) As I understand, all clients need to keep an open TCP connection to the MQ server (for obvious reasons) and subscribe to topics and when a topic is published, the subscribed clients get a notification. If the number of clients is in the millions, it would mean millions of active open TCP connections. What are the ramifications of such number of open connections to a server? Is there a better way to handle the situation? Would you recommend instead to connect say once a minute or so, query for the "State of the world" (i.e. the latest unread messages) and disconnecting the clients? or am I being paranoid and millions of open TCP connections is fine?
2) On the subject of security, I understand that I can use SSL which is great but I still don't understand how to prevent a third party from subscribing to all topics (or even individual topics)? As far as I understand, clients need to connect to a server and subscribe to a topic but what would prevent a third party application from doing the same? How would I prevent such a thing and make sure that only clients from my app are able to connect and subscribe to their own topics?
Many thanks in advance for your help,
If you do reconnect , you will need to make sure your clients create durable subscriptions. If not, you miss out on anything that was published between connections. This can also be a scalability issue that you will need to consider. With millions of connections you will want to consider using a cluster, or some sort of topology that supports load balancing, regardless of approach you take. Constantly reconnecting clients can cause unexpected overhead as well as the Broker(s) attempt to keep up with everything that is coming and going. A big part of your decision will be based on your hardware as well as with any server.
This only scratches the surface of topologies, but here is a good starting point.
Edit: Added information to try to address your concerns on millions of connections that I meant to add the first go.
You can set up role based security via authentication plug-ins within ActiveMQ. Each role/group can be set up with a pattern using wild cards to allow access to a specific topic tree.
See this page for more details on setting this up.
Here is another site with good information that should answer your questions on how to and how it works.
Related
I've always used GCM for push notifications in android apps, but for security constraint i'm obliged to avoid GCM and use local push server, i've found a lot of answers for this question but none of them is well structured, i hope to get a clear answer for step by step process to use some XMPP server and client libraries for this topic
You may not fully understand what is GCM and what GCM was designed for.
Traditionally, each desktop applications establish its own TCP connection to its own server, just like you can connect to your Openfire instance and send/receive messages when you want without any third-party services.
This approach have many issues in mobile environment:
mobile devices are resource limited: when you have many TCP connections open it will drain your battery fast
mobile devices are not have persistent network connection: you should care about cellular and/or wifi network changes (and it also have battery impact) and you should not lose any messages in these unstable environment.
To solve these problems, Apple, Google (and also their minor competitors) provide "notification services". They are consists of:
System-wide background application ("service") which keeps connection to Apple/Google "notifications" infrastructure servers.
API for developers, to "register" their applications both on server and client side: your server-side application send all events to Apple/Google, and you client app receive "pushes" from that system service, you no need to care about network outages and battery impacts - all issues are handled by OS manufacturer.
But when you are trying to "avoid GCM" you getting all things to be cared by you, plus:
Your "handmade" persistent network service will interfere with Google's one, and will at least double network and battery usage (or even more, just because your implementation will have errors/other corner cases which are already where solved by Apple/Google big engineering teams)
Your application will dramatically more complex, and you will have much more security issues (as security issues are now handled by you, too) and poor user experience (applications with "own" networking are often have bad UX design, prompting and boring end user with network error messages, for example).
So, to sum up:
Yes, technically, you can take smack library and connect to Openfire server. Basically it the same as GCM works.
No, in 99.9999% of cases you should not go this way. Unless you are a big company which develops competing mobile platform.
I'm trying to make a mobile application and I found ionic framework which seems cool.
It seems that REST is a common way to do things and I don't understand one thing with this protocol. How does the server notify that he has data to send. If we have a text application, how does the server notify the application (without push notification, or maybe they're mandatory ?) that another client sent you a message ?
It seems like REST is only good for the requests coming from the client. Does it means that I have to send request for update every X milliseconds to have a responsive application ? Do I have to use websockets for such purpose ? I also saw Comet, is it a good way to use rest ?
The REST architecture doesn't really address "Publication/Subscription" (pub/sub) paradigms. It's much more coarse than that.
The major issues with pub/sub are technical, especially today. In a fully connected internet where everything is online at known locations all the time, then the basic REST architecture just works. Simply, the roles of Client and Server swap back and forth (i.e. the Server becomes a Client when it needs to send an notification).
But that's not the real world.
The real world is while we have a lot of connected devices, we do not have a lot of known locations. Your phone moves around all the time, and who knows what IP address it's at at any one time. DNS doesn't help because your phone is likely not registered under any particular known name. Then there's infrastructure issues where the vast majority of clients are locked away to where they can send messages, but can not receive them, even if we did know who and where they were.
So, REST doesn't explore that area simply because it typically violates a few key attributes.
One is that the URL goes away. Since we don't know where or who you are, we can't get to you by name. So, one way folks get around it is through long lived connections. Your client hooks up to the server and retains its connection so that the server can talk back. But this is an implicitly stateful connection. REST drives towards statelessness, and is effectively connectionless at the architecture level.
So, in that environment, REST is not an appropriate architecture, since the mechanics can not support some of the fundamental precepts.
We have implemented rabbitmq chat in android. but java client of rabbitmq is power hungry.is rabbitmq good for android chat?. we have used direct exchange individual queue for persons and individual routing keys. what is the best design pattern for one to one chat in rabbitmq.
and also ways to reduce battery usage
I don't think that use RabbitMQ directly for developing a chat is the better solution.
There are other solution more light please read this post about MQTT.
Another standard solution is use XMPP protocol (for example whatsup uses a custom XMPP).
We made some test and we also had some problem about battery consuming.
BTW, you can mix the technologies, mixing RabbitMQ as back-end and some other server (XMPP,MQTT) as front-end.
Read this post about the MQTT.
MQTT used by Facebook Messenger
EDIT
I would add this post MQTT vs. XMPP Which Should I Choose?
EDIT2
Please read also this and this on the official rabbitmq-group
Maybe you could combine RabbitMQ with GCM to save power as GCM gets triggered by the system and doesn´t need to keep any extra connection alive.
For example:
The app gets notified via GCM when any new event comes in.
Then a new Rabbit connection gets established, retrieving the data and timeouting after a short while again, if no messages are coming in again.
So the actual "hungryness" exists only for a short moment and only when neccessary.
You can also analyze which users are tending to always write multiple messages close behind one other and vary the timeout based on that value
I think chat applications for real time delivery is sending messages over network instantly rather than reliability. Reliability means confirm receiving against delivery. But in real time chat application does not make sure it. RabbitMQ is a message broker server and it is dedicated for reliability first, but it is not responsible for real time message communication. You may use RabbitMQ for real time communication but sometimes may not be in real time.
You can use STOMP-WebSocket(For simple messaging), Firebase(easy development for instant messaging), but most professional and usable protocol is XMPP(Extensible Messaging and Presence Protocol) for real time communication.
If you have any suggestion or query against this answer, let me comment bellow.
http://www.liveprofile.com/
I was wondering if anyone would be able to explain roughly how an application like LiveProfile is implemented?
How do the messages get pushed over the network from one device to another. How do they know the address of the device they wish to send it to? Is there a map of IPs and LiveProfile PINs?
Is it HTTP? Just the general technology used would be a great help in improving my understanding of this aspect of Android and mobile engineering.
I don't know about that specific app, but there are several possibilities out there.
One is Google Labs' C2DM: http://code.google.com/android/c2dm/
Also, check this post. It has a very detailed explanation: http://tokudu.com/2010/how-to-implement-push-notifications-for-android/
Here is another one with useful info: http://blog.boxedice.com/2010/10/07/android-push-notifications-tutorial/
As a co-founder of LiveProfile I can shed a bit more light on what we've done.
LiveProfile was released before C2DM was available. Infact the market was fragmented and many were still using Android v2.0 and below. For this reason we ended implementing our own push based solution.
Our solution is a persisted socket to our servers at all times. When a user sends a message to another user, it gets routed through our servers and we decide who it should go to. If the persisted socket is connected then we push the data to them. If the user is not currently connected, we store the data in an queue and the user will receive all the messages on connect.
Update: A good resource is a talk from Google I/O 2009 which goes into detail about the battery life, how network / CPU effects it, etc. http://www.youtube.com/watch?v=OUemfrKe65c
can anybody confirm what are the currently allowed methods for peer-to-peer communications within the Android framework? I need to transfer json strings and I'm currently using SMS which works ok but the problem is that the data also ends up as lots of text messages. I've read Reto Meier's first edition of Professional Android Application Development where he says that the data transfer options were not implemented due to security concerns.
Has this changed at all and how would you do peer-to-peer transfer of data?
Have you looked at Qualcomm's AllJoyn library? It is designed to work over Bluetooth or wifi, so may fit, though if you're connecting over 3G or wider range networks it won't work.
Given the variation and reliability of networks between two remote devices not on the same network, I'd question whether peer-to-peer is the best solution, and would venture to suggest considering using an application server in between, so you could then use Cloud to Device Messaging [deprecated] (perhaps in tandem with Google App Engine). i.e. to send a message the sender passes it to the server, and the server then passes it on to the recipient.
In theory all devices on the net have a unique IP address and can talk to each other, but it's rarely that simple as routers/firewalls are configured differently so you'd need to pay great attention to the ports you use, especially considering many inbound ports are blocked by default for security reasons.
You can simply use UDP/TCP sockets. In a separate thread you set up the server-side listener socket and that's it. Of course your application has to be started first (or should run in the background all the time). Here's an example:
http://thinkandroid.wordpress.com/2010/03/27/incorporating-socket-programming-into-your-applications/
If you also need peer discovery that will make the thing more difficult.
You should also take a look at peerdroid, an open source project available here. I've been looking in to peer communication options from the point of view of having a collection of federated devices (pre-paired, if you like, similar to Bluetooth pairing); this library looks like it may give you the foundation for what you are trying to do.
If you are on your own network (for example a home or office WiFi) then you should be able to query for other connected devices. If the network you are on is not under your control (the mobile network or a public wifi) then the network will have been configured to isolate each device from everything else. In this case you will have no choice but to put a server up to act as the man in the middle. That brings its own architectural compromises - every device has to regularly poll the server or keep a connection open - unless you use Google App Engine which supports push notifications over Google's own infrastructure.
Thanks for your answer ldx but I would need peer discovery as you indicated. Some further research appears to indicate XMPP as a suitable technology and there are now some services on offer, although these appear to be aimed at 'server' to client notifications. There is a good discussion here on XMPP and some more here although it would appear that there are still some issues to deal with such as polling v push, long-running open http connections and battery life. Xtify looks promising, especially their web service. I hope this provides suitable information to others looking at the topic of peer-to-peer data communication.