As a capstone project, my group and I are designing a couple of applications for a bus line. There will be a bus side application for the bus driver to send his location to the server, the server then sends this info to a user side app letting him know whether the bus is late or on time.
I'm handling all the server side components. I'm familiar with setting up servers on desktops, but I've yet to decide on the best approach for this since I'm a little unfamiliar with mobile application servers.
The applications are being coded with Java and using XML for parsing, so we decided on an HTTP server, but I may also code an SQL database that has a schedule of the bus arrival and departure times. I just need advice on a starting point to best approach this since all of my research hasn't given a definitive answer.
Related
As for clarification, this question is not duplicated since the
situation differs from other related questions.
We are working on an client side application which will receive data from a server side PHP-powered web application. Data are critical and must be delivered to user as soon as possible. It doesn't matter if client request for data from server or server push data to client, the only thing we need is a reliable and fast option.
There are several methods but non of them fit our project:
Use GCM push notification ability:
This is a great option but in practice, we lost several pushes so it's not reliable and in other hand, delay is so much. I repeat, the situation is critical so it must be fast.
Request data from server by the client with a 1 or 2 second interval:
This is what we think is the best solution so far but is really expensive. It's reliable and fast. But in other hand, the pressure on our disturbed servers get extremely high and they become useless even with our current client numbers. If the number of clients get larger, we'll be down.
SMS based push:
The other option for us is to send SMS to client phones and use that data to operate application. Using this method, the pressure on our server will get really low (just like GCM option). But sending SMS in our countries mobile network is usually delayed, normally, 10 seconds. Although this option have good reliability but the speed so low that we can't use it.
FM radio signal based push:
We can use clients FM radio receiver to get data from local broadcasting stations. This method is reliable and very fast but the cost of stations will kill us! and even if we handle it (read: we can't), clients does not connect their earphone to the smartphones always.
So, what are the alternatives? what is a reliable and almost fast method which does not make a lot of pressure on our servers?
Would probably recommend using WebSockets for case you describe (using OkHttp library for example) - see following for nice overview of it's use https://medium.com/#ssaurel/learn-to-use-websockets-on-android-with-okhttp-ba5f00aea988. A common pattern would be use of WebSockets with Http REST requests (for an initial catch up query for example). Also you would typically only use WebSockets while app was in foreground and rely on push notifications otherwise.
I'm trying to make a mobile application and I found ionic framework which seems cool.
It seems that REST is a common way to do things and I don't understand one thing with this protocol. How does the server notify that he has data to send. If we have a text application, how does the server notify the application (without push notification, or maybe they're mandatory ?) that another client sent you a message ?
It seems like REST is only good for the requests coming from the client. Does it means that I have to send request for update every X milliseconds to have a responsive application ? Do I have to use websockets for such purpose ? I also saw Comet, is it a good way to use rest ?
The REST architecture doesn't really address "Publication/Subscription" (pub/sub) paradigms. It's much more coarse than that.
The major issues with pub/sub are technical, especially today. In a fully connected internet where everything is online at known locations all the time, then the basic REST architecture just works. Simply, the roles of Client and Server swap back and forth (i.e. the Server becomes a Client when it needs to send an notification).
But that's not the real world.
The real world is while we have a lot of connected devices, we do not have a lot of known locations. Your phone moves around all the time, and who knows what IP address it's at at any one time. DNS doesn't help because your phone is likely not registered under any particular known name. Then there's infrastructure issues where the vast majority of clients are locked away to where they can send messages, but can not receive them, even if we did know who and where they were.
So, REST doesn't explore that area simply because it typically violates a few key attributes.
One is that the URL goes away. Since we don't know where or who you are, we can't get to you by name. So, one way folks get around it is through long lived connections. Your client hooks up to the server and retains its connection so that the server can talk back. But this is an implicitly stateful connection. REST drives towards statelessness, and is effectively connectionless at the architecture level.
So, in that environment, REST is not an appropriate architecture, since the mechanics can not support some of the fundamental precepts.
I've currently setup an Android and iOS app to use ActiveMQ MQTT for my message relays. It all works fine but I have the following questions that I would be grateful if anyone could help with:
1) As I understand, all clients need to keep an open TCP connection to the MQ server (for obvious reasons) and subscribe to topics and when a topic is published, the subscribed clients get a notification. If the number of clients is in the millions, it would mean millions of active open TCP connections. What are the ramifications of such number of open connections to a server? Is there a better way to handle the situation? Would you recommend instead to connect say once a minute or so, query for the "State of the world" (i.e. the latest unread messages) and disconnecting the clients? or am I being paranoid and millions of open TCP connections is fine?
2) On the subject of security, I understand that I can use SSL which is great but I still don't understand how to prevent a third party from subscribing to all topics (or even individual topics)? As far as I understand, clients need to connect to a server and subscribe to a topic but what would prevent a third party application from doing the same? How would I prevent such a thing and make sure that only clients from my app are able to connect and subscribe to their own topics?
Many thanks in advance for your help,
If you do reconnect , you will need to make sure your clients create durable subscriptions. If not, you miss out on anything that was published between connections. This can also be a scalability issue that you will need to consider. With millions of connections you will want to consider using a cluster, or some sort of topology that supports load balancing, regardless of approach you take. Constantly reconnecting clients can cause unexpected overhead as well as the Broker(s) attempt to keep up with everything that is coming and going. A big part of your decision will be based on your hardware as well as with any server.
This only scratches the surface of topologies, but here is a good starting point.
Edit: Added information to try to address your concerns on millions of connections that I meant to add the first go.
You can set up role based security via authentication plug-ins within ActiveMQ. Each role/group can be set up with a pattern using wild cards to allow access to a specific topic tree.
See this page for more details on setting this up.
Here is another site with good information that should answer your questions on how to and how it works.
We have implemented rabbitmq chat in android. but java client of rabbitmq is power hungry.is rabbitmq good for android chat?. we have used direct exchange individual queue for persons and individual routing keys. what is the best design pattern for one to one chat in rabbitmq.
and also ways to reduce battery usage
I don't think that use RabbitMQ directly for developing a chat is the better solution.
There are other solution more light please read this post about MQTT.
Another standard solution is use XMPP protocol (for example whatsup uses a custom XMPP).
We made some test and we also had some problem about battery consuming.
BTW, you can mix the technologies, mixing RabbitMQ as back-end and some other server (XMPP,MQTT) as front-end.
Read this post about the MQTT.
MQTT used by Facebook Messenger
EDIT
I would add this post MQTT vs. XMPP Which Should I Choose?
EDIT2
Please read also this and this on the official rabbitmq-group
Maybe you could combine RabbitMQ with GCM to save power as GCM gets triggered by the system and doesn´t need to keep any extra connection alive.
For example:
The app gets notified via GCM when any new event comes in.
Then a new Rabbit connection gets established, retrieving the data and timeouting after a short while again, if no messages are coming in again.
So the actual "hungryness" exists only for a short moment and only when neccessary.
You can also analyze which users are tending to always write multiple messages close behind one other and vary the timeout based on that value
I think chat applications for real time delivery is sending messages over network instantly rather than reliability. Reliability means confirm receiving against delivery. But in real time chat application does not make sure it. RabbitMQ is a message broker server and it is dedicated for reliability first, but it is not responsible for real time message communication. You may use RabbitMQ for real time communication but sometimes may not be in real time.
You can use STOMP-WebSocket(For simple messaging), Firebase(easy development for instant messaging), but most professional and usable protocol is XMPP(Extensible Messaging and Presence Protocol) for real time communication.
If you have any suggestion or query against this answer, let me comment bellow.
What's the best way to send constantly updating data from a server (over a REST API or a socket) to an iOS or Android app? Should I create a socket connection and have a socket server that pumps out data, or should I have the app periodically poll a backend resource? Thanks!
Should I create a socket connection and have a socket server that
pumps out data
This is an option. I know some top apps on the play store that use web sockets for streaming data.
Have the app periodically poll a backend resource
I would recommend against this. Polling drains the battery. The android OS will keep the CPU in idle mode at times and constant polling can wake the CPU up and cause a drain in battery. Users wont like that. You are also wasting processing power when there are no results from the server.
The easiest way to send push notifications / minor updates to the android phone is via GCM. The GCM messages are delivered in near real-time (I noticed a lag of about a second for my apps). The payload is limited to 4k and the messages are stored for up to 4 weeks. This is another option you can consider based on your use case.
The typical model I follow is to hit the server on a specified interval and download the updated data.
Polling is bad, it drains out the battery
If you want to update the screen when the application is in the foreground, it is better to go with Socket Connection. You can see the changes in realtime.
For this, you need to create a socket server and open a connection between the device and the server.
https://www.raywenderlich.com/3437391-real-time-communication-with-streams-tutorial-for-ios
I would suggest to use MQTT protocol to publish and subscribe data. Libraries for iOS(Moscapsule) and Android(Paho) for integrating MQTT protocol are available and can easily be integrated in mobile apps.
Backend needs to be configured to support MQTT also. Detail information about how MQTT works can be found here.
I am going to paste my accepted answer from a similar question that was posted a few hours ago.
It can be done using sockets or polling the server but I wouldn't recommend either for a production level app. You will eventually need security features and as a frontend developer, it would be difficult to start from scratch.
I would suggest using a third party service. Take a look at Space Cloud. It is an open source alternative to Firebase where you can use your own database and has built in security features. If you want to go deeper, it uses Apache Kafka for realtime database features.
If those don't suit you then the best option would be to go with GCM.
should I have the app periodically poll a backend resource?
this must be the last option to put in consideration. as long as the client number increases you need to scale out backend/server and this is not as simple as putting an another server next to existing one.
the perfect solution will be a push oriented design and best approach will be to register clients to events and server will push clients if that specific events occur.
you need to be more specific about your business case. you had mentioned real-time in your question but what kind of real-time is that. is it a video streaming, chat application or just ordinary rest calls?
Typically, you'll want your server to push updates to the clients because polling is expensive, may not scale well with N number of clients, and will drain your phone batteries faster. Depending on your server technology:
.NET/Core:
May want to take a look at SignalR to push notifications from the backend to your client apps.
Java:
May want to take a look at Atmosphere to push notifications from the backend to your client apps.
I have used SignalR with good results publishing to phone clients. Have not used Atmosphere, but it has good reviews.