Mobile apps are updating their location to server and server responds with content.
Is it possible to use MQTT (Apache Artemis) for sending real time location updates from mobile application to server.
In this case is it required my server application (PHP) to subscribe for a topic?
Server has to handle all location updates through a single subscribed channel. This may slow down the server right?
Flow is like - Android clients sends user's location periodically (E.g. 30Sec) to server and the PHP server returns nearest users and app displays them on the map.
Here is the architecture I was planned using MQTT
Each mobile app users id will be considered as a topic so that we can send response to each user.
PHP server will considered as a topic e.g. "LOCATION_TRACKER".
All mobile apps publishes location to Server topic "LOCATION_TRACKER".
Server prepares the list and publish response to individual user's by using their id as topics.
In this archetecture the PHP server suscribes for topic "LOCATION_TRACKER" and in fact all mobile apps are publishing to a single topic.
Consider there are millions of users how to scale the PHP server or is there any other way of doing this?
The application will target at least 10 Million users. Should support half of concurrent users.
It is perfectly possible to send location updates from a mobile device to a MQTT broker, for an example look at the OwnTracks project.
For your seconds point, how a application behaves when processing a high volume of messages is entirely down to how that application architected. For high volume systems the usual pattern is to us a local queue and a thread pool to distribute the load, but it will all depend on what you need to do with the data.
Related
I have a IOT project where multiple devices update their current locations to the IOT server. The server parses the hex data received and stores them in a MySQL database. I have a independent REST API server that queries this database to retrieve the current position of the device and display it on a map in Android.
Now because of the refresh rates of the device and the Android API request, there is a considerable latency in updating the positions on the map. How can I convert this to a real-time design, eliminating the need to read from the database but just directly sending the data to the Android client?
Note: Multiple android devices may request live updates of the same GPS device.
I am considering using firebase GCM push notifications to deliver the position to an Android/iOS device which has requested for the live view updates. However, I think this would be inefficient. As this would not be very stateless as I would have to monitor the list of devices currently requesting push notifications and do some handshaking to start/stop the notifications.
Can I use web sockets instead to make a connection between the android and the server app itself? Does the quality of the internet connection matter excessively here? I realize that somewhat like UDP for updating the position the most recent packet alone matters here and any skipped position data won't contribute much.
based on our discussion
I think GCM is a good enough option for this kind of problems.
Your total number of users and concurrent users are not that much, As I know there is a limitation for FCM that is " For each sender ID, FCM allows 1000 connections in parallel." you can find it here
The other limit is about number of stored messages at FCM. Offline users do not receive messages until the become online and the limit is 100 messages. so this can be a problem, but when the user become online FCM will send a message that can be handled in client, I mean you will informed of the situation so you can ask the server to send last update messages, but if the only last one is important for you, this is not the case and you do not have any problem, because the client will receive the last update very soon.
Anyway, based on GCM/FCM you just need to worry about sending messages when you will get updates. Consider that GCM is not about only push notifications, you can handle the messages inside BroadcastReceiver.With FCM, you can send two types of messages to clients: reference
Notification messages, sometimes thought of as "display messages."
Data messages, which are handled by the client app.
in this case you need to use data messages.
So, based on these information, I suggest following scenario :
Devices send updates to the server side.
server will send GCM messages for the interested clients
on the server side, you can handle the interested clients and you need to make it parallel.
Hope this helps you. if there is any other aspect we should consider, let me know.
thanks
I have an Android App development for story Maker.
The very first time Registered user logged in our app and create the story and download an apk .
He now uploaded the apk file in play store
The end user download the story now.
When the Registered user logged again updating his App with one more story.
This time he will not upload the app into play store Google.
I want how to update the new contents to the end user from registered user without uploading the App?
If we are going to save the content in server and serve to the end user then what is the process to do this and what are the possible way to do this ?
What you need is a combination of real time messaging, and notifications. Create a server(node or php) that maintains a state listener ,this state listener as hook to any connected device(android or ios). Whenever user performs a subscribed action such as (When the Registered user logged again updating his App) use real time communication to relay this to the server that watches your changes.(Use socket.io or lightStreamer even crossbar.io is perfect). Then intercept those changes with static notifications for every update received, Use fcm(fire-base cloud messaging) this way every data will be always be delivered and synced between clients accordingly.
This is light Streamer.
Optimized data streaming for web and mobile.
Lightstreamer enables several forms of real-time messaging. It is flexible enough to be used in any scenario, including mission critical applications.
► Real-time Data Push and WebSockets
► In-App Messaging and Push Notifications
► Pub-sub with fan-out broadcasting and one-to-one messaging
► Firewall and proxy friendly
► Adaptive bandwidth throttling
This is socket.io
Socket.IO enables real-time bidirectional event-based communication. It works on every platform, browser or device, focusing equally on reliability and speed. Socket.IO is built on top of the WebSockets API(Client side) and Node.js.
This is crossbar.io
Crossbar.io is a networking platform for distributed and microservice applications, implementing the open Web Application Messaging Protocol (WAMP). It is feature rich, scalable, robust and secure. Let Crossbar.io take care of the hard parts of messaging so you can focus on your app's features.
This is firebase cloud Messaging.
Using FCM, you can notify a client app that new email or other data is available to sync. You can send notification messages to drive user reengagement and retention. For use cases such as instant messaging.
All the aforementioned is primarily based on javascript.For php consider this library instead.
This PHP Ratchet.
Ratchet is a loosely coupled PHP library providing developers with tools to create real time, bi-directional applications between clients and servers over WebSockets.
In gerneral, WebSockets is an advanced technology that makes it possible to open an interactive communication session between client and a server. With this API, you can send messages to a server and receive event-driven responses without having to poll the server for a reply.
Happy Coding #.
In my project I have to send user's location periodically (E.g. 30Sec) to server using Android application and the PHP server returns nearest users and app displays them on the map. I am looking for a publish subscribe protocol for the real time data transfer.
Here is the architecture I was planned. Each mobile app users id will be considered as a topic or channel so that we can send one to one message. PHP server will considered as a topic e.g. "APP_SERVER_TRACKER". All mobile apps publishes location to Server topic. Server prepares the list and publish response to individual user's by using their id as topics.
I checked Pusher API and I would like to know how my PHP server will get user's location. Is it possible to subscribe all channels at server?
Pusher can handle higher number of users by scaling, but all user's are connecting to my PHP Server through a single channel and the server then publishing response to end user. In this scenario do I need to scale my PHP Server? Is Pusher is good choice in this scenario? Please advice?
The application will target at least 10 Million users. should support half of concurrent users.
PHP Server Hosting will be in AWS EC2 m3.2xlarge 8 core instance
looking for links/resources and approach on connecting to server through web service API periodically in background and check for availability of updated data on the server instead of parsing server response every time.
I have a server which has some data and will be updated in a month time. I exposed a service on top of it and it is used by android app. So, to reduce server load & improve battery, I would like to call the service periodically to check latest data is available or not.
I really appreciate if anyone can help me out on this.
Your best chance would be to use GCM messages from the server, in such case a logic that could be used is the following:
Android devices registers with the GCM service and requests initial data
Server logs the last time a specific Android device has updated each data
On each data update, the server sends a message to the devices registered
Devices request new data
Server uses the last update log to send only new data and updates the update log
Another nice option, would be to use the Android Sync Adapter with longer refresh intervals.
On each response from the server, send the date of the last update
Devices will send this data along with the request for updated data i.e. /request/data?last_update=[epoch_here]
Server searches for data that are more recent than the date specified and if it doesn't find any it responds with a 304 Not Modified
You could combine the above options, for faster updates using GCM and very long intervals for cases where the device does receive the GVM message, i.e. if the device is not used for a certain period of time.
Polling periodically can cause phone to drain battery fast. Google Cloud Messaging (GCM) allows server to push notification on devices. So when the update is ready, you just tell the phone and it connect to server to grab the update.
Quoted from Android Developer Page about GCM:
This could be a lightweight message telling your app there is new data
to be fetched from the server (for instance, a movie uploaded by a
friend), or it could be a message containing up to 4kb of payload data
(so apps like instant messaging can consume the message directly).
I have an android app which i connect to my server using REST API (django rest framework)
here is a scenario(and maybe my plan):
data is sent back and forth as json
I have a user model and a task model where users are owners of some task.
Users typicaly sends a task to another user (with json similar to this: {"owner": "exampleuser", "from":"otheruser", "content":"example" ...} using POST method)
-The tasks has a boolean field "completed" and is deleted once the task is completed (using PUT or PATCH method: completed = true,)
once a new task gets created using POST method, the only way users can see any activities concerning their tasks is through an android activity that uses GET method to get a list of all the tasks owned by the user, by looking up all objects owned by the user
So my questions are:
Instead of having the user to check the app everytime. How can I use GCM to push notify the user?
How will it tell which user or device to send the push notification to?
How does GCM knows when there has been a change to tasks or when a user POST a task?
Android keeps one active connection to Google's servers, but it doesn't use much power or data, because no traffic is sent along it until something sends a GCM message to an app on your phone. There's only one connection on the phone, used by all apps: installing a new app that uses GCM doesn't add any extra load.
The first step in GCM is that a third-party server (such as an email server) sends a request to Google's GCM server. This server then sends the message to your device, through that open connection. The Android system looks at the message to determine which app it's for, and starts that app. The app must have registered with Android to use GCM, and it must have the relevant permission. When the app starts, it might create a notification straight away with the data from the message. GCM messages are very limited in size, so the app might instead open a normal connection to the third-party server to get more information (for example, downloading the headers of new emails).
The advantage of using push notifications is that apps don't have to run at regular intervals to check for new data, saving both power and data. The advantage of having a centralized mechanism like GCM is that the device only needs one open network connection and the Android GCM system is the only thing that needs to keep running, rather than each app having to stay running in the background to keep its own network connection to its own server.
As per the GCM implementation, it requires that you implement a remote server which will manage all the requests, both incoming and outgoing. You can do this simply with a web server acting as a webservice, which will get (for instance) the requests from the clients with a HTTP POST request, and process act consequently.
Instead of having the user to check the app everytime. How can I use GCM to push notify the user?
This will be managed by the server that I just described. It will know who is subscribed and who should receive the notifications.
How will it tell which user or device to send the push notification to?
Same goes here. The server, upon a subscription, should store the users in some kind of storage (a SQL database, for instance), and this way it will know who to send notifications. This implies you'll have to implement some timeout mechanism. In my case, I make clients send a dummy HTTP POST every 30 seconds. If I don't get this request from a reasonable amount of time, I consider the client timed-out and therefore I remove them from the database.
How does GCM knows when there has been a change to tasks or when a user POST a task?
Same story, it's all handled by the server. You implement the logic of what should happen upon any request.
You might want to read this: How to send location of the device on server when needed
And also this references:
Reference on Google Cloud Messaging
Android Push Notifications using Google Cloud Messaging GCM - Android Example
Google Cloud Messaging using PHP
Connection between PHP (server) and Android (client) Using HTTP and JSON
Notificaciones Push Android: Google Cloud Messaging (GCM). Implementación Cliente (Nueva Versión) (spanish)