I have a IOT project where multiple devices update their current locations to the IOT server. The server parses the hex data received and stores them in a MySQL database. I have a independent REST API server that queries this database to retrieve the current position of the device and display it on a map in Android.
Now because of the refresh rates of the device and the Android API request, there is a considerable latency in updating the positions on the map. How can I convert this to a real-time design, eliminating the need to read from the database but just directly sending the data to the Android client?
Note: Multiple android devices may request live updates of the same GPS device.
I am considering using firebase GCM push notifications to deliver the position to an Android/iOS device which has requested for the live view updates. However, I think this would be inefficient. As this would not be very stateless as I would have to monitor the list of devices currently requesting push notifications and do some handshaking to start/stop the notifications.
Can I use web sockets instead to make a connection between the android and the server app itself? Does the quality of the internet connection matter excessively here? I realize that somewhat like UDP for updating the position the most recent packet alone matters here and any skipped position data won't contribute much.
based on our discussion
I think GCM is a good enough option for this kind of problems.
Your total number of users and concurrent users are not that much, As I know there is a limitation for FCM that is " For each sender ID, FCM allows 1000 connections in parallel." you can find it here
The other limit is about number of stored messages at FCM. Offline users do not receive messages until the become online and the limit is 100 messages. so this can be a problem, but when the user become online FCM will send a message that can be handled in client, I mean you will informed of the situation so you can ask the server to send last update messages, but if the only last one is important for you, this is not the case and you do not have any problem, because the client will receive the last update very soon.
Anyway, based on GCM/FCM you just need to worry about sending messages when you will get updates. Consider that GCM is not about only push notifications, you can handle the messages inside BroadcastReceiver.With FCM, you can send two types of messages to clients: reference
Notification messages, sometimes thought of as "display messages."
Data messages, which are handled by the client app.
in this case you need to use data messages.
So, based on these information, I suggest following scenario :
Devices send updates to the server side.
server will send GCM messages for the interested clients
on the server side, you can handle the interested clients and you need to make it parallel.
Hope this helps you. if there is any other aspect we should consider, let me know.
thanks
Related
We're building an application server which will be used to send push notifications to android and ios clients via GCM connection server. Our plan is to use HTTP interface to connect with GCM as we only want downstream messaging. could you please tell me what is the performance of GCM for HTTP interface? I'm looking for some kind of performance data like how many push notifications messages per second GCM can support? I checked GCM site but couldn't find this. Please help to guide from where i can get such data?
as per the official Google answer
The rate limit exception code indicates that you are sending messages from a backend server too frequently. To ensure a stable service, there is a per minute / per device app upper limit on the number of messages that can be sent from a backend server. This limit is set high so most well behaving apps should not be affected, all apps should however be prepared to receive this error code.
They do not specify the "actual limit" but they do say This limit is set high so most well behaving apps should not be affected so as long as you re not spamming a single device like crazy you should be ok with whatever you send at it.
However please note that you can get throttled and have the messages be delayed
Mobile apps are updating their location to server and server responds with content.
Is it possible to use MQTT (Apache Artemis) for sending real time location updates from mobile application to server.
In this case is it required my server application (PHP) to subscribe for a topic?
Server has to handle all location updates through a single subscribed channel. This may slow down the server right?
Flow is like - Android clients sends user's location periodically (E.g. 30Sec) to server and the PHP server returns nearest users and app displays them on the map.
Here is the architecture I was planned using MQTT
Each mobile app users id will be considered as a topic so that we can send response to each user.
PHP server will considered as a topic e.g. "LOCATION_TRACKER".
All mobile apps publishes location to Server topic "LOCATION_TRACKER".
Server prepares the list and publish response to individual user's by using their id as topics.
In this archetecture the PHP server suscribes for topic "LOCATION_TRACKER" and in fact all mobile apps are publishing to a single topic.
Consider there are millions of users how to scale the PHP server or is there any other way of doing this?
The application will target at least 10 Million users. Should support half of concurrent users.
It is perfectly possible to send location updates from a mobile device to a MQTT broker, for an example look at the OwnTracks project.
For your seconds point, how a application behaves when processing a high volume of messages is entirely down to how that application architected. For high volume systems the usual pattern is to us a local queue and a thread pool to distribute the load, but it will all depend on what you need to do with the data.
I am doing a prototype that involves messaging between clients.
What I want to do - from MyApp on device1, able to send message to MyApp on device2. Device2 should receive this and show a notification.
I don't have my own application server to push notifications from GCM to GCM clients.Is this possible ? How to do it ?
What I investigated - PubNub, which has a trial license that seems to answer my need to send messages on a channel without needing a server.
This discussion at SO didn't help much.
sending client to client messages without server interaction
Can anyone suggest better ways ?
You basically have to use a server in one way or another. I think Firebase would be perfect for your purposes. It has an Android plugin you can use that will basically alert your app when something changes in the database. Here is the documentation for that feature. Take a look at this example for implementation. Before you can use the plugin, you need to create an account and whatnot (basic one is free). Here are the instructions.
PubNub Realtime Messaging and Push Notifications
(server not required)
With PubNub, you do not need your own server to do the realtime messaging or the mobile push notifications. You likely have your own server for your database (MySQL or similar, traditional RDBMS, a No SQL DB like MongoDB, or a realtime DB sync, like Firebase) to authenticate users and such.
For realtime DB or non-DB type change notifications and signaling (any type of realtime messaging), PubNub makes is super simple to receive messages in realtime with a mobile push notification fallback in one publish action. So the GCM message will be sent along with the realtime message. If the app is actively running, you get it in realtime (< 1/4 on average - typically faster) and the GCM message will be received if the app is not active (background or not started at all).
And PubNub's free plan is not a trial (anymore). It is free (including all add-ons) for as long as you stay within the free plan limits.
See the PubNub Android docs for more details and contact PubNub support if you have any further questions with getting started or getting answers to any questions you have as you progress with your app implementation.
Another possibility is to use the SMS.
Android Send and Recieve Messages
"Send Binary (Data) SMS
We can send binary messages (as opposed to text based messages that we covered earlier) to specific application ports using sendDataMessage(). According to this Stack Exchange thread data sms is one which is sent over 2G/3G as well as GSM. I’ve tested it with mobile data turned off and it works fine charging me the same amount, so not very sure on whether it uses 2G/3G or not, but generally the term data is used in telephony when it’s related to network (tcp/ip). Anyway, SMS’s are generally sent to a specific port on the device (which is probably port 0 [zero]). But using sendDataMessage() we can send SMS’s to some other random port on which our app can listen for incoming SMSs and do something with that. In this case the default messaging app will not store the SMSs in their inbox for both the sender as well as the receiver.
"
looking for links/resources and approach on connecting to server through web service API periodically in background and check for availability of updated data on the server instead of parsing server response every time.
I have a server which has some data and will be updated in a month time. I exposed a service on top of it and it is used by android app. So, to reduce server load & improve battery, I would like to call the service periodically to check latest data is available or not.
I really appreciate if anyone can help me out on this.
Your best chance would be to use GCM messages from the server, in such case a logic that could be used is the following:
Android devices registers with the GCM service and requests initial data
Server logs the last time a specific Android device has updated each data
On each data update, the server sends a message to the devices registered
Devices request new data
Server uses the last update log to send only new data and updates the update log
Another nice option, would be to use the Android Sync Adapter with longer refresh intervals.
On each response from the server, send the date of the last update
Devices will send this data along with the request for updated data i.e. /request/data?last_update=[epoch_here]
Server searches for data that are more recent than the date specified and if it doesn't find any it responds with a 304 Not Modified
You could combine the above options, for faster updates using GCM and very long intervals for cases where the device does receive the GVM message, i.e. if the device is not used for a certain period of time.
Polling periodically can cause phone to drain battery fast. Google Cloud Messaging (GCM) allows server to push notification on devices. So when the update is ready, you just tell the phone and it connect to server to grab the update.
Quoted from Android Developer Page about GCM:
This could be a lightweight message telling your app there is new data
to be fetched from the server (for instance, a movie uploaded by a
friend), or it could be a message containing up to 4kb of payload data
(so apps like instant messaging can consume the message directly).
I am developing mobile client for emailing service. One of the key features is notifications about new messages in the mailbox. As recommended by GCM architecture guidelines we are using a "Pusher" that is responsible for sending messages to the Google servers once we received a new message. The issue is that testing process has reported about serious problems with push notification delivery to devices.
So the question: is there an approaches for monitoring average statistics about push notification delivery percentage, time etc? Or maybe somebody have experience in how to set up test environment for efficient monitoring of how much notifications are getting lost during the application work?
All the "tips&tricks" related to the improving Android GCM experience are welcome.
Google claims that the processing at their GCM server takes less than a millisecond. Link below for a great video on GCM from Google's developer. And it's believable coz I could get push notifications almost instantaneously using my company's server to my device now.
http://www.youtube.com/watch?v=YoaP6hcDctM
They don't guarantee delivery, but they try for a max of 4 weeks to deliver the message depending on the duration you set in the message you send to Google's GCM servers and if you wish to let Google keep the data for eventual delivery of message to the device in case the device was offline when the message was to be delivered.
However, there are certain conditions under which the GCM messages are not delivered.
Background data is unchecked under Account and Sync settings.
Prior to 4.0.4.(ICS), a Google account on the device is a pre-requisite for GCM. Maybe, Users are not logged into their Google account.
The only way to do so is to report back to your server with the timestamp of the received push.
You can either
Report back to the server once you receive the notification in your GCM service. To implement, you will have to add a push id for your push notifications and send the id along with the push data. The client will have to get the timestamp once it receive the message and send it back along with the notification id. A simple php script can be done (when you send a push notification, you set the time of the send-notification and once it receives the device's timestamp it sets the receive-notification. This boils down to two fields in your database (marked in bold). In this approach you will probably not so much care about errors since it is very probable that the device will have a connection when it receives the notification and as such its request to your server will go through.
Keep a list of notifications received in your app and their timestamps. And when the sync is done, send the this data in your sync operation. This is ultimately the same approach but your server's data won't be as realtime as the first approach. However, the extra request is not required from the client's side but saving the received notifications and their timestamps is.
All in all, you will have to keep track of the notifications sent using a notification-id and their sending time (send-notification) and their receive time (receive-notification). A simple query will help you analyze this data.
Google has added support so that you can receive delivery receipts from Cloud Connection Server (CCS):
You can use upstream messaging to get delivery receipts (sent from CCS to your 3rd party app server) when a device confirms that it received a message sent by CCS.
To enable this feature, the message your 3rd-party app server sends to CCS must include a field called "delivery_receipt_requested". When this field is set to true, CCS sends a delivery receipt when a device confirms that it received a particular message.
https://developer.android.com/google/gcm/ccs.html#receipts
Google does not make these statistics available to you. There are some statistics available on the android developer console. This only shows the number of messages and registrations.
You would have to implement your own data collection, which could be done fairly easily. You could record the time & id of each message sent and have your android client report back to your server with the time of message receipt. You could then store the data on your server and query as needed.
Since that time Google has provided developers with advanced monitoring tool.
The Gcm Diagnostic Tool is available in Google Play developer console. Additional information is here https://support.google.com/googleplay/android-developer/answer/2663268
So you can easily track the particular message status via registration token.