In my project I have to send user's location periodically (E.g. 30Sec) to server using Android application and the PHP server returns nearest users and app displays them on the map. I am looking for a publish subscribe protocol for the real time data transfer.
Here is the architecture I was planned. Each mobile app users id will be considered as a topic or channel so that we can send one to one message. PHP server will considered as a topic e.g. "APP_SERVER_TRACKER". All mobile apps publishes location to Server topic. Server prepares the list and publish response to individual user's by using their id as topics.
I checked Pusher API and I would like to know how my PHP server will get user's location. Is it possible to subscribe all channels at server?
Pusher can handle higher number of users by scaling, but all user's are connecting to my PHP Server through a single channel and the server then publishing response to end user. In this scenario do I need to scale my PHP Server? Is Pusher is good choice in this scenario? Please advice?
The application will target at least 10 Million users. should support half of concurrent users.
PHP Server Hosting will be in AWS EC2 m3.2xlarge 8 core instance
Related
I have a IOT project where multiple devices update their current locations to the IOT server. The server parses the hex data received and stores them in a MySQL database. I have a independent REST API server that queries this database to retrieve the current position of the device and display it on a map in Android.
Now because of the refresh rates of the device and the Android API request, there is a considerable latency in updating the positions on the map. How can I convert this to a real-time design, eliminating the need to read from the database but just directly sending the data to the Android client?
Note: Multiple android devices may request live updates of the same GPS device.
I am considering using firebase GCM push notifications to deliver the position to an Android/iOS device which has requested for the live view updates. However, I think this would be inefficient. As this would not be very stateless as I would have to monitor the list of devices currently requesting push notifications and do some handshaking to start/stop the notifications.
Can I use web sockets instead to make a connection between the android and the server app itself? Does the quality of the internet connection matter excessively here? I realize that somewhat like UDP for updating the position the most recent packet alone matters here and any skipped position data won't contribute much.
based on our discussion
I think GCM is a good enough option for this kind of problems.
Your total number of users and concurrent users are not that much, As I know there is a limitation for FCM that is " For each sender ID, FCM allows 1000 connections in parallel." you can find it here
The other limit is about number of stored messages at FCM. Offline users do not receive messages until the become online and the limit is 100 messages. so this can be a problem, but when the user become online FCM will send a message that can be handled in client, I mean you will informed of the situation so you can ask the server to send last update messages, but if the only last one is important for you, this is not the case and you do not have any problem, because the client will receive the last update very soon.
Anyway, based on GCM/FCM you just need to worry about sending messages when you will get updates. Consider that GCM is not about only push notifications, you can handle the messages inside BroadcastReceiver.With FCM, you can send two types of messages to clients: reference
Notification messages, sometimes thought of as "display messages."
Data messages, which are handled by the client app.
in this case you need to use data messages.
So, based on these information, I suggest following scenario :
Devices send updates to the server side.
server will send GCM messages for the interested clients
on the server side, you can handle the interested clients and you need to make it parallel.
Hope this helps you. if there is any other aspect we should consider, let me know.
thanks
Mobile apps are updating their location to server and server responds with content.
Is it possible to use MQTT (Apache Artemis) for sending real time location updates from mobile application to server.
In this case is it required my server application (PHP) to subscribe for a topic?
Server has to handle all location updates through a single subscribed channel. This may slow down the server right?
Flow is like - Android clients sends user's location periodically (E.g. 30Sec) to server and the PHP server returns nearest users and app displays them on the map.
Here is the architecture I was planned using MQTT
Each mobile app users id will be considered as a topic so that we can send response to each user.
PHP server will considered as a topic e.g. "LOCATION_TRACKER".
All mobile apps publishes location to Server topic "LOCATION_TRACKER".
Server prepares the list and publish response to individual user's by using their id as topics.
In this archetecture the PHP server suscribes for topic "LOCATION_TRACKER" and in fact all mobile apps are publishing to a single topic.
Consider there are millions of users how to scale the PHP server or is there any other way of doing this?
The application will target at least 10 Million users. Should support half of concurrent users.
It is perfectly possible to send location updates from a mobile device to a MQTT broker, for an example look at the OwnTracks project.
For your seconds point, how a application behaves when processing a high volume of messages is entirely down to how that application architected. For high volume systems the usual pattern is to us a local queue and a thread pool to distribute the load, but it will all depend on what you need to do with the data.
I need to implement a group chat server which can support more than 50 users at a time.
The users will be only on mobile clients which include Android/Windows Phone.
Sending push notifs to the apps will not be difficult, as the server needs to do a basic curl request. So "polling" is not an issue.
My current skill set includes.
Flask
Google App Engine( I intent to use it or a web hosting
server.)
Basic php
So from what I know, I can make http requests to the server with user specific information+chat msg.
The server processes this and sends a broadcast using push notifs to all the devices stored in database.The server identifies the device from the data in the http request.
How efficient will this be?
If this is not the correct approach, what all do I need learn(sockets programming?) and any framework which can make my work easier (it will help if it is also supported on GAE).
I think you need XMPP messaging. It's support by App Engine but not in PHP : https://developers.google.com/appengine/features/
XMPP permit to send and receive chat messages.
I have an android app which i connect to my server using REST API (django rest framework)
here is a scenario(and maybe my plan):
data is sent back and forth as json
I have a user model and a task model where users are owners of some task.
Users typicaly sends a task to another user (with json similar to this: {"owner": "exampleuser", "from":"otheruser", "content":"example" ...} using POST method)
-The tasks has a boolean field "completed" and is deleted once the task is completed (using PUT or PATCH method: completed = true,)
once a new task gets created using POST method, the only way users can see any activities concerning their tasks is through an android activity that uses GET method to get a list of all the tasks owned by the user, by looking up all objects owned by the user
So my questions are:
Instead of having the user to check the app everytime. How can I use GCM to push notify the user?
How will it tell which user or device to send the push notification to?
How does GCM knows when there has been a change to tasks or when a user POST a task?
Android keeps one active connection to Google's servers, but it doesn't use much power or data, because no traffic is sent along it until something sends a GCM message to an app on your phone. There's only one connection on the phone, used by all apps: installing a new app that uses GCM doesn't add any extra load.
The first step in GCM is that a third-party server (such as an email server) sends a request to Google's GCM server. This server then sends the message to your device, through that open connection. The Android system looks at the message to determine which app it's for, and starts that app. The app must have registered with Android to use GCM, and it must have the relevant permission. When the app starts, it might create a notification straight away with the data from the message. GCM messages are very limited in size, so the app might instead open a normal connection to the third-party server to get more information (for example, downloading the headers of new emails).
The advantage of using push notifications is that apps don't have to run at regular intervals to check for new data, saving both power and data. The advantage of having a centralized mechanism like GCM is that the device only needs one open network connection and the Android GCM system is the only thing that needs to keep running, rather than each app having to stay running in the background to keep its own network connection to its own server.
As per the GCM implementation, it requires that you implement a remote server which will manage all the requests, both incoming and outgoing. You can do this simply with a web server acting as a webservice, which will get (for instance) the requests from the clients with a HTTP POST request, and process act consequently.
Instead of having the user to check the app everytime. How can I use GCM to push notify the user?
This will be managed by the server that I just described. It will know who is subscribed and who should receive the notifications.
How will it tell which user or device to send the push notification to?
Same goes here. The server, upon a subscription, should store the users in some kind of storage (a SQL database, for instance), and this way it will know who to send notifications. This implies you'll have to implement some timeout mechanism. In my case, I make clients send a dummy HTTP POST every 30 seconds. If I don't get this request from a reasonable amount of time, I consider the client timed-out and therefore I remove them from the database.
How does GCM knows when there has been a change to tasks or when a user POST a task?
Same story, it's all handled by the server. You implement the logic of what should happen upon any request.
You might want to read this: How to send location of the device on server when needed
And also this references:
Reference on Google Cloud Messaging
Android Push Notifications using Google Cloud Messaging GCM - Android Example
Google Cloud Messaging using PHP
Connection between PHP (server) and Android (client) Using HTTP and JSON
Notificaciones Push Android: Google Cloud Messaging (GCM). Implementación Cliente (Nueva Versión) (spanish)
stackoverflow fellows,
Since I am very new in Cloud Computing matters, I would like to ask you a question: Google Cloud Messaging for Android works from server to client, but also viceversa(from client to server)?
I am asking you this, because I am currently developing an Android application. The structure, in general lines, it's something like: I would like to use my laptop as a server - and perform all the computations here. Then, from my server (laptop), to send everything in the cloud, and from the cloud, directly to my Android running device. I want to perform the computing and database storage on my local server (because, since I am a student, I won't have enough money to actually pay for a server from google (let's say), so I am focusing on what's free and suitable to my needs) and just communicate with my Android device using Google Cloud Messaging for Android. (I want my application to be independent from the hardware of my mobile device, and also, I want to write the program that will perform the computations on my server in Java).
I know that I can send messages from the server to my device(through cloud), but the question is: can I also send messages from my device to the server ? I am asking this, because the computations will be made after I submit something from my device (basically, my Android application will be an interface, from where I can send the data to my server (through cloud) and then receive (also through cloud) the results of the computation on my device)
According to what I've been currently reading (http://developer.android.com/google/gcm/index.html) it seems that I can only send messages from the server to my device, but I decided to ask, since, so far, was the only free messaging framework(using cloud) I have found.
Thank you.
You can use GCM Cloud Connection Server to send messages from device to server.
Paragraph from this page:
http://developer.android.com/google/gcm/ccs.html
The GCM Cloud Connection Server (CCS) allows third party servers to communicate with Android devices by establishing a persistent TCP connection with Google servers using the XMPP protocol. This communication is asynchronous and bidirectional.
Google Cloud Messaging for Android works from server to client, but
also viceversa(from client to server)?
Sorry but GCM(Google Cloud Messaging) is only made up for sending Push Notification from server to device. Not for Viceversa.
I know that I can send messages from the server to my device(through
cloud), but the question is: can I also send messages from my device
to the server ?
For sending the message or anything(For Example: Photo,files..) to Server why Don't you Use Web services for it. I think it will Be suitable according to your Needs.
It is correct what KashifB wrote:
You can use GCM Cloud Connection Server to send messages from device
to server.
But keep in mind that Messages with Payload have a size limit of 4kb per message.