Since I want to build reliable communication in mobile apps, could I get push failed reports(maybe device is offline) from the third party push services (C2DM, APN, urban airship) ?
Or We need to build it by ourselves?
The intended purpose of Android C2DM is to be a battery-saving way for your server app to signal the mobile device that it wants to start reliable communications.
You can structure your message so that each new C2DM encompasses everything that has occurred since the last two-way interaction with the server (i.e., "come and get whatever I've got"). Your failed delivery report is implicit in the mobile device not responding promptly (you can do this because you know C2DM activates your app with an Intent).
Is that really any worse than guaranteed delivery of each message in a lossy medium? Okay, it's worse in that you also have to implement a primary communication method. But you had to do that anyway because C2DM is inbound-only, right?
As Vinay says, MQTT may offer you the feature you desire. When a client connects to the server, it can register a "last will and testament" message with the server. If the client disconnects unexpectedly, the server sends this message to the topic it was instructed to do.
In this scheme, your client could send a message "online" to something like client//status and register the message "offline" as a LWT for the same topic. You could then have a server local client that listened to the topic client/+/status and it would know which clients were online and which offline.
I would suggest that the tokudu demo isn't the best place to look. This blog post by Dale Lane gives insight into using MQTT on Android: http://dalelane.co.uk/blog/?p=1599 and there is a review of MQTT power usage (again on Android) at http://stephendnicholas.com/archives/219
There are client implementations to suit both IOS and Android, see http://mqtt.org/software
No one service does not provide reports about failed pushes.
Failed push report makes little sence with APN/C2DM/Helium
All services are aimed to deliver the push-message under all circumstances.
If device is offline now, then push will be delivered when device becomes online.
Moreover, for iOS push-message is just a notification for a user, not for an application!
Simple case will illustrate it:
Assume that push is received when the application is turned off. In that case, notification for the user will occur. But, application will receive data from push, only if user taps on that notification! If user will tap on application's icon, then data will not be received.
So techically, push is delivered to the iOS device and application is started, but data is not delivered.
UrbanAirhip with APN and Helium
You can consider to implement your own transport for pushes. MQTT seems to be a good option.
But in this case, you have to deal with keepalives, device sleeps and battery optimisations.
All that hard work is already done by engineers from Apple, Google and UrbanAirship.
Depending on your business needs, it can be easier to adapt your architecture for existing solutions, then reimplement push-service again.
Take a closer look to UrbanAirship. In fact, C2DM has some limitations and sometimes timings for delivery of push-message is too big. Because of that UA has implemented their own transport - Helium, it works pretty good. Helium is a paid service, but UA provides good SLA.
I am suggesting for push notification IBM MQTT protocol. This fine enough for push notification.
see the demo from https://github.com/tokudu/AndroidPushNotificationsDemo
I did something similar where I had a database keep track of push queues to known subscribers and had reporting when failed. It was very simple and went something like this...
The schema was like so:
pushMessages
messageID , GUID, PK
message , nvarchar (256),
expires , datetime
messageQueues
subscriberID , GUID, PK
messageID , GUID PK
failedPushMessages
subscriberID, GUID, PK
messageID , GUID PK
(subscriber table omitted)
Once a client received the message successfully, the client would ping back to the push server and notify it via the unique queueItems ID that it received on the push notification. There would also be a daily database process that would check for expired push messages. When found, it would do a join on the queueMessages matching the messageID then remove them from the messagesQueues table and copy them to the failedPushMessages table.
This was very easy to understand and maintain but I don't have experience doing it another way.
Push Services are an efficient and reliable way to alert your users. They allow even background applications to inform users of new information in real-time. Push Services are widely used for a variety of fields in mobile applications, such as weather updates, messaging services, mail notification, coupon services, and so on. Push Services are no longer optional but have become essential.
Related
I want to create a notification system provider that is not based on Google services or similar. In fact, I want to get information about its overall architecture and needed Android sdk functionality.
The most weird point for me is to understand how to send a notification to an Android device.
I mean, how can I identify the Android device on which my application is installed from millions and millions of other Android devices on the Internet?
And how do I send information to him?
Should I use sockets for this or similar stuff?
simple answer: YOU CAN NOT
before everything else i should correct your question, pusher and FCM are not in the same group at all! you can build somthing like pusher or oneSignal or etc but you can not build something like FCM/APNS
you should understand three simple yet important sentences below:
when you want to pull anything from place_1(e.g. api) to place_2(e.g. browser_client) you most have an identifier of the place_1_resource (which commonly is the uniform-resource-locator of api)
when you want to push anything from place_1(e.g. notification_central_server) to place_2(cellphone_client) you most have an identifier of the place_2_resource
you must know the differences between a real server push with server-push-like technologies like long-pulling or ... and you should be aware that what is intended in this concept is a real server push not any kind of pulling with a push jacket!
if you don't have any identifier for a cellphone which you want to send it a notification, your server dont know where to send that notification so we need a resource_identifier_like for cellphones which is actually a device_token_like and you have just one approach to get this device_token_like and that is the FCM/APNS
FCM is like a dns server containing all identifiers of every android device that google supports (almost every android device) and APNS is just the same but for apple devices
note1: even if your app can obtain it's corresponding device device_token_like it can not be used for push notification if its not registered on FCM/APNS
so when you get that device_token_like identifier of your desired clinet_device now you can use different approaches for sending sth to that clinet_device. there are several approaches like SSE, Webpush, HTTP_server_push, Pushlet and etc but none of these approaches supported by mother_companies of these devices, the only approach that is completely supported and standard is the same approach that FCM/APNS official websites suggests
for example an iranian Incorporation named najva uses webpush to send notifications because of USA sanctions but webPush method works good on browsers and android devices but they didn't even apear on an apple devices
finally i should say that i admire your curiosity to less using anything from a benefit_based Inc. like FCM/APNS in your developing but i strongly recommend these articles and books for you cause i think you didn't learn enough:
wikipedia of push technology
story of some guy who tries to make his own push notification service
Push Technology A Complete Guide - 2020 Edition
Data Push Apps with HTML5 SSE
Short, direct answer
You can't (At least till you create your own ROM)
TL;DR, Reason why?
Before you build your own push notification server, you first need to know how it works internally in android.
Whenever you/your server sends a push notification message to the android client, the SDK processes it and shows you the notification. But when your app is not running (or being killed), your app cannot respond to it since it was not running. In such a case, your notification message is sent to a system service which is known as Google play service. For this even to work, you will first need to bind your app with Google play service and that is what FCM does. FCM SDK registers your app to the operating system service on the first initialization. That FCM service is opened to a port which listened to the incoming message from the server and when it receives the message, it publishes a notification on behalf of your app with a PendingIntent containing the data. Then the PendingIntent is delivered to your app when the user clicks it and then finally your app process the data (or the push message)
So basically, for your server to communicate with the client, It first needs to communicate with the FCM service and for that, FCM gives you a token which identifies the application to register with the internal Google play service.
Simplified furthermore, the workflow is as follows:-
Server send push message ---> FCM ---> Google play service,
If your app is running, it is directly handled by the client SDK So, Google play service --> Your app
If not, then it is delivered by the service itself using PendingIntent So, Google play service --> PendingIntent --->| Publish notification
Totally impossible, Workaround?
There is nothing like impossible because an absolute impossibility doesn't exist. Saying impossible generally means near to impossible. (This is similar to math where also we say tends to infinity because no one has achieved it yet).
To make it work, you need to somehow bind your app to the Google play service and you can't because Google hasn't exposed any direct API to do that. The only possible way is using the FCM ;-) (Bad luck again). So the only possible way is to build your own custom ROM with a custom push service that acts as a client for your Push server and a Server for your Push client (which is your app).
Since the above option tends to impossibility, you have to choose a workaround.
The best among the worst workarounds are:-
To make a malicious SDK.
Malicious because it needs to keep the app running in the background with a service that is connected to a WebSocket endpoint of your server. (Harder in new android versions).
Make use of a database where your push notification is saved and your app checks it periodically using AlarmManager.
Hope you have got the point.
I am doing a prototype that involves messaging between clients.
What I want to do - from MyApp on device1, able to send message to MyApp on device2. Device2 should receive this and show a notification.
I don't have my own application server to push notifications from GCM to GCM clients.Is this possible ? How to do it ?
What I investigated - PubNub, which has a trial license that seems to answer my need to send messages on a channel without needing a server.
This discussion at SO didn't help much.
sending client to client messages without server interaction
Can anyone suggest better ways ?
You basically have to use a server in one way or another. I think Firebase would be perfect for your purposes. It has an Android plugin you can use that will basically alert your app when something changes in the database. Here is the documentation for that feature. Take a look at this example for implementation. Before you can use the plugin, you need to create an account and whatnot (basic one is free). Here are the instructions.
PubNub Realtime Messaging and Push Notifications
(server not required)
With PubNub, you do not need your own server to do the realtime messaging or the mobile push notifications. You likely have your own server for your database (MySQL or similar, traditional RDBMS, a No SQL DB like MongoDB, or a realtime DB sync, like Firebase) to authenticate users and such.
For realtime DB or non-DB type change notifications and signaling (any type of realtime messaging), PubNub makes is super simple to receive messages in realtime with a mobile push notification fallback in one publish action. So the GCM message will be sent along with the realtime message. If the app is actively running, you get it in realtime (< 1/4 on average - typically faster) and the GCM message will be received if the app is not active (background or not started at all).
And PubNub's free plan is not a trial (anymore). It is free (including all add-ons) for as long as you stay within the free plan limits.
See the PubNub Android docs for more details and contact PubNub support if you have any further questions with getting started or getting answers to any questions you have as you progress with your app implementation.
Another possibility is to use the SMS.
Android Send and Recieve Messages
"Send Binary (Data) SMS
We can send binary messages (as opposed to text based messages that we covered earlier) to specific application ports using sendDataMessage(). According to this Stack Exchange thread data sms is one which is sent over 2G/3G as well as GSM. I’ve tested it with mobile data turned off and it works fine charging me the same amount, so not very sure on whether it uses 2G/3G or not, but generally the term data is used in telephony when it’s related to network (tcp/ip). Anyway, SMS’s are generally sent to a specific port on the device (which is probably port 0 [zero]). But using sendDataMessage() we can send SMS’s to some other random port on which our app can listen for incoming SMSs and do something with that. In this case the default messaging app will not store the SMSs in their inbox for both the sender as well as the receiver.
"
Can anyone please tell me how whatsapp messages are received on my mobile? Like, does my phone requests whatsapp serve every second or does the server send my phone a ping when a message for me is received at the server end. I don't want to know the exact procedure, but an overview will do the job.
Well i guess, its just not that easy to answer in this scope however here is how it might work.
For this you may need understanding of technology on mobile side and server side.So let me try to explain it in a very simple manner which may be the implementation.
1) No, your mobile does not ping the server everytime. Polling like this adds lots of overhead on the network, also multiple calls can be stacked up if there is delay in responding thus leading to huge traffic on server side. Instead , its other way round. Whenever the server recieves any msg it pings you instead to say that you recieved a msg.
2) Hope point 1 makes it clear that its other way around. Now would like you to know about push notification. Push notification is something by whichserver can notify mobile of any new msgs or information or update.Here's a link which can give more detail on push notification for android https://parse.com/tutorials/android-push-notifications
3)How the server processes the request. This itself is a big topic. Basically it goes like this.I assume there are two users, A and B
First Mr A send a msg to Mr B
Server recieves this request with information of the user A(i.e the person who sent the msg) and user B(i.e the person to whom the sg is to be sent)
The server does some processing to get to know the user id of Mr B and other infomation about Mr B and then does a push so that user B can recieve this notification.
Of course a lot of technology is involved to do this job in order to handle millions of concurrent connections and responding fast without blocking other connections.
If you want to have a basic view of the technology, i would suggest you to look at socket io and node js. Heres a chat example
http://socket.io/get-started/chat/
Hope this helps
WhatsApp uses XMPP (eXtensible Messaging and Presence Protocol) to handle the message delivery system.
XMPP is mostly like HTTP where the client opens the socket with the XMPP server and keeps it open as long as the client is logged in. It's not like the regular REST API where the client opens the socket send/receive the data and close the socket. The socket is opened as long as you are signed in. In case of WhatsApp that's eternity (not really, WhatsApp reconnects automatically if the connection terminates)
XMPP protocol has been used in various chat applications such as Google Talk, Facebook messenger etc.
As far as actual technology goes, WhatsApp uses heavily customized version of Smack library on Android to build their client and uses customized eJabberd Server to handle the XMPP traffic. They might have different backend solution for handling the data though which might be on one of the cloud storage/computing network (I think it's heroku, no real idea though).
On iOS and other platforms, I suppose they might have developed their own libraries. Developing own libraries is not a lot of work, especially when you have customized needs and have a team of developers. I have used one of the libraries available for Windows Phone and heavily customized them to work for us. I made some improvement on the library, but due to time shortage I couldn't submit them to the original repo (Documentation is really tough).
Anyways, if you are interested in learning the tech, you can read Oriely's "XMPP: The Definitive Guide" and can visit The XMPP Standards Foundation.
Basically to say that when there is a message, Server will sends a notification to Client via Push Service. In before Client must register to use Service with Server, Server will provide an Id for Client and then uses it to specific which Client will receive the message.
Read more in Here and Here
Our organization has an Android app and an iOS app.
We want to start pushing notification to these apps.
Android has GCM.
Apple has APNS.
But we want to create an API which will work on both android and iOS.
What is the easiest way to setup a server so that when a push notification needs to be sent, it knows exactly which server to send the message to?
I use a service called Parse to do my notification pushes to both Android and iOS. They have great documentation and libraries available. You can get some details here: https://parse.com/products/push
As a little background this is for a university setting where multiple colleges apps as well as distance education may be using the service. Here is the approach that we are using in our organization. If you look at the way APNS works it can be used by just sending a web call to the APNS service with the token id. GCM is very close to the same type of system. Basically create a JSON package and send it to the desired service.
Here is our steps we used to create this service.
Server admins created a server and database that can be called that will collect the tokens from both android and ios devices. When the device registers we also send what type of device it is. This is possible since we are just sending data to the database that is has been created.
From here we then created a couple of python scripts that send the data do the desired service whether it is ios or android. These scripts gather the appropriate data from the database and sends the packaged data (JSON package) to APNS for ios message and GCM for google cloud.
We also created a web interface so that those who need to send messages to the devices can.
The rest of the implementation is up to you to decide the best way to utilize the service. For example when to check for invalid devices,
Because we are planning on using this same server for multiple applications we can send the type of device, token, application, or whatever else is needed for an application to distinguish it from others we produce so that each application that wants to use the service can. I hope this helps and gives you some idea on how to accomplish this.
For APNS, Maybe you may consider this forked version of PyAPNS that has enhanced message support.
https://github.com/jimhorng/PyAPNs
which means it will catch error response for failure messages and resent the message which are discarded by APNS while sending between failure messages and receiving error response.
Solution:
Non-blocking ssl socket connection to send notification without waiting for response.
A separate thread for constantly checking error-response from read connection.
A sent notification buffer used for re-sending notification that were sent after failed notification, or arbitrary connection close by apns.
(Reference to non-blocking apns pull request by minorblend, enhanced message by hagino3000)
Result:
Send notification at throughput of 1000/secs
In worse case of when 1st notification sent failed, error-response respond after 1 secs and 999 notification sent are discarded by APNS at the mean time, all discarded 999 notifications will be resent without loosing any of them. With the same logic, if notification resent failed, it will resent rest of resent notification after the failed one.
For GCM, you may consider https://github.com/geeknam/python-gcm
For generic wrapper that support both or more mobile provider:
https://github.com/Redth/PushSharp
I am developing mobile client for emailing service. One of the key features is notifications about new messages in the mailbox. As recommended by GCM architecture guidelines we are using a "Pusher" that is responsible for sending messages to the Google servers once we received a new message. The issue is that testing process has reported about serious problems with push notification delivery to devices.
So the question: is there an approaches for monitoring average statistics about push notification delivery percentage, time etc? Or maybe somebody have experience in how to set up test environment for efficient monitoring of how much notifications are getting lost during the application work?
All the "tips&tricks" related to the improving Android GCM experience are welcome.
Google claims that the processing at their GCM server takes less than a millisecond. Link below for a great video on GCM from Google's developer. And it's believable coz I could get push notifications almost instantaneously using my company's server to my device now.
http://www.youtube.com/watch?v=YoaP6hcDctM
They don't guarantee delivery, but they try for a max of 4 weeks to deliver the message depending on the duration you set in the message you send to Google's GCM servers and if you wish to let Google keep the data for eventual delivery of message to the device in case the device was offline when the message was to be delivered.
However, there are certain conditions under which the GCM messages are not delivered.
Background data is unchecked under Account and Sync settings.
Prior to 4.0.4.(ICS), a Google account on the device is a pre-requisite for GCM. Maybe, Users are not logged into their Google account.
The only way to do so is to report back to your server with the timestamp of the received push.
You can either
Report back to the server once you receive the notification in your GCM service. To implement, you will have to add a push id for your push notifications and send the id along with the push data. The client will have to get the timestamp once it receive the message and send it back along with the notification id. A simple php script can be done (when you send a push notification, you set the time of the send-notification and once it receives the device's timestamp it sets the receive-notification. This boils down to two fields in your database (marked in bold). In this approach you will probably not so much care about errors since it is very probable that the device will have a connection when it receives the notification and as such its request to your server will go through.
Keep a list of notifications received in your app and their timestamps. And when the sync is done, send the this data in your sync operation. This is ultimately the same approach but your server's data won't be as realtime as the first approach. However, the extra request is not required from the client's side but saving the received notifications and their timestamps is.
All in all, you will have to keep track of the notifications sent using a notification-id and their sending time (send-notification) and their receive time (receive-notification). A simple query will help you analyze this data.
Google has added support so that you can receive delivery receipts from Cloud Connection Server (CCS):
You can use upstream messaging to get delivery receipts (sent from CCS to your 3rd party app server) when a device confirms that it received a message sent by CCS.
To enable this feature, the message your 3rd-party app server sends to CCS must include a field called "delivery_receipt_requested". When this field is set to true, CCS sends a delivery receipt when a device confirms that it received a particular message.
https://developer.android.com/google/gcm/ccs.html#receipts
Google does not make these statistics available to you. There are some statistics available on the android developer console. This only shows the number of messages and registrations.
You would have to implement your own data collection, which could be done fairly easily. You could record the time & id of each message sent and have your android client report back to your server with the time of message receipt. You could then store the data on your server and query as needed.
Since that time Google has provided developers with advanced monitoring tool.
The Gcm Diagnostic Tool is available in Google Play developer console. Additional information is here https://support.google.com/googleplay/android-developer/answer/2663268
So you can easily track the particular message status via registration token.