I have an issue which I can't even start to solve after hours of researching. I need to use GCM to facilitate peer to peer file transfer (JPEG image of circa 1.5 megabytes in file size to be specific).
Client A opens a socket and uses AirBOP to notify client B that it wants to send a picture and gives B relevant information (IP address + port to use, ephemeral I guess). B then connects to A and downloads the picture. All this behind NAT and via 3G/4G if at all possible. No server involved.
I looked into TTorrent, Snark, Cling, NAT hole punching, PWnat etc. but I still don't know how to achieve the above, and if it is possible at all. What i did understand is that i will need to use port mapping or UPNP.
Sorry for poor English, and I'm looking for a road map/guide lines/library/existing tutorials/projects on solving the above problem.
Thank you.
Google Cloud Messaging (GCM) was not designed to deliver large payloads. It's intention was to provide a light-weight messages for your application to then handle the heavy lifting (e.g. a new email notification informing your application it is out-of-date and that it should retrieve the new email). With that said, it does however have the ability to carry a payload of up to 4Kb. Which is obviously not enough to handle your 1.5Mb payload. You might think about implementing some type of protocol that will assemble the complete payload after x number of GCM messages. First, there is a quota on the number of GCM requests allowed per day per ip address. Second, GCM explicitly does not want its service to be used in this way and will block GCM messages if it detects it is being used that way. See http://developer.android.com/google/gcm/adv.html for more information.
What you need to implement a p2p application is either a STUN server or TURN server. A STUN server is used to discover the public IP and port that is being used by the device to communicate with the internet. You can either implement your own STUN server but requires a server that is publicly accessible/routable from both peers or use a public STUN server and use GCM to notify each peer of the others public ip and port. A major limitation with this technique is that if one of the devices is behind a router/NAT that has been configured to use symmetric nating, it will not punch through. There are exceptions to this but generally this is the case. Using a TURN server basically just relays the data from one peer to another peer via a central/public server. This isn't much of a challenge except it will cost you since this type of solutions puts a heavy load onto the server as oppose to a STUN server and you may need to purchase several servers to handle your user load.
I'm in the middle of my own p2p project and what I've discovered is that Verizon implements Port-retricted cone nat, AT&T symmetric cone nat, Sprint symmetric cone nat. Considering the popularity of mobile carriers (Verizon #1, At&t #2, Sprint #3), this is something to consider.
STUN Protocol
https://www.rfc-editor.org/rfc/rfc5389
Related
just want to know is there any methods to allow 2 different apps to communicate. Both of the apps are installed on different devices as well. I had gone throught quite a lot of research, but seem that INTENT, BROADCAST RECEIVER failed to meet my scenario. INTENT, BROADCAST RECEIVER can be work if both of the apps installed on the same device.
Here i can say you can use to make your things work :-
Sockets are typically used to accomplish this between Android devices (or between any peer devices).
When two devices desire to interact, you configure one or both of them to "listen" for connections on a socket and accept a connection from the other when that happens (or you can have a dedicated client and server and the client always initiates the connections).
You can exchange messages after the link has been made.
Android client server socket applications come in a variety of forms, but one that I found handy was:
Example of Android Server/Client using Socket on the client side (and its companion server side blog article - link included in the client blog)
It should be noted that you might need to add your own "protocol" on top of this. For instance, if you are sending a file that is unknown in length without a special "end" character, you might want to add a byte (or several bytes to represent an int, long, etc.) at the beginning to indicate the length of the transmission so the receiving side can tell when it has received everything (or that it has not received everything in case of an error).
connecting via networks (such as most 3G/4G) that forbid inbound connections
Even though there is nothing theoretically blocking sockets from functioning in these situations, many mobile operators will not permit inbound socket connections in practise. You would also need to determine the mobile's public IP address, which is doable but adds complexity. Whether your solution will only ever operate on a single operator's network, you can test it out to see if it works; but, if it doesn't, you could discover that using a server in the "middle" is preferable and easier: Devices A and B establish connections with servers Device A "discovers" device B after requesting the addresses of connected devices from the server. Device A sends device B a message.
Actually, it indicates that the messages should be forwarded to device B while sending them to the server. Device B is informed by the server that a message is available for it (using some sort of message notification like Google Cloud Messaging for example, or simply by the devices polling regularly to see if they have any messages). Device B accesses the server and downloads the messages. The aforementioned will function on pretty much any network that permits internet connectivity. It does have the drawback of having a server, but for the majority of mobile networks, it is probably a necessary approach.
You make one app a server using ServerSocket.
You make the other app a client using a Socket.
With both devices in the same network the client can connect to the server knowing its local ip.
After connection established they can communicate.
I have a situation where I want my computer in my local network to send a notification (push notification, SMS or whatever) to either an IOS of Android device that is on this same LAN. The trick is that I want to send this notification only when the LAN connection to the internet goes down. This means of course that my LAN cannot communicate with email servers, it cannot communicate with any APNs or GCMs, etc.
Is there any way to do this? My computer (the sender) is a Linux box and it will know the local IP addresses for any IOS or Android devices locally.
Yes this is possible.
You either need:
A) A server on the LAN to connect to, that the app knows about (compile-time or runtime configurable). The apps can connect to the server to exchange data, including messages. The server might be HTTPS, and the clients might poll every 5 seconds, or something more refined - using WebSockets.
OR
B) A peer-to-peer scheme. This can get quite complex, particularly given the varying nature of networking for mobile operating systems. As an extreme, you could use UDP for multicasting to multiple devices.
These are just two general examples, and you would need to research how to implement them for your needs. You should start with [A] for prototyping.
I need to be able to discover the services on the local network (so say I am running a chat application and I want to discover other devices on local network running this chat), but sadly I need to be able to use it on devices with API < 16 (so I cannot use android.net.nsd) I am sure this can be done without using NSD API. Question is how. Any help?
There are at least three or four options.
You can use
TCP or UDP
Broadcast address
Multicast address
Iterate over all adresses in your network
HTTP
4 Make use of external server or something like GCM (Google Cloud Messaging)
ad.1 and 2
Pros:
+Easy to implement, server is sending predefined "hello" message on broadcast/multicast address of your subnet/network on specific port, client is listening on that port and when "hello" message arrives, he automatically knows server IP address (contained in packet)
+Connection to the Internet is not needed
Cons:
Some public networks blocks this addresses to avoid attacks (e.g flooding ).
ad 3.
Pros:
When Multicast and Broadcast is blocked what you can do is simply iterate over whole subnet. It is " brute force " method, but works, especially on typical home/small networks where mask is /24 - there are only 255 addresses to iterate through.
Connection to the Internet also is not needed
Cons
When mask is let say /16 it will take pretty much time to iterate all IPs
can cause battery drain
and will flood network.
ad. 4
Another approach is to make some external server that will be "orientation point" for devices. Each of them sends up its own Address, and looks if there are some entries from another devices from the same network.
Of course Internet connection is needed.
So, I think it is good idea to start with broadcast and multicast since it is very simply and will work in many places, but keep in mind that there are networks where it is not allowed to use.
Good reference can be found here http://docs.oracle.com/javase/tutorial/networking/datagrams/broadcasting.html
You should also check this one http://home.heeere.com/tech-androidjmdns.html
I'm building a multi-OS mirroring system which I would like to implement using a hybrid client-server and p2p communication method (at least that's the best way I have of describing it).
My issue is that at some point I have a central server (appengine, so there are limitations to what I could do because of time and networking capability constraints) that would need to get a message to a host of different devices which are not necessarily running the same OS (Windows, Android, iOS, Linux, etc...).
Android and iOS (or any other mobile platform) are the main problems it looks like I will be having on 2 levels.
1 - They are both limited by battery power (more so than a laptop and desktops shouldn't have that issue at all), so whichever method I use needs to take that into account.
2 - NAT (harder because the user has relatively less control over their firewall than on a network that they are running). My central server will maintain a table of which device has what IP address, but from what I understand if there is NAT or a firewall it won't be able to get to it if the port was not forwarded.
Since I will be writing a specific client for each OS I prefer a solution that is more universal. I have been leaning towards writing an extremely simple HTTP server that sits on each client and takes requests (which appengine is able to send) and treats them as messages that alert the client to perform an action (either with the server or another client). However, I run into the issue of NAT/firewall. For instance if appengine needs to send a message to AndroidDevice1 it would grab its IP address from a table and make a request to it. However this doesn't work if the ports aren't forwarded correctly, and if the user is on 3g/4g the firewall is controlled by the data provider.
Because of this, I started thinking about using Android C2DM but I want a solution I could implement across platforms.
The only other method I could think of is to just have the client poll the server for messages. This has the battery and network consumption issue though.
Would there be any other way to implement this, and if not, which one of the above methods are best in terms of balancing usability, power and data consumption and user input (the less the user has to do to get the client set up (ie port forwarding, etc...) the better)? Please note that I do not intend for this to become a discussion/flame war but a logical presentation of facts.
Thanks in advance!
You can create a persistent TCP connection from the device to the server and then communicate over this open connection. This would be a very simple connection with keepalive packets for the most part.
In theory this would consume some battery through the radio, but in practice I have experienced that the battery is not affected much at all. Key is to keep the communication over this line to a minimum.
If AppEngine does not allow this approach, you can run your own socket server and then communicate between this server and the appengine server using REST. A socket server I have used is Apache MINA and had no issues with scalability.
Another problem you will have with this approach or any other approach is that on iOS (afaik) you cannot keep a tcp socket open when the App goes into background. The only thing to communicate with the iOS device is Apple Push Notification Service
I would prefer rather than having HTTP Connection you should create TCP/IP tunnel and make communication fast and reliable. I have one Chat application which runs perfact for me using TCP/IP. If you use this you will have same logic for multiple platforms. Only thing you need to write is different code for iOS and android.
I'm in the early stages of writing an app that will need to broadcast data to several other devices.
My first thought was using an UDP broadcast, however according to both
http://code.google.com/p/boxeeremote/wiki/AndroidUDP and
Android 3G UDP Broadcast
he UDP will not be able to push through the NAT when on the mobile network
(which is essential for my app).
I know that i could either use a server to broadcast however i'd rather avoid generating to much traffic on my home server.
The last alternative that i can think of is having several tcp/ip connections and looping through all connected clients and sending the broadcast. But since I'm counting on having at least 30 listeners I believe this will be to expensive.
I do not have any broadcast associated code yet, that's why I haven't posted any;)
Is there a way to break through the NAT? Will the phone be able to handle 30 simultaneous tcp/ip connections? Or should i look into some other method of broadcasting?
Any hint would be greatly appreciated!
Kind regards
Johan Risch
:::::::::::::::::::::::::ADDED::::::::::::::::::::::::::::::::
I will be sending strings of length 10-20 characters once every 30-360 seconds
(will be controllable by the user) containing geo points in string format. The order in which the data will be sent is not important, that's why I thought of udp first.
I've set up my server so that when a user logs in he/she updates my database with his/her current ip.
Preferably i'd like it to work globally, but as long as it would work within the mobile networks in the same country.
That's about all the relevant information i can think of, hope this clears some things up!
:::::::::::::::::::::::::/ADDED:::::::::::::::::::::::::::::::::
The NAT doesn't pass UDP through because UDP has no destination address, so technically UDP has to be sent to all subnetworks of the network. When the network is your home LAN, it's no big deal, but when the network is your ISP or university backbone or cellular provider, the UDP could be replicated to hundreds of thousands of subnets. That's a packet storm that will degrade network performance for everyone, and it's massive overkill for your particular app since you really only want to talk to a small number of clients.
This problem has been solved many times already. Instant Messaging clients have a similar situation as yours: want to do direct P2P messaging when possible, through firewalls. How do they do it? The original NAT traversal / firewall traversal solution was to set up a message relay server. All clients talk to the server, and the server echos messages from one client to the others as appropriate. It works with NATs and firewalls because the client initiates the outgoing connection to the server.
If the clients are able to establish a peer to peer connection, then the relay server can just give the clients each other's IP addresses and stop relaying messages.
UPNP is a protocol that enables clients to request a firewall to open a port for incoming traffic. BitTorrent clients use UPNP to enable clients to connect P2P for file sharing. Clients find each other via the torrent server. Most home LAN firewall routers support UPNP now, but it seems doubtful to me that a cellular network provider would provide UPNP support for over the air connections.
Another (remote) possibility is multicast TCP/IP, but as I recall this is really optimized for "push" content flowing from the server to clients rather than client-originated peer to peer.
Your best bet is to take a look at the open source IM clients out there, particularly ones with Android implementations, and see how they're doing IM. Jabber is one that comes to mind, I'm sure there are others. You could even use an IM system's messaging API as your data transport layer and more or less stay out of the wire level details completely.