Develop Reference

Connect to Android App's Socket from Internet: Possible?

I am trying to develop an Android (server) app that will accept TCP connections directly from the Internet. I tried this a couple of years ago with no joy. Is this even possible?
I asked this question differently way back then, on StackOverflow and elsewhere, and never got a satisfactory answer. The answers fell into two categories:
No, you can't
Most authoritative answer given: Most carriers won't route connection requests to mobile devices.
Yes, you can
These answers did not provide details or examples to help me.
What I Tried
I can successfully get my mobile device's external IP through various means. But this is only part of the puzzle. See Note 1 below.
I created a ServerSocket that listened on a port (e.g., 2000). But the app would not accept connections from the Internet on this socket. Apparently, this socket was listening on the local (internal) network of the device (i.e., 192.168.1.0).
I was able to connect from other apps on the device (which are on the same subnet).
I was also able to connect through a NAT router in my shop that translated Internet addresses to my device's local net. But this doesn't help me in the field (where no router is available). See Note 2 below.
I tried asking AT&T customer-support-type people but I don't think they understood my question.
Notes:
Yes, the device's IP address can vary with time or location, but I can always get the correct and current IP through various means (e,g., whatmyip.com, etc.). Nontheless using the current device IP doesn't work either. BTW - my public IP has never changed. Thus, for all intents and purposes, my IP is static. In any case, the IP issue is a well-flogged horse, I am fairly sure that this is a ROUTING issue, not an IP address issue.
I can connect via WiFi because my Internet modem/router offers DHCP connections on the back side via Wifi on the device's local subnet (192.168.1.). My device has address 192.168.1.65, therefore I can connect to it without ever going through my carrier.
AT&T tells me that I cannot buy a static IP for a mobile device (update 9/23/19).
My Current Theory (9/23/19)
It is my current humble belief that there is a NAT-like entity at play here. Either:
The public IP actually points to a NAT router at AT&T HQ.
The device has a built-in NAT-like router
In either case, under this theory, inbound TCP connections are not forwarded to the device's local net, but outbound connections pass through just fine.
Summary
Maybe the answer's still the same, but I thought I'd ask again in case things have changed or somebody's got some better insight nowadays,

It's true that you can't establish any TCP or UDP connection which is listening behind the main network (i.e. behind the NAT). Nowadays all the ISP allocate a private IP to every device (due to shortage of IP) connected to it hence creating a private network of devices behind NAT. (After the complete adoption of IPv6 the situation will change with every device becoming the part of main network)
Since NAT doesn't allow an inbound connection, the problem is solved using some of the NAT traversal technique such as:
Port forwarding
TCP/UDP Hole punching
In the Port forwarding process, we need to deal with the router setting which will forward the connection on a specific port to the same port on the internal network node. It is mainly used in gaming, VoIP, and p2p application.
In Hole punching technique (no need to deal with the router setting) we use an external server which is not behind any NAT to get the IP address (both public and private) of the devices which are behind NAT. Further, the two hosts create there own outbound session among each other in their respective NAT devices. Since the NAT devices allow the outbound session, the session is stored, and this same session is used for further two-way communication.
More information about Hole punching.

(UPDATED 9/26/19)
I am posting this "answer" not so much as an answer per se but rather to solicit constructive criticism and/or affirmation. Any statements I make below should definitely be implicitly prefixed by "I humbly believe that...".
Yesterday, I had a personal epiphany. I finally got it through my thick skull what some responders were saying. I present my new understanding below.
Heretofore I believed that AT&T's Mobile Data Network was just a direct extension to the Internet implemented with cell sites and trunk lines. Now I realize that it is a closed, private data transport mechanism. Since it is owned by AT&T, they can implement whatever protocols they choose to, including "private" data connections from their head-end to my registered phone. These connections are analogous to the LAN-side ports on a typical desktop NAT router. The head-end (i.e., their NAT router) is the only entity that can be accessed from the Internet.
The head-end connections between the Internet and my phone are over these AT&T-private connections. The "public IP" reported for my device (and, I believe, nearby devices) actually points to the AT&T head-end, in my case a 150 miles away in Miami). The head-end fowards outbound connections from my phone to the Internet, but does not forward inbound connection request.
About my "Public IP"
I believe that this doesn't exist and is, in fact, a meaningless concept. What would be the point of such an address if you cannot connect to it.
Summary
I now believe that the answer to my original question is "No way, Jose". This is not due to an addressing issue, but rather AT&T will not route incoming connections to devices. They also do not offer static IPs for devices over their private network (this wouldn't make much sense anyway given the above).
IF ANY STATEMENTS IN THIS PROTO-ANSWER ARE INCORRECT, PLEASE PLEASE LET ME KNOW WHICH ONE AND WHY IT IS INCORRECT

It depends on what type of IP your mobile phone provider assigns you
For your mobile device to accept TCP connections from the outside, it is only possible if your provider assigns you a public IP, but this is increasingly unlikely since given the shortage of available IPs many providers assign a private IP, like if your phone were inside a LAN, in this case your TCP port is unreachable from the outside.
Websites like whatsmyip tells you the public IP with which your provider finally connects you to the internet but it may be different from the one your provider assigns to your phone.
To know if you are in a private network install from Play Store an app called "PingTools Network Utilities".
When you start it you will see the IP of your phone, if it is different from the whatsmyip IP or is an IPV6 one that means that your provider assigns you a private IP, an unreachable one.
... always talking about connections through the cellular network and not through WiFi

Write a simple client for DynDNS or alike, in order to provide it with name resolution. Then you can serve pages with NanoHTP - or raw sockets. If you're behind a DSL router, you may need to read the external IP with TR-604 (unless DynDNS might eventually return the source IP) and setup a NAT port-forward for the incoming traffic on port :80. At home this is easier to setup than through UMTS, because some UMTS networks don't have public IP addresses, but may have private IP addresses.

I want to give a non - technical answer as I myself am not very proficient with network theory.
One thing is that Internetproviders usually expect extra cash if you want to run a server - hence the forced disconnects on consumer subscriptions like DSL(changing IP in the process).
Next thing is that establish a connection and send data out to a server is very unreliable on a mobile device. You can have android check if you have network. If it says no you don't need to try it - no means no. But yes doesn't mean a thing!
I write software for asset tracking using android. Usualy I want to send like one location per minute to that server. So I try to open a port. Android blocks on new Socket and it can go for like 2 Minutes until a timeout occurs even if I checked with ConnectivityManager right beforehand and it got me isConnected() == true.
And if I have a socket and even received something a second ago: I can write to that socket and get no error but get no answer, write again, no error no anser again and again.. And after like 2 minutes I get a whole bunch of errors in the log all at the same time.
When sitting in the bus visiting my mother is like driving through a 3rd world country judging by the internet connection. It might say 4g and 3 bars but chrome won't load page OR it says 2g and only 2 bars yet browser has no problems at all..
The whole android network-os (and I guess apple has to do it quite the same) is programmed more like: "here android I want to send this" and android saying "Ok" but thinking "I will send it as soon I feel fit to do so" :D

you need a static ip(public ip) address for that .
Mobile device ip address are not static so u cant .
To make it work .
Run a web server on a static ip & route the queries to your app (tunnelling).
When android app starts ,you android app needs to register with web server .
Now server knows your android device availablity ,the queries from other end will be be rediredcted.

UDP multicast over the internet with an Android receiver?

I'm essentially trying to take a stream of UDP packets created on my local network, and get an android phone to receive them whether on that network, or cellular data as if it was on the source's local network.
I don't have tons of network experience so bear with me.. I have spent the last week researching this problem and found bits and pieces of ideas to make it happen, but I have no idea how to implement any of it.
It seems a sort of multicast with PIM sparse mode seems to be the best option. If I can have the packets routed to the rendezvous point and the phones subscribe I should be set. Looked at VPN a bit too, however Android apis would make it an annoyance to the user.
So I guess my questions in a nutshell for anyone will to put me on the right path would be:
1:How do I route local UDP packets to a rendezvous point? A special router and a port forward?
2:How can I make Android subscribe to a rendezvous point
3:Lastly, when the phone is subscribed does it act as if those packets are from its local network? Or is there more required?

Multicast is not a good solution for your problem. Multicast packets do not usually get past routers. This post's answer explains why multicasting is not a good choice.
Instead write your own client/server application using standard sockets (not multicast) where the server streams the data to each client individually.

NAT traversal techniques and idea

So I was in a middle of making NAT traversal.
The scenario is next: I got two Android phones and I want to connect them(sockets) using HTTP server(both devices are behind NAT).
So far so good, both clients connect to HTTP server, HTTP server records their IP address&PORTS,
however there is a little problem, since I use Java HttpDefaultClient(), it will change the port each time I send request from client to the server. Okay that sounds like simple problem:
let's just use Socket() to actually maintain a valid TCP connection into server.
public Socket(InetAddress address,
int port,
InetAddress localAddr,
int localPort)
throws IOException
I will just use this class and put the localPort something random which I will remember.
Now I do everything again, this time it seems that port wont change, like I want.
Now after I have opponent's IP&port(he's also behind NAT), theoretically I could drop the SERVER connection and use the same localPort I already used to actually host a clientServer?
unless, and now comes the part where I have question:
1)If I drop the HTTP server socket, will NAT understand that and remove the port mapping?(that's bad)
2) How does actually passing symmetric-cone nat work?
3) does the STUN libraries work somehow differently?

NATs don't all work the same, but you can count on a few things:
1) if your client thinks it's on port X, the NAT will translate that to a different port
2) NATs usually let in packets that are in response to outgoing packets.
STUN tries to guess, with a server's help, what the actual outgoing port is, and then pass off the address+port to the other client. It's not terribly reliable. TURN just routes everything through the server. That's more reliable, although it incurs CPU and bandwidth cost on the server.
If you can use existing code to do NAT traversal, you'll save yourself a lot of trouble. Otherwise, do something like TURN through sockets, OR use something like Urban Airship. I've also used binary SMS, but that was for a special case.

With most (but not all) NATs these days, you can assume some consistency and predictable port mapping behavior. (That is, using the same local port for different server connections, maps to the same local port on the NAT). But it sounds like you want to do NAT traversal over TCP, which is a harder problem than UDP.
The fundamental problem is that most NATs are also acting as firewalls. They won't allow an inbound connection from a remote ip:port. I believe the trick is to do a simultaneous connect on both sides.
You can read more about TCP hole punching here: http://en.wikipedia.org/wiki/TCP_hole_punching

I think you are running into a couple issues with regards to Nats. I actually got this to work awhile back(creating a peer to peer android connection utilizing a third part server(mine) and a public STUN server.
I'd highly suggest your read RFC5389 - Nat traversal It's complicated. I'd also suggest you use something like the JStun library or implement your own as i did.
I will just use this class and put the localPort something random which I will remember. Now I do everything again, this time it seems that port wont change, like I want.
My guess would be your behind a nat, so the internal port you requested got mapped to a different outside port.
Now after I have opponent's IP&port(he's also behind NAT), theoretically I could drop the SERVER connection and use the same localPort I already used to actually host a clientServer?
Not yet, most nats not only record the ports that clients made connecttions two but the IP address they connected to so they will block traffic from other IP's. For instance, Phone1 is on IP-a and connects to server from ip-a on port-b The nat translates port-b to port-c. From the servers point of view the phone is at IP-a on port-c. It relays this information to phone2. The nat will block all communication from phone2 until phone1 sends data to phone2 from port-b.
1)If I drop the HTTP server socket, will NAT understand that and remove the port mapping?(that's bad) 2)
Something I learned from experiance is not to expect anything from the nats port mapping behaivor except that your port will be mapped, and some nats will change said mapping for what from your point of view seems like silly reasons. This requires a lot of updating. But in general no.

Android broadcasting over 3g

I'm in the early stages of writing an app that will need to broadcast data to several other devices.
My first thought was using an UDP broadcast, however according to both
http://code.google.com/p/boxeeremote/wiki/AndroidUDP and
Android 3G UDP Broadcast
he UDP will not be able to push through the NAT when on the mobile network
(which is essential for my app).
I know that i could either use a server to broadcast however i'd rather avoid generating to much traffic on my home server.
The last alternative that i can think of is having several tcp/ip connections and looping through all connected clients and sending the broadcast. But since I'm counting on having at least 30 listeners I believe this will be to expensive.
I do not have any broadcast associated code yet, that's why I haven't posted any;)
Is there a way to break through the NAT? Will the phone be able to handle 30 simultaneous tcp/ip connections? Or should i look into some other method of broadcasting?
Any hint would be greatly appreciated!
Kind regards
Johan Risch
:::::::::::::::::::::::::ADDED::::::::::::::::::::::::::::::::
I will be sending strings of length 10-20 characters once every 30-360 seconds
(will be controllable by the user) containing geo points in string format. The order in which the data will be sent is not important, that's why I thought of udp first.
I've set up my server so that when a user logs in he/she updates my database with his/her current ip.
Preferably i'd like it to work globally, but as long as it would work within the mobile networks in the same country.
That's about all the relevant information i can think of, hope this clears some things up!
:::::::::::::::::::::::::/ADDED:::::::::::::::::::::::::::::::::

The NAT doesn't pass UDP through because UDP has no destination address, so technically UDP has to be sent to all subnetworks of the network. When the network is your home LAN, it's no big deal, but when the network is your ISP or university backbone or cellular provider, the UDP could be replicated to hundreds of thousands of subnets. That's a packet storm that will degrade network performance for everyone, and it's massive overkill for your particular app since you really only want to talk to a small number of clients.
This problem has been solved many times already. Instant Messaging clients have a similar situation as yours: want to do direct P2P messaging when possible, through firewalls. How do they do it? The original NAT traversal / firewall traversal solution was to set up a message relay server. All clients talk to the server, and the server echos messages from one client to the others as appropriate. It works with NATs and firewalls because the client initiates the outgoing connection to the server.
If the clients are able to establish a peer to peer connection, then the relay server can just give the clients each other's IP addresses and stop relaying messages.
UPNP is a protocol that enables clients to request a firewall to open a port for incoming traffic. BitTorrent clients use UPNP to enable clients to connect P2P for file sharing. Clients find each other via the torrent server. Most home LAN firewall routers support UPNP now, but it seems doubtful to me that a cellular network provider would provide UPNP support for over the air connections.
Another (remote) possibility is multicast TCP/IP, but as I recall this is really optimized for "push" content flowing from the server to clients rather than client-originated peer to peer.
Your best bet is to take a look at the open source IM clients out there, particularly ones with Android implementations, and see how they're doing IM. Jabber is one that comes to mind, I'm sure there are others. You could even use an IM system's messaging API as your data transport layer and more or less stay out of the wire level details completely.

Peer-to-Peer communication options

can anybody confirm what are the currently allowed methods for peer-to-peer communications within the Android framework? I need to transfer json strings and I'm currently using SMS which works ok but the problem is that the data also ends up as lots of text messages. I've read Reto Meier's first edition of Professional Android Application Development where he says that the data transfer options were not implemented due to security concerns.
Has this changed at all and how would you do peer-to-peer transfer of data?

Have you looked at Qualcomm's AllJoyn library? It is designed to work over Bluetooth or wifi, so may fit, though if you're connecting over 3G or wider range networks it won't work.
Given the variation and reliability of networks between two remote devices not on the same network, I'd question whether peer-to-peer is the best solution, and would venture to suggest considering using an application server in between, so you could then use Cloud to Device Messaging [deprecated] (perhaps in tandem with Google App Engine). i.e. to send a message the sender passes it to the server, and the server then passes it on to the recipient.
In theory all devices on the net have a unique IP address and can talk to each other, but it's rarely that simple as routers/firewalls are configured differently so you'd need to pay great attention to the ports you use, especially considering many inbound ports are blocked by default for security reasons.

You can simply use UDP/TCP sockets. In a separate thread you set up the server-side listener socket and that's it. Of course your application has to be started first (or should run in the background all the time). Here's an example:
http://thinkandroid.wordpress.com/2010/03/27/incorporating-socket-programming-into-your-applications/
If you also need peer discovery that will make the thing more difficult.

You should also take a look at peerdroid, an open source project available here. I've been looking in to peer communication options from the point of view of having a collection of federated devices (pre-paired, if you like, similar to Bluetooth pairing); this library looks like it may give you the foundation for what you are trying to do.
If you are on your own network (for example a home or office WiFi) then you should be able to query for other connected devices. If the network you are on is not under your control (the mobile network or a public wifi) then the network will have been configured to isolate each device from everything else. In this case you will have no choice but to put a server up to act as the man in the middle. That brings its own architectural compromises - every device has to regularly poll the server or keep a connection open - unless you use Google App Engine which supports push notifications over Google's own infrastructure.

Thanks for your answer ldx but I would need peer discovery as you indicated. Some further research appears to indicate XMPP as a suitable technology and there are now some services on offer, although these appear to be aimed at 'server' to client notifications. There is a good discussion here on XMPP and some more here although it would appear that there are still some issues to deal with such as polling v push, long-running open http connections and battery life. Xtify looks promising, especially their web service. I hope this provides suitable information to others looking at the topic of peer-to-peer data communication.