I am going to build a client-server application. The client here is an iPad (or an android-based) tablet. The server is a normal pc. Both the clients and the server are connected to the same network (using WiFi).
Is there a standard way (protocol) for communication between the clients and the server? Are there any framework that can be used to ease this communication?
Thank you
The answer depends by what you define by "server", "client", and "protocol".
Technically, the answer is "yes"; from a practical standpoint the framework you are looking for is called "socket", but regarding the protocol things may get complicated.
A protocol is a syntax structure governing data exchange, i.e., a set of rules you use to request/provide a service (see the IETF website for a list of standard ones).
Sockets, on the other hand, provide you merely a communication channel to bring bytes from one side to another and, on top of which, you are required to implement the protocol.
The good news is that socket are language independent and you can send messages between heterogeneous devices (ipad/android/linux/windows).
Using sockets in java is easy (I am making it very short here)
server side
ServerSocket ss = new ServerSocket(port);
Socket s = ss.accept();
InputStream is = s.getInputStream();
client side
Socket s = new Socket("server.address", port); // same port as above
OutputStream os = s.getOutputStream();
When you write something using os.write() the same bytes will be read by is.read().
What you write on os is the implementation of your protocol.
This topic is covered (for java language) quite well by "Thinking in Enterprise java" by Bruce Eckel, you can access the digital edition for free.
In C/C++/Objective C things are more complicated but you can easily google for tutorials.
Each service defines its own protocol and you should decide if one of the existing will do or you have to define your own, depending on which service you want to implement between the two devices.
If, as in the standard approach, the PC plays the role of server and clients want to retrieve information from it, you might want to consider installing a (very) lightweight web server and access data using HTTPUrlConnection. This is a wrapper for a socket with HTTP protocol management already implemented.
Beware, this is for Java; there is no "standard framework equivalent" for C/C++, I honestly have no idea about objective C.
Please, be also aware of the following:
If client and servers has different architectures binary data exchange may get painful, better define your protocol as a sequence of strings (like SMTP) and encode/decode binaries using base64 or some other method you may want to implement
In order to link the two sockets client has to know the server IP address; if you are running DHCP on your WIFi network then you also need to implement a discovery phase for your service
As a last side note: "client" and "server" are just labels you put on communicating entities depending on who is requesting a service/information (client) and who is providing it (server). Communication is in reality symmetrical and you can use the same structures/functions/code on both endpoints.
Related
I want to ask some question regarding the development of my push-to-talk application for Android.
Recently, I've been developing push-to-talk Apps for Android. I use DatagramSocket to send voice and receive voice as a Packet over Local Wireless network (W-LAN). I use peer-to-peer network, so no server.
I don't have a problem with the code, but I don't understand the basic of VoIP theory, so I want to ask some question, hope somebody can give me simple answer :)
1. Is my push-to-talk Apps considered a VOIP-based?
2. There is several VOIP protocol such as SIP, H.323 and many more. If my PTT Apps considered a VOIP-based, and I use Socket-Packet (UDP, am I right?) to exchange voice, then what VOIP protocol that I use? Is it considered RTP protocol?
I would like to understand the theory behind my PTT apps, I understand my java code, but I don't have proper VOIP knowledge.
I've tried to find some information in google, but I still don't understand what is the relation between my PTT Apps and the VOIP technology.
Thanks before, I'm new here and sorry for my english!
I. "VoIP" is a very broad term, but, if your app transfers voice over IP network, it's definitely VoIP one, despite it may use totally proprietary protocols (as e.g. Skype does).
II. VoIP stack is basically split to two meta-layers - 1) signaling and 2) media transport. Each of them is in turn consisting multiple own layers (e.g. for SIP: session, dialog, transaction and transport layer). Examples of signaling protocols are H.323, SIP, MGCP. The most standard media transport is RTP. You can use your own transport; RTP applies specific restrictions (as AVP profile) but is compatible with variety of libraries and other implementations.
There are protocols which use the same socket and the same transport type for both signaling and media; the widely used one is IAX. Most others separate signaling and media, so sockets are separated and likely they have different type. A standard-compliant SIP implementation shall function over both UDP and TCP, and switch to TCP for large requests (>=1300 bytes by default); SCTP is also suggested. For all transports, protocol implementation details as retransmission policy and request timeout are specified in different way, but there is no principal problem with using any correct L4 protocol.
The totally another story is with media transport (under RTP or equivalent). Here, typical TCP manner to transfer all data at the cost of floating delays is really nasty for our ears. TCP is good for bulk traffic class, as file transfer or database interaction. In interactive communication between humans, we prefer more noise and sporadic voice loss than a voice strain. So, TCP is a very bad choice here, and a synchronous transport class shall be used, and UDP is the good default choice. SCTP can also be used as media transport, but with limited retransmit option (not all stacks support it). (There are attempts to use TCP to punch through NAT points but all this is an act of despair.)
If your application supposes sending a voice message more than to one recipient at a time (i.e. a kind of broadcast or multicast), this rejects use of connection-oriented media transports, effectively retaining only UDP. This also requires proper negotiation at signaling level.
III. Selection of voice codec is very platform-specific, I don't use which ones are native for Android. In "big" VoIP, there are licensed set and free set, with very small intersection (AFAIR, G.711 and GSM). Despite this, there are good codecs (e.g. Opus) which can be adapted into wide range of requirements, including partial packet loss.
I'm making an app using bluetooth, and I realized it'd be pretty neat to exchange data between 2 running devices via bluetooth.
However, as far as I know most Bluetooth devices use a traditional server/client architecture, so the information exchange is one sided, ie, client requests data from server, server sends it, but not the other way around.
To get around this problem, I tried to make each device have a server instance as well as a client instance, so that the client can hook up the the server in the other device. However this does not seem efficient as this requires 2 channels of communication instead of one.
Then I tried going to Bluetooth website, and they suggested "role switching", that is, when server wants some data from client, server makes itself a client and client becomes a server, so the role is reversed. I don't quite like it either, because why can't 2 devices play the same role and exchange data at the same time? What kind of connection is this called then?
I have not had experience in networking so my question might seem naive, but I'd like to have someone point out the answer, or tell me what information/which chapters of textbook I'm missing so I can wiki it.
Role switching affects the underlying Bluetooth topology: which device is master in the piconet. That can have implications for the quality of your connection, but it does not directly affect the direction of communication in the way that you are imagining: it is at a much lower level.
Yes, in Android Bluetooth comm is done using sockets abstraction and the setup of the socket connection is done in a client-server manner. But once the socket is connected both sides have a BluetoothSocket instance and I'm pretty sure that both sides can get an output stream and initiate communication. What makes you think otherwise?
Basic question before I get too far into coding. I was told once that in order for the Android Phone to contact a PC server, that server must be written in Java. I find this a little convoluted, but is this the case? Or by using the TCP/IP classes (Socket), can I just read and write binary data over the pipe regardless of who or how the server was written?
TCP/IP is language agnostic, you can create a client/server implementation in any language or platform that supports TCP/IP and communicate with any other TCP/IP connection.
The communication over the protocol is dependent on implementation though so you need to make sure that both client and server implementations understand the communication going over the sockets.
You can write data regardless of what kind of server it is. Whether it's written in Java, PHP, C++, or whatever does not matter. As long as the server knows how to read and write to/from sockets. I've had my Android phone connect to a PHP server today, so either my phone is disobeying the laws of the universe, or...
Also, the same applies for the other way around, it is possible to make clients for other platforms than Android to connect to a server.
I was told once that in order for the Android Phone to contact a PC
server, that server must be written in Java.
That is incorrect. The whole point of TCP/IP was to create a set of rules, a protocol to allow communication across different devices, architectures, operating systems etc.
Any TCP/IP client can communicate with any TCP/IP server period. No exceptions. (provided they implement a common version of the protocol that is)
I'm building a multi-OS mirroring system which I would like to implement using a hybrid client-server and p2p communication method (at least that's the best way I have of describing it).
My issue is that at some point I have a central server (appengine, so there are limitations to what I could do because of time and networking capability constraints) that would need to get a message to a host of different devices which are not necessarily running the same OS (Windows, Android, iOS, Linux, etc...).
Android and iOS (or any other mobile platform) are the main problems it looks like I will be having on 2 levels.
1 - They are both limited by battery power (more so than a laptop and desktops shouldn't have that issue at all), so whichever method I use needs to take that into account.
2 - NAT (harder because the user has relatively less control over their firewall than on a network that they are running). My central server will maintain a table of which device has what IP address, but from what I understand if there is NAT or a firewall it won't be able to get to it if the port was not forwarded.
Since I will be writing a specific client for each OS I prefer a solution that is more universal. I have been leaning towards writing an extremely simple HTTP server that sits on each client and takes requests (which appengine is able to send) and treats them as messages that alert the client to perform an action (either with the server or another client). However, I run into the issue of NAT/firewall. For instance if appengine needs to send a message to AndroidDevice1 it would grab its IP address from a table and make a request to it. However this doesn't work if the ports aren't forwarded correctly, and if the user is on 3g/4g the firewall is controlled by the data provider.
Because of this, I started thinking about using Android C2DM but I want a solution I could implement across platforms.
The only other method I could think of is to just have the client poll the server for messages. This has the battery and network consumption issue though.
Would there be any other way to implement this, and if not, which one of the above methods are best in terms of balancing usability, power and data consumption and user input (the less the user has to do to get the client set up (ie port forwarding, etc...) the better)? Please note that I do not intend for this to become a discussion/flame war but a logical presentation of facts.
Thanks in advance!
You can create a persistent TCP connection from the device to the server and then communicate over this open connection. This would be a very simple connection with keepalive packets for the most part.
In theory this would consume some battery through the radio, but in practice I have experienced that the battery is not affected much at all. Key is to keep the communication over this line to a minimum.
If AppEngine does not allow this approach, you can run your own socket server and then communicate between this server and the appengine server using REST. A socket server I have used is Apache MINA and had no issues with scalability.
Another problem you will have with this approach or any other approach is that on iOS (afaik) you cannot keep a tcp socket open when the App goes into background. The only thing to communicate with the iOS device is Apple Push Notification Service
I would prefer rather than having HTTP Connection you should create TCP/IP tunnel and make communication fast and reliable. I have one Chat application which runs perfact for me using TCP/IP. If you use this you will have same logic for multiple platforms. Only thing you need to write is different code for iOS and android.
I'm building native mobile applications in both iOS and Android. These apps require "realtime" updates from and to the server, same as any other network-based application does (Facebook, Twitter, social games like Words with Friends, etc)
I think using HTTP long polling for this is over kill in the sense that long polling can be detrimental to battery life, especially with a lot of TCP setup/teardown. It might make sense to have the mobile applications use persistent TCP sockets to establish a connection to the server, and send RPC style commands to the server for all web service communication. This ofcourse, would require a server to handle the long-lived TCP connection and be able to speak to a web service once it makes sense of the data passed down the TCP pipe. I'm thinking of passing data in plain text using JSON or XML.
Perhaps an Erlang based RPC server would do well for a network based application like this. It would allow for the mobile apps to send and receive data from the server all over one connection without multiple setup/teardown that individual HTTP requests would do using something like NSURLConnection on iOS. Since no web browser isn't involved, we don't need to deal with the nuances of HTTP at the mobile client level. A lot of these "COMET" and long-polling/streaming servers are built with HTTP in mind. I'm thinking just using a plain-text protocol over TCP is good enough, will make the client more responsive, allow for receiving of updates from the server, and preserve battery life over the traditional long polling and streaming models.
Does anyone currently do this with their native iOS or Android app? Did you write your own server or is there something open sourced out there that I can begin working with today instead of reinventing the wheel? Is there any reason why using just a TCP based RPC service is a worse decision than using HTTP?
I also looked into HTTP pipelining, but it doesn't look to be worth the trouble when it comes to implementing it on the clients. Also, I'm not sure if it would allow for bi-directional communication in the client<->server communication channel.
Any insight would be greatly appreciated.
Using TCP sockets with your own protocol rolled down is quite better than HTTP especially with the nature of resources on the mobile devices. Erlang will do quite well, however lets start from your protocol. Erlang excels well at this especially with the Bit Syntax expressions. However still, you could use plain text as you wish. JSON (would need a parser: Mochijson2.erl found in Mochiweb library) and XML (will need a parser: Erlsom).
I have personally worked on a project in which we were using raw TCP Sockets with our Erlang Servers and Mobile Devices. However, depending on the Port numbers you choose, Routers along the way would block/Drop packets depending on the security policies of service providers. However, i still think that HTTP can work. People chat on Facebook Mobile, send Twits e.t.c from their devices and am sure these social engines use some kind of Long Polling or Server Push or whatever but using HTTP. The mobile devices have advanced in capability of late.
Rolling your own TCP Based protocol comes with a number of challenges: Port selection, Parsing of data both at the client and server, Security issues e.t.c. Using HTTP will let you think of the actual problem than spending time correcting protocol issues at client or server. The Devices you've mentioned above like Android and IOS (Ipad, Iphone e.t.c) are very capable of handling HTTP COMET (Long polling). Am sure when you follow the standards for Web Applications on Mobile devices as well as these W3C Mobile Web Best Practices, your app will function well using HTTP.
Using HTTP methods will quicken the work and there are a lot of libraries on the SDKs of these Devices which would assist you prototype the solution you want as compared to the situation of rolling your own TCP-based plain text protocol. To back up this reasoning, look through these W3C findings.
Let me finally talk of the HTTP benefits on these Devices. If you are to use Web technologies for Mobile devices, such as Opera Widgets, Phone Gap, Sencha Touch, and JQuery Mobile, their SDKs and Libraries have Optimizations already done for you or have well documented ways in which your app can be made efficient. Further still, these technologies have the APIs to access the native Devices' resources like Battery check, SMS, MMS, GSM broadcast channels, Contacts, Lighting, GPS , and Memory; all as APIs in the JavaScript classes. It would become hard (inflexible) if you use native programming languages like J2ME, Mobile Python or Symbian C++ / Qt as compared to using Web technologies like CSS3, HTML5 and JavaScript tools mentioned above. Using the Web tools mentioned above will make your app easily distributable by say Ovi Store or Apple Store, from experience.
Take note that if you use HTTP, testing will be easy. All you need is a public Domain so the Widgets on the mobile device locates your servers over the Internet. If you role your own TCP/IP protocol, the Network Routers may be disruptive against the Port number you use unless you plan on using port 80 or another well known port, but then still your Server IP would have to be made Public. There is a short cut to this: if you put your TCP Server behind the same ISP as your testing Mobile's Internet connection, the ISP routers will see both source and destination as behind its Network. But all in all, there are challenges with rolling your own protocol.
Edit: Using HTTP, you will benefit from REST. Web Servers implemented in Erlang (especially Yaws and Mochiweb) excel at REST services. Look at this article: RESTFUL services with Yaws. For mochiweb, there is an interesting article about: A million User comet application using Mochiweb which is broken into 3 parts. Further still, you could look at the solution given to this question.
There are ZeroMQ builds for android and iOS. Java and ObjC bindings exist as well.
HTTP was created for infrequent requests with large responses. It is highly inefficient for transferring very big amounts of small data chunks. In typical situation, http headers can be twice in size of actual payload. The only strong side of HTTP is its habitualness, its 'One size fits all' karma.
If you want lightweight and fast solution, I guess ZeroMQ can be a perfect solution.
One reason to go with HTTP instead of a custom service is that it's widely supported on a transport level.
With mobile devices, a user might be on Wi-Fi at a hotel, airport, coffee shop, or corporate LAN. In some cases this means having to connect via proxy. Your application's users will be happiest if the application is able to use the device's proxy settings to connect. This provides the least surprise -- if web browsing works, then the application should work also.
HTTP is simple enough that it isn't difficult to write a server that will accept HTTP requests from a custom client. If you decide to go this route, the best solution is the one that you don't have to support. If you can write something in Erlang that is supportive of application changes, then it sounds like a reasonable solution. If you're not comfortable doing so then PHP or J2EE gets bonus points for the availability of cheap labor.
While HTTP does benefit from being widely supported, some successful projects are based on other protocols. The Sipdroid developers found that persistent TCP connections do greatly improve battery life. Their article on the topic doesn't address the server side but it does give a high-level description of their approach on the client.
Erlang is very well suited for your use case. I'd prefer using TCP over HTTP for the sake of saving battery life on the phone as you noted already.
Generally getting the communication between device and server up and running will be very easy. The protocol which you are using between the two is what will require most work. However writing protocols in Erlang is strikingly straight forward when using gen_fsm
You should checkout metajack's talk at the Erlang Factory which highlights his solution to a very similar use case for his iPhone game Snack Words.
I work on a application that connects to a Microsoft http server with long lived http/https connections to mobile devices to allow for push type data to be sent to the mobile. It works but there are lots of little gotcha's on the mobile side.
For the client to get 'packets' of data, we put the http connection into Chucked Encoding mode so that each packet is in one chucked packet.
Not all native http API services on each mobile will support calling you back when a 'chuck' of data has arrived, on the ones that don't normally wait until all the data from the server has arrived before calling the application back with the data. Platforms that support callbacks with partial data are (that I have found):
Symbian
Windows Mobile
Platforms that don't support partial data callbacks:
IOS
Blackberry
For the platforms that don't support partial callbacks, we have written our own http connection code with chucked encoding support using the native sock support. It's actually not very hard.
Don't rely on the fact that one chuck is one of your packets, http proxies or the native http api implementations may break that assumption.
On IOS with this background multitasking rules, means you can't keep this connection going while your application is in the background. You really need to use Apples Push Notification service and live by it's limitations.
Never trust mobile cellular networks, I have seen the weirdest stuff going on like the server side seeing the http connection drop and then reconnect (and replay of the original http request) while on the mobile end you don't see any drop in the connection. Basically treat the connection as unreliable where data can go missing. We ended up implementing a 'tcp' like sequence number scheme to ensure we didn't lose data.
Using http/https makes it easier to get past firewall rules on customer sites.
I'm not sure using http/https long-lived connections was the wisest decision we ever made, but it was made long before I turned up so I have to live with the fall-out of it.
As a alterative, we are looking at web sockets as well, but with the web-socket spec in the state of flux atm and generally being not to good to follow, I don't know if it will work out or not.
So that is my experience with using http/https as a long-lived realtime connection.
Your milage may vary.
It all depends on what data you are sending - the size of it, the criticality of timeliness, frequency of update etc.
If you are looking for a reasonably lazy update and verbose data (JSON say) then go with a HTTP comet pattern, as you will find it much easier to navigate standard network gear as other answers have highlighted. If you are behind a corporate firewall/proxy for example, http will be a much safer bet.
However, if you are doing fast things with small data sizes then go with something homegrown and leverage a TCP connection. It's much more to the point and you'll find the performance in real terms much better. Simple data structures and use fast operators to slice you data up as you need it.
Again as other posters have noted, battery usage is a big concern. You will eat a battery by literally burning a hole in your pocket if you are not careful. It is very easy to turn a battery that lasts 2 days into one that will last 6hours.
Lastly, don't trust the network if you are time sensitive. If you are not then a long poll over HTTP will be just fine for you. But if you are looking for high performance messaging, then be acutely aware that a mobile network is not an end-to-end TCP connection. Your requests will varying in trip time and latency.
So back to what you want to do with the app. As you are building for iOS (native obviously dictated) and Andriod, I would leverage Apple Push Services and their notification framework. Build you back end services to talk to that and also provide interfaces for non-apple devices (i.e. http or tcp level listeners). That way one platform and multiple 'gateways' for your apps. You can then do RIM via their push service too if you wanted to.