I'm developing an none market appliction which run on 20-30 android devices (target specific to tablets with android honeycomb / ICS OS) maintaining connection over local WIFI network for a 1-2 hours period of time, and need to exchange data (simple objects representing commands) between them.
most of the time one specific tablet behave like a server which sending the commands, and the other devices like clients which receives the commands, but the "clients" also sending commands to the "server" sometimes.
as solution to this communication demand - I'm using for a while an open source
library which encapsulates TCP client/server protocol, called - Kryonet.
I found it very easy to use, and basically doing the job, although it sometimes "unstable" - a lot of disconnections accrues. I can't afford this disconnections, it's dammege the whole flow and use-case, causing the client's to lose commands.
I'm doing some recovery logic which re-connect the clients and send them what they have missed, but it's not good enough for the use-case.
recently I've heard about multicast broadcast protocol, and found even an open source library calls - JGroups which implement this protocol optimally, and expose easy and simple to use interface. still didn't tried it, but got advice from someone who knows, saying it should be better the the TCP client/server for my purpose.
what is the best approach I should use for implement the behavior I described ? (not necessarily one of the two I suggested)
TIA
Although JGroups has promise as a better solution for your situation, you may want to experiment a bit more to determine why the disconnects are happening. Since your clients and server are all tablets, there are a few other causes that are unrelated:
1) If the connections are not being maintained in a Service then they will be extremely unreliable by default. (See this question about singletons being destroyed in Android)
2) If the sockets have not been set to 'keepalive' then they will time out after an arbitrary number of seconds.
3) The devices you are using may shut down some persistent connections when they go to sleep.
4) The tablets may be exiting WiFi range, and switching over to a mobile network.
Try the network portion of your code on a number of desktop machines to determine if the problem is with Kryonet or your code, or if the problem is in running it on Android.
Related
Can someone answer on my dilemma which method to use for connecting Android device to mySQL or Postgresql?
I can do it in both ways without any errors and problems, with no noticeable difference but everyone recommend web service instead of using jdbc driver and direct connection,
Can someone explain why with some facts?
EDIT: I did'n mention that is more simple and needs less time to do it over jdbc. So, why web service, or why not?
You think it's simpler and faster to do it with JDBC because you aren't considering the real world operating environment of phones and portable devices. They often have flakey connectivity through buggy traffic rewriting proxies and insane firewalls. They're typically using a network transport layer that has high and variable packet loss rates and latencies that vary over many orders of magnitude in short spans of time. TCP really isn't great in this environment and particularly struggles with long lived connections.
The key benefit of a web service is that it:
Has short-lived connections with minimal state, so it's easy to get back to where you were when the device switches WiFi networks, to/from cellular, loses connectivity briefly, etc; and
Can pass through all but the most awful and draconian web proxies
You will routinely encounter problems with a direct JDBC connection. One challenge is reliably timing out dead connections, re-establishing sessions and releasing locks held by the old session (as the server may not decide it's dead at the same time the client does). Another is packet loss causing very slow operations, long-running database transactions, and consequent problems with lock durations and transactional cleanup tasks. You'll also meet every variety of insane and broken proxy and firewall under the sun - proxies that support CONNECT but then turn out to assume all traffic is HTTPs and mangle it if it isn't; firewalls with buggy stateful connection tracking that cause connections to fail or go to a half-open zombie state; every NAT problem you can imagine; carriers "helpfully" generating TCP ACKs to reduce latency, never mind the problems that causes with packet loss discovery and window sizing; wacky port blocking; etc.
Because everyone uses HTTP, you can expect that to work - at least, vastly more often than anything else does. This is particularly true now that common websites use REST+JSON communication style even in mobile web apps.
You can also write your web service calls to be idempotent using unique request tokens. That lets your app re-send modification requests without fear that it'll perform an action against the database twice. See idempotence and definining idempotence.
Seriously, JDBC from a mobile device might look like a good idea now - but the only way I'd even consider it would be if the mobile devices were all on a single high-reliably WiFi network under my direct control. Even then I'd avoid it for reasons of database performance management if I possibly could. You can use something like PgBouncer to pool connections among many devices at the server side so connection pooling isn't a big problem, but cleanup of lost and abandoned connections is, as is the tcp keepalive traffic required to make it work and the long stalled transactions from abandoned connections.
I can think of a few reasons
JDBC android driver support for your database.
Connection pooling across various Android devices make it difficult to monitor and cap them.
Result sets sent from the DB to android will consume a lot of bandwidth and battery power.
Proxies usuall allow HTTP access to your device.
Exposing your database directly to the client has security implications.
Web services can provide additional features on top of the JDBC connection like authentication / quality of service / authorization / conditional GET requests / error handling etc. JDBC cannot do any of these.
Besides all things Craig Ringer said, which I completely agree, JDBC has another problem: it will force to expose your database to the world. If you want android devices to access it, you will need to provide your app with database credentials, and the database will have to have public access.
Using a WebService or RESTful API is clearly the way to go to make your application secure.
Another option would be to use a database sync tool like SymmetricDS.
This would let you have say a Postgres database on your server, and a SQLite database on your tablet.
SymmetricDS would synchronize the databases over HTTP, when a connection is available. You don't have to sync the whole db of course, just the relevant parts.
(I am not affiliated with SymmetricDS)
TL;DR: It depends!
(Sorry to all the "never ever ever ever do it, direct conns are always evil"-devs)
When creating a public domain / general app for the playstore kind of thing, I am mainly with my fellow responders. Opening your DB to "everyone" (especially when permissions are badly or not at all configured) is typically not a great idea!!
However(!), the story might be totally different, when you e.g. create something for internal use within the network boundaries of your company, like Android handheld devices for logistics, inventory, etc. In these cases I would even most of the time definately recommend going with JDBC or a similar direct connection. Reaons being:
One less point of failure
One less development (sub-)project
One less thing to maintain and keep up to date with your data-structure
One less thing to keep up and running, CI/CD, test, etc. (you get the draft)
Which - im my humble opinion - is worse than the (implement it once) effort of connection pooling, reestablishment, etc. (if it really becomes necesseary, be careful with premature optimization there).
But for public projects ... well, if they only ever require read access, I could possibly imagine it as well, or if there are only certain tables were you allow adding, but not delete or modifications. There are some tricks you could apply to make it still secure (allowing adds but not reads with id-secrets for a certain table, triggers, and general reads for other tables, etc.), but there is a lot to think and a lot to miss about these. So generally, I would say it is bad practice to allow your public domain client to get hold of your SQL connection. But still, don't let that hinder you to ask yourself (and understand) "why" and look at the specific situation. There might even be good cause/use for that. Especially since it is "less", which is also often better. It definately depends.
Just be careful and aware that (even if permissions are set correctly) a lot can be misused (and only little hindered), with a direct connection at your client. (Plus possible connection issues to be taken care of.)
As a sidenote: A lot of these considerations are relevant again with the use of technologies like GraphQL, which shares some similarities (however without connection issues and with a little bit more secure control).
Background:
I have several apps that must connect to the same usb device on my android phone. In order to suppress constant permission requests and repeated unplug/plugs, I have built a background service to connect to the usb device. From there apps can communicate with the service.
Problem:
The apps need to pass every touch event to the usb device. All this traffic seems to create some delay (I have measured the delay for the passing the data on the order 10s and 100s of milliseconds). I use a bound service with a messenger/handler to pass the data right now.
Does anyone know of a faster service or a better way to accomplish this?
If your distribution scenario is constrained to your own firm, and not to the general public, you could go with a standard socket-based server approach. Rather than using a bound service and a Messenger, have your service open up a ServerSocket on a well-known port, and have the client apps connect to it.
It's conceivable that there's a way to use Unix-style domain sockets instead of TCP/IP sockets, but I'm not clear on whether that's possible through the Android SDK, only through the NDK, or not an option.
IOW, do it the same way you might on a desktop or a server.
Depending upon the nature of your device, you will still encounter delays, simply because the device is running M processes with N total threads on a CPU with C cores, and N usually is substantially greater than C. Context-switching between threads on the available cores takes time. This, of course, is substantially worse on single-core CPUs.
This approach would be scary for wide distribution, just for security reasons. Locking down the sockets to only be used by the desired apps would be your responsibility and may not be simple. But, if you are only going to use these devices in your own office, that's less of an issue.
Is it possible to set up the Android Bluetooth Chat sample app to connect more than one person at a time, and have a mini chat room? What would that entail?
tl;dr version: Bluetooth sucks for this, don't use it, use wifi instead, probably backed by a web backend.
I have investigated this issue thoroughly throughout the years in the interests of a social wireless network research project. My general advice is: it doesn't work with more than two / three people. Bluetooth just isn't designed with wireless peer to peer networks in mind.
In general, it seems that the cheap Bluetooth controllers included on Android devices (especially HTC's devices, iirc) don't really handle any more than two or three connections at a time. I'm unsure if this is a hardware or firmware problem, but I can recount some basic anecdotes. I was working to implement this idea at the SDK level (i.e., without firmware modifications) around the beginning of 2011, and was able to get a peer to get two additional connections (i.e., three devices, each connecting to the other two) to work for a period of a few minutes to an hour before the connections would suddenly die and the socket would become unusable, requiring reconnection. Unfortunately, 20 minutes was an upper bound, and generally it was impossible to get connections to more than one other device at all reliably.
The goal of the project was to support multiple people interacting with each other silently in the background, but this never materialized, instead we ditched Bluetooth and went with wifi instead, which worked much much better. In the abstract, I think people view Bluetooth as a possible medium for reliable peer to peer communication, but it wasn't really designed that way: it's more of a medium used for short range communication between small devices (think headsets).
Be aware that if you want to do this, the maximum number of devices to which you can connect is fixed, because as per the Bluetooth spec, a piconet supports a maximum of seven devices. (See the wikipedia article.)
The required change is simple: you use a different UUID for each device. This can be implemented a number of ways, using an out of band exchange mechanism, or simple scheme where you assign UUIDs in an increasing fashion and when connecting to the network, try each in succession.
Here are some relevant Google groups threads:
Bluetooth peer to peer networks
Multiple connections on Android Bluetooth
I remember posting a more elaborate one detailing how to do this (with code) that I might dig up as well.., if I can find it. It should be from late 2010 or early 2011.
So the answer is, in the abstract, yes, you can try to do this, by using multiple UUIDs (after you use one, that's it, and you have to try another using some assignment protocol). However, in practice, after a lot of trial and error, this doesn't really work for what you probably want to use it for, and it's a lot better to go with an internet backend instead. By the way, this is also good for another reason, most users don't really like to turn on their Bluetooth for fear of their battery being drained..
Leaving this here, in case it helps someone else.
I was able to make my custom chat room following official bluetooth tutorial and modifying it a little.
Unfortunately, I cannot provide most of my code, but main idea is:
Every device is acts both as server and as a client. When Chat is started, device starts its server thread. Server thread is the same as official but doesn't ends when accept connection. It just keep listening.
Client thread is identical as in tutorial.
Both server and client thread manages connection same. I created separated threads for accepting messages following this tutorial and one for sending them.
private void manageConnectedSocket(BluetoothSocket socket) {
//create thread responsible for sending messages.
SendingThread w = new SendingThread(socket);
MainActivity.addSendingThread(w);
//Creates listener for messages to accept.
MainActivity.addListener(socket);
}
Now in main activity always when user click send button, for each worker (sending thread) send message to remote device. Listening is running asynchronously.
IMPORTANT:
You need to handle exceptions when message send fails and remove sending and recieving thread for device when you detect it is disconected. In my case I used well known UUID "00001101-0000-1000-8000-00805f9b34fb". For every device.
You need to wait 3 second between atempts to connect as client because some devices has weak bluetooth hardware and it is refusing connect as client.
Bt connection is supporting up to 7 -10 connections. So you will be limited in that range. I think it is designed for extensions of main device and not for random comunication
Source: search "bluetooth programming" on google
I'm building a multi-OS mirroring system which I would like to implement using a hybrid client-server and p2p communication method (at least that's the best way I have of describing it).
My issue is that at some point I have a central server (appengine, so there are limitations to what I could do because of time and networking capability constraints) that would need to get a message to a host of different devices which are not necessarily running the same OS (Windows, Android, iOS, Linux, etc...).
Android and iOS (or any other mobile platform) are the main problems it looks like I will be having on 2 levels.
1 - They are both limited by battery power (more so than a laptop and desktops shouldn't have that issue at all), so whichever method I use needs to take that into account.
2 - NAT (harder because the user has relatively less control over their firewall than on a network that they are running). My central server will maintain a table of which device has what IP address, but from what I understand if there is NAT or a firewall it won't be able to get to it if the port was not forwarded.
Since I will be writing a specific client for each OS I prefer a solution that is more universal. I have been leaning towards writing an extremely simple HTTP server that sits on each client and takes requests (which appengine is able to send) and treats them as messages that alert the client to perform an action (either with the server or another client). However, I run into the issue of NAT/firewall. For instance if appengine needs to send a message to AndroidDevice1 it would grab its IP address from a table and make a request to it. However this doesn't work if the ports aren't forwarded correctly, and if the user is on 3g/4g the firewall is controlled by the data provider.
Because of this, I started thinking about using Android C2DM but I want a solution I could implement across platforms.
The only other method I could think of is to just have the client poll the server for messages. This has the battery and network consumption issue though.
Would there be any other way to implement this, and if not, which one of the above methods are best in terms of balancing usability, power and data consumption and user input (the less the user has to do to get the client set up (ie port forwarding, etc...) the better)? Please note that I do not intend for this to become a discussion/flame war but a logical presentation of facts.
Thanks in advance!
You can create a persistent TCP connection from the device to the server and then communicate over this open connection. This would be a very simple connection with keepalive packets for the most part.
In theory this would consume some battery through the radio, but in practice I have experienced that the battery is not affected much at all. Key is to keep the communication over this line to a minimum.
If AppEngine does not allow this approach, you can run your own socket server and then communicate between this server and the appengine server using REST. A socket server I have used is Apache MINA and had no issues with scalability.
Another problem you will have with this approach or any other approach is that on iOS (afaik) you cannot keep a tcp socket open when the App goes into background. The only thing to communicate with the iOS device is Apple Push Notification Service
I would prefer rather than having HTTP Connection you should create TCP/IP tunnel and make communication fast and reliable. I have one Chat application which runs perfact for me using TCP/IP. If you use this you will have same logic for multiple platforms. Only thing you need to write is different code for iOS and android.
I'm currently working on an android project and I am trying to find the best way to go about setting up communication between two android phones.
One android phone will be docked on a mobile platform e.g. an R/C car. I want this phone to receive simple control signals ("forward", "backward", "left", "right", "gotoCoordinate") sent from another android phone. I also want the docked phone to be able to return status signals.
Preferably I want the communication to happen via GPRS. I'm aware of the difficulties concerning P2P-communications and I'm currently looking into "Android Cloud To Device Messaging." (http://code.google.com/android/c2dm/index.html)
I'd like to hear about your experience with Android C2DM (glad to hear about delay from transmit to receive) and your thoughts on utilizing it in my project. I'd appreciate other suggestions on how to go about this. I'm expecting to have to deal with relatively high latency using this method, but of course preferably lowest possible.
C2DM makes no guarantee about the "delivery or order" of the messages, and it is limited in the number of messages you can send (a high limit, but still a limit). It's not really for low-latency stuff like controlling an RC car. It's better for non-realtime events.
http://code.google.com/android/c2dm/
For lower latency stuff using GPRS you can setup a third party server on your own and have both phones communicate through it. I've done that for several Android apps using straight up TCP sockets and it works reasonably well (and it would be even faster/better if you went UDP). Using GPRS may still have too much latency, depending on your needs, but it's a tradeoff (it's very convenient, almost always there, other methods are not).
The ideal way to do this would be to combine whatever is available and fallback gracefully, and test the latency once connected to make sure the network is up to par, or bail out. For example, use the local WiFi network if it's available. That is to say, have both devices "register" with a third party server as they startup, then if they're both on the same WiFi just have them communicate directly (run a server on or both, and clients on one or both, get information about discovery and such from the registration). If they are not on WiFi then fall back to GPRS, but realize there will be more latency, of course. Finally, once any method has been established send some test messages to check latency.
I know this isn't really an "answer," it's more of a stream of consciousness about this, but it wouldn't fit in a comment, and I thought it might help ;).
(Full disclosure: I've worked on Android apps that connect multiple mobile devices and multiple TVs, some over GPRS, some Wifi, some directly. I work for a company (MOVL) that makes a platform for stuff like that, it's more focused on mobile-TV-mobile, but it supports mobile-mobile also. In all it's not too hard to do yourself with regular networking, the tricky part is getting the latency down and picking the correct method for each device.)