I'm considering using a persistent connection to a "cloud service" from an Android app. This would run all the time in a background service (or something like that).
I'm thinking of using web sockets or XMPP to keep the connection, basically looking for a light weight connection that won't drain battery. I want to be able to push notifications in real time to this connection, so periodic polling is not desired. I am aware of C2DM and other commercial solutions, but am looking to roll my own. This is why a web socket (or other light weight connection) is what I'm investigating. So if I go this route, what are some best practices I should be aware of?
I'm thinking of stuff like:
how to prevent the battery from draining,
How to handle IP address changes, etc?
This might not be the answer you are looking for but I think you may want to rethink your architecture.
Things you can expect out of a mobile platform
Your IP address to change randomly
Your physical internet connection to be lost randomly
The OS to decide your not doing anything useful and killing your process
The connection type changing randomly (from WIFI to 4G to 3G to edge) and thus your IP to change
Basically your app needs to be able to handle a loss of connection, because its almost guaranteed to happen.
That being said, it is totally doable depending on your definition of real-time. If your willing to continually check that there is still a viable connection, you could keep any delays down to the minutes range. But this will drain the battery and there is not much you can do about it.
Some things just don't go well together. That is "push notifications in real time" and "prevent battery draining". You sure have to make compromises here.
I can only recommend to try some Android Apps that use XMPP to get a feeling how they handle persistent connections, IP address changes and battery consumption. If they are open-source you can also view the code and learn from it. Yaxim, Project MAXS and Beem to name a few. Maybe you shoud also have a look at XEP-0286: XMPP on Mobile Devices
That said, are you sure that you want to reinvent the wheel when Google offers you C2DM? Which is optimized for this use case. I think that it has some delay, so it's no where "real-time". But again, either you will end up with an solution that tries aggressivly to establish a persisent connection and drains the battery, or you will have to live with some kind of delay (~ 0-30 min).
Related
My app is single activity app which is used to generate token slip using WiFi printer over wifi which are connected locally. So the WiFi is always ON. Also the screen is always ON.
I have set android:keepScreenOn="true" in my activity xml file for this.
The tablet is exclusively used for this single app only not more than that. Even though, the battery drains around 4 hours.
Is this common? Or Would I change anything to achieve good battery backup?
Generally if your device is old, There is a chance to expect this kind of situation.
To debug about your network traffic, Follow this link
The network traffic generated by an app can have a significant impact
on the battery life of the device where it is running. In order to
optimize that traffic, you need to both measure it and identify its
source. Network requests can come directly from a user action,
requests from your own app code, or from a server communicating with
your app.
Link is here
Can someone answer on my dilemma which method to use for connecting Android device to mySQL or Postgresql?
I can do it in both ways without any errors and problems, with no noticeable difference but everyone recommend web service instead of using jdbc driver and direct connection,
Can someone explain why with some facts?
EDIT: I did'n mention that is more simple and needs less time to do it over jdbc. So, why web service, or why not?
You think it's simpler and faster to do it with JDBC because you aren't considering the real world operating environment of phones and portable devices. They often have flakey connectivity through buggy traffic rewriting proxies and insane firewalls. They're typically using a network transport layer that has high and variable packet loss rates and latencies that vary over many orders of magnitude in short spans of time. TCP really isn't great in this environment and particularly struggles with long lived connections.
The key benefit of a web service is that it:
Has short-lived connections with minimal state, so it's easy to get back to where you were when the device switches WiFi networks, to/from cellular, loses connectivity briefly, etc; and
Can pass through all but the most awful and draconian web proxies
You will routinely encounter problems with a direct JDBC connection. One challenge is reliably timing out dead connections, re-establishing sessions and releasing locks held by the old session (as the server may not decide it's dead at the same time the client does). Another is packet loss causing very slow operations, long-running database transactions, and consequent problems with lock durations and transactional cleanup tasks. You'll also meet every variety of insane and broken proxy and firewall under the sun - proxies that support CONNECT but then turn out to assume all traffic is HTTPs and mangle it if it isn't; firewalls with buggy stateful connection tracking that cause connections to fail or go to a half-open zombie state; every NAT problem you can imagine; carriers "helpfully" generating TCP ACKs to reduce latency, never mind the problems that causes with packet loss discovery and window sizing; wacky port blocking; etc.
Because everyone uses HTTP, you can expect that to work - at least, vastly more often than anything else does. This is particularly true now that common websites use REST+JSON communication style even in mobile web apps.
You can also write your web service calls to be idempotent using unique request tokens. That lets your app re-send modification requests without fear that it'll perform an action against the database twice. See idempotence and definining idempotence.
Seriously, JDBC from a mobile device might look like a good idea now - but the only way I'd even consider it would be if the mobile devices were all on a single high-reliably WiFi network under my direct control. Even then I'd avoid it for reasons of database performance management if I possibly could. You can use something like PgBouncer to pool connections among many devices at the server side so connection pooling isn't a big problem, but cleanup of lost and abandoned connections is, as is the tcp keepalive traffic required to make it work and the long stalled transactions from abandoned connections.
I can think of a few reasons
JDBC android driver support for your database.
Connection pooling across various Android devices make it difficult to monitor and cap them.
Result sets sent from the DB to android will consume a lot of bandwidth and battery power.
Proxies usuall allow HTTP access to your device.
Exposing your database directly to the client has security implications.
Web services can provide additional features on top of the JDBC connection like authentication / quality of service / authorization / conditional GET requests / error handling etc. JDBC cannot do any of these.
Besides all things Craig Ringer said, which I completely agree, JDBC has another problem: it will force to expose your database to the world. If you want android devices to access it, you will need to provide your app with database credentials, and the database will have to have public access.
Using a WebService or RESTful API is clearly the way to go to make your application secure.
Another option would be to use a database sync tool like SymmetricDS.
This would let you have say a Postgres database on your server, and a SQLite database on your tablet.
SymmetricDS would synchronize the databases over HTTP, when a connection is available. You don't have to sync the whole db of course, just the relevant parts.
(I am not affiliated with SymmetricDS)
TL;DR: It depends!
(Sorry to all the "never ever ever ever do it, direct conns are always evil"-devs)
When creating a public domain / general app for the playstore kind of thing, I am mainly with my fellow responders. Opening your DB to "everyone" (especially when permissions are badly or not at all configured) is typically not a great idea!!
However(!), the story might be totally different, when you e.g. create something for internal use within the network boundaries of your company, like Android handheld devices for logistics, inventory, etc. In these cases I would even most of the time definately recommend going with JDBC or a similar direct connection. Reaons being:
One less point of failure
One less development (sub-)project
One less thing to maintain and keep up to date with your data-structure
One less thing to keep up and running, CI/CD, test, etc. (you get the draft)
Which - im my humble opinion - is worse than the (implement it once) effort of connection pooling, reestablishment, etc. (if it really becomes necesseary, be careful with premature optimization there).
But for public projects ... well, if they only ever require read access, I could possibly imagine it as well, or if there are only certain tables were you allow adding, but not delete or modifications. There are some tricks you could apply to make it still secure (allowing adds but not reads with id-secrets for a certain table, triggers, and general reads for other tables, etc.), but there is a lot to think and a lot to miss about these. So generally, I would say it is bad practice to allow your public domain client to get hold of your SQL connection. But still, don't let that hinder you to ask yourself (and understand) "why" and look at the specific situation. There might even be good cause/use for that. Especially since it is "less", which is also often better. It definately depends.
Just be careful and aware that (even if permissions are set correctly) a lot can be misused (and only little hindered), with a direct connection at your client. (Plus possible connection issues to be taken care of.)
As a sidenote: A lot of these considerations are relevant again with the use of technologies like GraphQL, which shares some similarities (however without connection issues and with a little bit more secure control).
I've read this tutorial about data transfer in a battery efficient way.
All the lessons are based on one, simple concept: polling the server is Android is battery inefficient. For this reason, Google Cloud Messaging is introduced in order to send messages from the server to the device only when needed.
There is only one problem: I'm trying to implement a "mobile cloud", so a cloud composed by mobile devices, where each device can join/leave the network with high frequency. So I need some mechanism to detect when a device is not reachable anymore. Until now, in all the works that I've seen on the topic, the only solution was to periodically ping the main server to say "Hey, I'm still alive!" from the mobile device. Obviously this solution is battery killing, but until now I've not seen/found any better solution.
Do you know any battery efficient solution for this problem?
There's no reason why pinging the server periodically (heartbeat) is necessarily wasteful of the battery/inefficient. That depends upon how frequently you need to ping, and whether your ping needs to initiate its own transmission vs piggy backing on some other transmission.
Let me explain. Battery inefficiency depends upon whether or not you are increasing the frequency or duration of the transceiver being in an active state. If the transceiver is continually active anyway, such as it is continuously exchanging data or audio, then a heartbeat adds no additional burden. If it is not active, then there will be additional energy usage but that depends upon the frequency of your heartbeat compared to how long a ping will cause the transceiver to be powered. Even then, it's probably irrelevant to your application as I suspect "cloud" means the devices are active and connected.
Let's assume that your heartbeat is such that it will increase the duration of your transceiver being active. There are still techniques you can use to decrease this impact, such as caching your beat and sending it only when it can piggy back on another transmission. Of course, such solutions depend upon whether your heartbeat is implemented in an application, OS or kernel.
I suggest you do actual tests to see if there is truly an impact on your devices.
PS I'm not saying the tutorial is wrong. It isn't. But it is addressing a broader and more general problem then what you have.
Can someone answer on my dilemma which method to use for connecting Android device to mySQL or Postgresql?
I can do it in both ways without any errors and problems, with no noticeable difference but everyone recommend web service instead of using jdbc driver and direct connection,
Can someone explain why with some facts?
EDIT: I did'n mention that is more simple and needs less time to do it over jdbc. So, why web service, or why not?
You think it's simpler and faster to do it with JDBC because you aren't considering the real world operating environment of phones and portable devices. They often have flakey connectivity through buggy traffic rewriting proxies and insane firewalls. They're typically using a network transport layer that has high and variable packet loss rates and latencies that vary over many orders of magnitude in short spans of time. TCP really isn't great in this environment and particularly struggles with long lived connections.
The key benefit of a web service is that it:
Has short-lived connections with minimal state, so it's easy to get back to where you were when the device switches WiFi networks, to/from cellular, loses connectivity briefly, etc; and
Can pass through all but the most awful and draconian web proxies
You will routinely encounter problems with a direct JDBC connection. One challenge is reliably timing out dead connections, re-establishing sessions and releasing locks held by the old session (as the server may not decide it's dead at the same time the client does). Another is packet loss causing very slow operations, long-running database transactions, and consequent problems with lock durations and transactional cleanup tasks. You'll also meet every variety of insane and broken proxy and firewall under the sun - proxies that support CONNECT but then turn out to assume all traffic is HTTPs and mangle it if it isn't; firewalls with buggy stateful connection tracking that cause connections to fail or go to a half-open zombie state; every NAT problem you can imagine; carriers "helpfully" generating TCP ACKs to reduce latency, never mind the problems that causes with packet loss discovery and window sizing; wacky port blocking; etc.
Because everyone uses HTTP, you can expect that to work - at least, vastly more often than anything else does. This is particularly true now that common websites use REST+JSON communication style even in mobile web apps.
You can also write your web service calls to be idempotent using unique request tokens. That lets your app re-send modification requests without fear that it'll perform an action against the database twice. See idempotence and definining idempotence.
Seriously, JDBC from a mobile device might look like a good idea now - but the only way I'd even consider it would be if the mobile devices were all on a single high-reliably WiFi network under my direct control. Even then I'd avoid it for reasons of database performance management if I possibly could. You can use something like PgBouncer to pool connections among many devices at the server side so connection pooling isn't a big problem, but cleanup of lost and abandoned connections is, as is the tcp keepalive traffic required to make it work and the long stalled transactions from abandoned connections.
I can think of a few reasons
JDBC android driver support for your database.
Connection pooling across various Android devices make it difficult to monitor and cap them.
Result sets sent from the DB to android will consume a lot of bandwidth and battery power.
Proxies usuall allow HTTP access to your device.
Exposing your database directly to the client has security implications.
Web services can provide additional features on top of the JDBC connection like authentication / quality of service / authorization / conditional GET requests / error handling etc. JDBC cannot do any of these.
Besides all things Craig Ringer said, which I completely agree, JDBC has another problem: it will force to expose your database to the world. If you want android devices to access it, you will need to provide your app with database credentials, and the database will have to have public access.
Using a WebService or RESTful API is clearly the way to go to make your application secure.
Another option would be to use a database sync tool like SymmetricDS.
This would let you have say a Postgres database on your server, and a SQLite database on your tablet.
SymmetricDS would synchronize the databases over HTTP, when a connection is available. You don't have to sync the whole db of course, just the relevant parts.
(I am not affiliated with SymmetricDS)
TL;DR: It depends!
(Sorry to all the "never ever ever ever do it, direct conns are always evil"-devs)
When creating a public domain / general app for the playstore kind of thing, I am mainly with my fellow responders. Opening your DB to "everyone" (especially when permissions are badly or not at all configured) is typically not a great idea!!
However(!), the story might be totally different, when you e.g. create something for internal use within the network boundaries of your company, like Android handheld devices for logistics, inventory, etc. In these cases I would even most of the time definately recommend going with JDBC or a similar direct connection. Reaons being:
One less point of failure
One less development (sub-)project
One less thing to maintain and keep up to date with your data-structure
One less thing to keep up and running, CI/CD, test, etc. (you get the draft)
Which - im my humble opinion - is worse than the (implement it once) effort of connection pooling, reestablishment, etc. (if it really becomes necesseary, be careful with premature optimization there).
But for public projects ... well, if they only ever require read access, I could possibly imagine it as well, or if there are only certain tables were you allow adding, but not delete or modifications. There are some tricks you could apply to make it still secure (allowing adds but not reads with id-secrets for a certain table, triggers, and general reads for other tables, etc.), but there is a lot to think and a lot to miss about these. So generally, I would say it is bad practice to allow your public domain client to get hold of your SQL connection. But still, don't let that hinder you to ask yourself (and understand) "why" and look at the specific situation. There might even be good cause/use for that. Especially since it is "less", which is also often better. It definately depends.
Just be careful and aware that (even if permissions are set correctly) a lot can be misused (and only little hindered), with a direct connection at your client. (Plus possible connection issues to be taken care of.)
As a sidenote: A lot of these considerations are relevant again with the use of technologies like GraphQL, which shares some similarities (however without connection issues and with a little bit more secure control).
I'm currently working on an android project and I am trying to find the best way to go about setting up communication between two android phones.
One android phone will be docked on a mobile platform e.g. an R/C car. I want this phone to receive simple control signals ("forward", "backward", "left", "right", "gotoCoordinate") sent from another android phone. I also want the docked phone to be able to return status signals.
Preferably I want the communication to happen via GPRS. I'm aware of the difficulties concerning P2P-communications and I'm currently looking into "Android Cloud To Device Messaging." (http://code.google.com/android/c2dm/index.html)
I'd like to hear about your experience with Android C2DM (glad to hear about delay from transmit to receive) and your thoughts on utilizing it in my project. I'd appreciate other suggestions on how to go about this. I'm expecting to have to deal with relatively high latency using this method, but of course preferably lowest possible.
C2DM makes no guarantee about the "delivery or order" of the messages, and it is limited in the number of messages you can send (a high limit, but still a limit). It's not really for low-latency stuff like controlling an RC car. It's better for non-realtime events.
http://code.google.com/android/c2dm/
For lower latency stuff using GPRS you can setup a third party server on your own and have both phones communicate through it. I've done that for several Android apps using straight up TCP sockets and it works reasonably well (and it would be even faster/better if you went UDP). Using GPRS may still have too much latency, depending on your needs, but it's a tradeoff (it's very convenient, almost always there, other methods are not).
The ideal way to do this would be to combine whatever is available and fallback gracefully, and test the latency once connected to make sure the network is up to par, or bail out. For example, use the local WiFi network if it's available. That is to say, have both devices "register" with a third party server as they startup, then if they're both on the same WiFi just have them communicate directly (run a server on or both, and clients on one or both, get information about discovery and such from the registration). If they are not on WiFi then fall back to GPRS, but realize there will be more latency, of course. Finally, once any method has been established send some test messages to check latency.
I know this isn't really an "answer," it's more of a stream of consciousness about this, but it wouldn't fit in a comment, and I thought it might help ;).
(Full disclosure: I've worked on Android apps that connect multiple mobile devices and multiple TVs, some over GPRS, some Wifi, some directly. I work for a company (MOVL) that makes a platform for stuff like that, it's more focused on mobile-TV-mobile, but it supports mobile-mobile also. In all it's not too hard to do yourself with regular networking, the tricky part is getting the latency down and picking the correct method for each device.)