I'm trying to learn into android (linux) kernel, and I know that android needs very fast (zero-copy) IPC, but still, I don't get the reason why binder needs to be there.
Can the same thing done with unix socket + mmap /dev/shm file ?
Let's say using dbus, but to achieve zero-copy, create and open file in tmpfs (e.g. /dev/shm) delete it (so another process can accidentally open it), send file descriptor to other process and mmap it.
EDIT:
instead of create file in tmpfs, you can use shm_open too
There are mainly three reason to create new Mechanism of Binder
Complexity
Performance
Security
Complexity - In the smartphone platform, especially the Android system, in order to provide application developers with a variety of functions, this communication method is ubiquitous, such as media playback, video and audio capture, and various sensors that make mobile phones smarter (acceleration, orientation, temperature, brightness, etc.) are all managed by different Servers, and the application can use these services only by establishing a connection with these Servers as a Client
Example : MediaPlayBack -media playback, video and audio capture, and various sensors that make mobile phones smarter (acceleration, orientation, temperature, brightness, etc.) are all managed by different Servers, and the application can use these services only by establishing a connection with these Servers as a Client. It takes a little time and effort to develop dazzling Function.The widespread adoption of the Client-Server approach poses a challenge to inter-process communication (IPC) mechanisms
Performance - As a general-purpose interface, socket has low transmission efficiency and high overhead. It is mainly used for inter-process communication across the network and low-speed communication between processes on the machine.
The message queue and pipeline adopt the store-and-forward method, that is, the data is first copied from the sender buffer to the buffer opened by the kernel, and then copied from the kernel buffer to the receiver buffer. There are at least two copy processes. Although shared memory does not need to be copied, it is complicated to control and difficult to use.
Security- As an open platform with many developers, Android comes from a wide range of sources, so it is very important to ensure the security of smart terminals. Traditional IPC does not have any security measures and relies entirely on upper-layer protocols to ensure it. First of all, the receiver of traditional IPC cannot obtain the reliable UID/PID (user ID/process ID) of the other party's process, so it cannot identify the other party's identity.
Android assigns its own UID to each installed application, so the UID of the process is an important symbol to identify the identity of the process. Using traditional IPC, only the user can fill in the UID/PID in the data packet, but this is unreliable and easy to be used by malicious programs.
Secondly, traditional IPC access points are open, and private channels cannot be established.
Based on the above reasons, Android needs to establish a new IPC mechanism to meet the system's requirements for communication methods, transmission performance and security, which is Binder
Related
Can someone answer on my dilemma which method to use for connecting Android device to mySQL or Postgresql?
I can do it in both ways without any errors and problems, with no noticeable difference but everyone recommend web service instead of using jdbc driver and direct connection,
Can someone explain why with some facts?
EDIT: I did'n mention that is more simple and needs less time to do it over jdbc. So, why web service, or why not?
You think it's simpler and faster to do it with JDBC because you aren't considering the real world operating environment of phones and portable devices. They often have flakey connectivity through buggy traffic rewriting proxies and insane firewalls. They're typically using a network transport layer that has high and variable packet loss rates and latencies that vary over many orders of magnitude in short spans of time. TCP really isn't great in this environment and particularly struggles with long lived connections.
The key benefit of a web service is that it:
Has short-lived connections with minimal state, so it's easy to get back to where you were when the device switches WiFi networks, to/from cellular, loses connectivity briefly, etc; and
Can pass through all but the most awful and draconian web proxies
You will routinely encounter problems with a direct JDBC connection. One challenge is reliably timing out dead connections, re-establishing sessions and releasing locks held by the old session (as the server may not decide it's dead at the same time the client does). Another is packet loss causing very slow operations, long-running database transactions, and consequent problems with lock durations and transactional cleanup tasks. You'll also meet every variety of insane and broken proxy and firewall under the sun - proxies that support CONNECT but then turn out to assume all traffic is HTTPs and mangle it if it isn't; firewalls with buggy stateful connection tracking that cause connections to fail or go to a half-open zombie state; every NAT problem you can imagine; carriers "helpfully" generating TCP ACKs to reduce latency, never mind the problems that causes with packet loss discovery and window sizing; wacky port blocking; etc.
Because everyone uses HTTP, you can expect that to work - at least, vastly more often than anything else does. This is particularly true now that common websites use REST+JSON communication style even in mobile web apps.
You can also write your web service calls to be idempotent using unique request tokens. That lets your app re-send modification requests without fear that it'll perform an action against the database twice. See idempotence and definining idempotence.
Seriously, JDBC from a mobile device might look like a good idea now - but the only way I'd even consider it would be if the mobile devices were all on a single high-reliably WiFi network under my direct control. Even then I'd avoid it for reasons of database performance management if I possibly could. You can use something like PgBouncer to pool connections among many devices at the server side so connection pooling isn't a big problem, but cleanup of lost and abandoned connections is, as is the tcp keepalive traffic required to make it work and the long stalled transactions from abandoned connections.
I can think of a few reasons
JDBC android driver support for your database.
Connection pooling across various Android devices make it difficult to monitor and cap them.
Result sets sent from the DB to android will consume a lot of bandwidth and battery power.
Proxies usuall allow HTTP access to your device.
Exposing your database directly to the client has security implications.
Web services can provide additional features on top of the JDBC connection like authentication / quality of service / authorization / conditional GET requests / error handling etc. JDBC cannot do any of these.
Besides all things Craig Ringer said, which I completely agree, JDBC has another problem: it will force to expose your database to the world. If you want android devices to access it, you will need to provide your app with database credentials, and the database will have to have public access.
Using a WebService or RESTful API is clearly the way to go to make your application secure.
Another option would be to use a database sync tool like SymmetricDS.
This would let you have say a Postgres database on your server, and a SQLite database on your tablet.
SymmetricDS would synchronize the databases over HTTP, when a connection is available. You don't have to sync the whole db of course, just the relevant parts.
(I am not affiliated with SymmetricDS)
TL;DR: It depends!
(Sorry to all the "never ever ever ever do it, direct conns are always evil"-devs)
When creating a public domain / general app for the playstore kind of thing, I am mainly with my fellow responders. Opening your DB to "everyone" (especially when permissions are badly or not at all configured) is typically not a great idea!!
However(!), the story might be totally different, when you e.g. create something for internal use within the network boundaries of your company, like Android handheld devices for logistics, inventory, etc. In these cases I would even most of the time definately recommend going with JDBC or a similar direct connection. Reaons being:
One less point of failure
One less development (sub-)project
One less thing to maintain and keep up to date with your data-structure
One less thing to keep up and running, CI/CD, test, etc. (you get the draft)
Which - im my humble opinion - is worse than the (implement it once) effort of connection pooling, reestablishment, etc. (if it really becomes necesseary, be careful with premature optimization there).
But for public projects ... well, if they only ever require read access, I could possibly imagine it as well, or if there are only certain tables were you allow adding, but not delete or modifications. There are some tricks you could apply to make it still secure (allowing adds but not reads with id-secrets for a certain table, triggers, and general reads for other tables, etc.), but there is a lot to think and a lot to miss about these. So generally, I would say it is bad practice to allow your public domain client to get hold of your SQL connection. But still, don't let that hinder you to ask yourself (and understand) "why" and look at the specific situation. There might even be good cause/use for that. Especially since it is "less", which is also often better. It definately depends.
Just be careful and aware that (even if permissions are set correctly) a lot can be misused (and only little hindered), with a direct connection at your client. (Plus possible connection issues to be taken care of.)
As a sidenote: A lot of these considerations are relevant again with the use of technologies like GraphQL, which shares some similarities (however without connection issues and with a little bit more secure control).
I want to ask some question regarding the development of my push-to-talk application for Android.
Recently, I've been developing push-to-talk Apps for Android. I use DatagramSocket to send voice and receive voice as a Packet over Local Wireless network (W-LAN). I use peer-to-peer network, so no server.
I don't have a problem with the code, but I don't understand the basic of VoIP theory, so I want to ask some question, hope somebody can give me simple answer :)
1. Is my push-to-talk Apps considered a VOIP-based?
2. There is several VOIP protocol such as SIP, H.323 and many more. If my PTT Apps considered a VOIP-based, and I use Socket-Packet (UDP, am I right?) to exchange voice, then what VOIP protocol that I use? Is it considered RTP protocol?
I would like to understand the theory behind my PTT apps, I understand my java code, but I don't have proper VOIP knowledge.
I've tried to find some information in google, but I still don't understand what is the relation between my PTT Apps and the VOIP technology.
Thanks before, I'm new here and sorry for my english!
I. "VoIP" is a very broad term, but, if your app transfers voice over IP network, it's definitely VoIP one, despite it may use totally proprietary protocols (as e.g. Skype does).
II. VoIP stack is basically split to two meta-layers - 1) signaling and 2) media transport. Each of them is in turn consisting multiple own layers (e.g. for SIP: session, dialog, transaction and transport layer). Examples of signaling protocols are H.323, SIP, MGCP. The most standard media transport is RTP. You can use your own transport; RTP applies specific restrictions (as AVP profile) but is compatible with variety of libraries and other implementations.
There are protocols which use the same socket and the same transport type for both signaling and media; the widely used one is IAX. Most others separate signaling and media, so sockets are separated and likely they have different type. A standard-compliant SIP implementation shall function over both UDP and TCP, and switch to TCP for large requests (>=1300 bytes by default); SCTP is also suggested. For all transports, protocol implementation details as retransmission policy and request timeout are specified in different way, but there is no principal problem with using any correct L4 protocol.
The totally another story is with media transport (under RTP or equivalent). Here, typical TCP manner to transfer all data at the cost of floating delays is really nasty for our ears. TCP is good for bulk traffic class, as file transfer or database interaction. In interactive communication between humans, we prefer more noise and sporadic voice loss than a voice strain. So, TCP is a very bad choice here, and a synchronous transport class shall be used, and UDP is the good default choice. SCTP can also be used as media transport, but with limited retransmit option (not all stacks support it). (There are attempts to use TCP to punch through NAT points but all this is an act of despair.)
If your application supposes sending a voice message more than to one recipient at a time (i.e. a kind of broadcast or multicast), this rejects use of connection-oriented media transports, effectively retaining only UDP. This also requires proper negotiation at signaling level.
III. Selection of voice codec is very platform-specific, I don't use which ones are native for Android. In "big" VoIP, there are licensed set and free set, with very small intersection (AFAIR, G.711 and GSM). Despite this, there are good codecs (e.g. Opus) which can be adapted into wide range of requirements, including partial packet loss.
Background:
I have several apps that must connect to the same usb device on my android phone. In order to suppress constant permission requests and repeated unplug/plugs, I have built a background service to connect to the usb device. From there apps can communicate with the service.
Problem:
The apps need to pass every touch event to the usb device. All this traffic seems to create some delay (I have measured the delay for the passing the data on the order 10s and 100s of milliseconds). I use a bound service with a messenger/handler to pass the data right now.
Does anyone know of a faster service or a better way to accomplish this?
If your distribution scenario is constrained to your own firm, and not to the general public, you could go with a standard socket-based server approach. Rather than using a bound service and a Messenger, have your service open up a ServerSocket on a well-known port, and have the client apps connect to it.
It's conceivable that there's a way to use Unix-style domain sockets instead of TCP/IP sockets, but I'm not clear on whether that's possible through the Android SDK, only through the NDK, or not an option.
IOW, do it the same way you might on a desktop or a server.
Depending upon the nature of your device, you will still encounter delays, simply because the device is running M processes with N total threads on a CPU with C cores, and N usually is substantially greater than C. Context-switching between threads on the available cores takes time. This, of course, is substantially worse on single-core CPUs.
This approach would be scary for wide distribution, just for security reasons. Locking down the sockets to only be used by the desired apps would be your responsibility and may not be simple. But, if you are only going to use these devices in your own office, that's less of an issue.
Can someone answer on my dilemma which method to use for connecting Android device to mySQL or Postgresql?
I can do it in both ways without any errors and problems, with no noticeable difference but everyone recommend web service instead of using jdbc driver and direct connection,
Can someone explain why with some facts?
EDIT: I did'n mention that is more simple and needs less time to do it over jdbc. So, why web service, or why not?
You think it's simpler and faster to do it with JDBC because you aren't considering the real world operating environment of phones and portable devices. They often have flakey connectivity through buggy traffic rewriting proxies and insane firewalls. They're typically using a network transport layer that has high and variable packet loss rates and latencies that vary over many orders of magnitude in short spans of time. TCP really isn't great in this environment and particularly struggles with long lived connections.
The key benefit of a web service is that it:
Has short-lived connections with minimal state, so it's easy to get back to where you were when the device switches WiFi networks, to/from cellular, loses connectivity briefly, etc; and
Can pass through all but the most awful and draconian web proxies
You will routinely encounter problems with a direct JDBC connection. One challenge is reliably timing out dead connections, re-establishing sessions and releasing locks held by the old session (as the server may not decide it's dead at the same time the client does). Another is packet loss causing very slow operations, long-running database transactions, and consequent problems with lock durations and transactional cleanup tasks. You'll also meet every variety of insane and broken proxy and firewall under the sun - proxies that support CONNECT but then turn out to assume all traffic is HTTPs and mangle it if it isn't; firewalls with buggy stateful connection tracking that cause connections to fail or go to a half-open zombie state; every NAT problem you can imagine; carriers "helpfully" generating TCP ACKs to reduce latency, never mind the problems that causes with packet loss discovery and window sizing; wacky port blocking; etc.
Because everyone uses HTTP, you can expect that to work - at least, vastly more often than anything else does. This is particularly true now that common websites use REST+JSON communication style even in mobile web apps.
You can also write your web service calls to be idempotent using unique request tokens. That lets your app re-send modification requests without fear that it'll perform an action against the database twice. See idempotence and definining idempotence.
Seriously, JDBC from a mobile device might look like a good idea now - but the only way I'd even consider it would be if the mobile devices were all on a single high-reliably WiFi network under my direct control. Even then I'd avoid it for reasons of database performance management if I possibly could. You can use something like PgBouncer to pool connections among many devices at the server side so connection pooling isn't a big problem, but cleanup of lost and abandoned connections is, as is the tcp keepalive traffic required to make it work and the long stalled transactions from abandoned connections.
I can think of a few reasons
JDBC android driver support for your database.
Connection pooling across various Android devices make it difficult to monitor and cap them.
Result sets sent from the DB to android will consume a lot of bandwidth and battery power.
Proxies usuall allow HTTP access to your device.
Exposing your database directly to the client has security implications.
Web services can provide additional features on top of the JDBC connection like authentication / quality of service / authorization / conditional GET requests / error handling etc. JDBC cannot do any of these.
Besides all things Craig Ringer said, which I completely agree, JDBC has another problem: it will force to expose your database to the world. If you want android devices to access it, you will need to provide your app with database credentials, and the database will have to have public access.
Using a WebService or RESTful API is clearly the way to go to make your application secure.
Another option would be to use a database sync tool like SymmetricDS.
This would let you have say a Postgres database on your server, and a SQLite database on your tablet.
SymmetricDS would synchronize the databases over HTTP, when a connection is available. You don't have to sync the whole db of course, just the relevant parts.
(I am not affiliated with SymmetricDS)
TL;DR: It depends!
(Sorry to all the "never ever ever ever do it, direct conns are always evil"-devs)
When creating a public domain / general app for the playstore kind of thing, I am mainly with my fellow responders. Opening your DB to "everyone" (especially when permissions are badly or not at all configured) is typically not a great idea!!
However(!), the story might be totally different, when you e.g. create something for internal use within the network boundaries of your company, like Android handheld devices for logistics, inventory, etc. In these cases I would even most of the time definately recommend going with JDBC or a similar direct connection. Reaons being:
One less point of failure
One less development (sub-)project
One less thing to maintain and keep up to date with your data-structure
One less thing to keep up and running, CI/CD, test, etc. (you get the draft)
Which - im my humble opinion - is worse than the (implement it once) effort of connection pooling, reestablishment, etc. (if it really becomes necesseary, be careful with premature optimization there).
But for public projects ... well, if they only ever require read access, I could possibly imagine it as well, or if there are only certain tables were you allow adding, but not delete or modifications. There are some tricks you could apply to make it still secure (allowing adds but not reads with id-secrets for a certain table, triggers, and general reads for other tables, etc.), but there is a lot to think and a lot to miss about these. So generally, I would say it is bad practice to allow your public domain client to get hold of your SQL connection. But still, don't let that hinder you to ask yourself (and understand) "why" and look at the specific situation. There might even be good cause/use for that. Especially since it is "less", which is also often better. It definately depends.
Just be careful and aware that (even if permissions are set correctly) a lot can be misused (and only little hindered), with a direct connection at your client. (Plus possible connection issues to be taken care of.)
As a sidenote: A lot of these considerations are relevant again with the use of technologies like GraphQL, which shares some similarities (however without connection issues and with a little bit more secure control).
I'm developing an none market appliction which run on 20-30 android devices (target specific to tablets with android honeycomb / ICS OS) maintaining connection over local WIFI network for a 1-2 hours period of time, and need to exchange data (simple objects representing commands) between them.
most of the time one specific tablet behave like a server which sending the commands, and the other devices like clients which receives the commands, but the "clients" also sending commands to the "server" sometimes.
as solution to this communication demand - I'm using for a while an open source
library which encapsulates TCP client/server protocol, called - Kryonet.
I found it very easy to use, and basically doing the job, although it sometimes "unstable" - a lot of disconnections accrues. I can't afford this disconnections, it's dammege the whole flow and use-case, causing the client's to lose commands.
I'm doing some recovery logic which re-connect the clients and send them what they have missed, but it's not good enough for the use-case.
recently I've heard about multicast broadcast protocol, and found even an open source library calls - JGroups which implement this protocol optimally, and expose easy and simple to use interface. still didn't tried it, but got advice from someone who knows, saying it should be better the the TCP client/server for my purpose.
what is the best approach I should use for implement the behavior I described ? (not necessarily one of the two I suggested)
TIA
Although JGroups has promise as a better solution for your situation, you may want to experiment a bit more to determine why the disconnects are happening. Since your clients and server are all tablets, there are a few other causes that are unrelated:
1) If the connections are not being maintained in a Service then they will be extremely unreliable by default. (See this question about singletons being destroyed in Android)
2) If the sockets have not been set to 'keepalive' then they will time out after an arbitrary number of seconds.
3) The devices you are using may shut down some persistent connections when they go to sleep.
4) The tablets may be exiting WiFi range, and switching over to a mobile network.
Try the network portion of your code on a number of desktop machines to determine if the problem is with Kryonet or your code, or if the problem is in running it on Android.