I want to understand these IPC mechanism concepts in OS - Shared Memory, Message System, Sockets, RPC, RMI
How do different operating systems implement these. Specifically Android operating system?
IPC is inter-process communication mechanisms in OS is large discussion concept so, I think here we can't cover all this,
Some Low Level stuff:
The IPC mechanism discussed here is at the lowest level—all other inter-CPU IPC mechanisms use it as the base. For example, a TCP/IP connection through the ARM11 processor to another processor ends up going through this IPC mechanism. Diagnostic messages are another example of messages that rely on this low-level IPC.
The IPC mechanism is implemented with two sides—a "client side," which faces the kernel and provides a callback-based style of interface, and a "CPU side," which provides the interface to the other CPUs.
The CPU side is implemented as a shared-memory interface, with
interrupts and a "doorbell" mechanism. At the highest level, to send
messages from the ARM11 to another CPU, the message content is placed
in a buffer in shared memory and a hardware port is tickled to
indicate to the other CPU that data is available.
In the reverse direction, the data is placed into shared memory by
the other CPU and a hardware interrupt is triggered on the ARM11.
This hardware interrupt causes the ARM11 to examine the shared
memory's buffer, retrieve the message and route it to the client.
But to more specific to Android:
IPC in android, It describes the mechanism how different types of android components are communicated.
Android implements a few key tools used to communicate with or coordinate between programs securely. These mechanisms give Android applications the ability to run processes in the background, offer services consumed by other applications, safely share relational data, start other programs, and reuse components from other applications safely.
Much of the interprocess communication (IPC) that occurs on Android
is done through the passing around of a data structures called
Intents. These are collections of information that have a few
expected properties the system can use to help figure out where to
send an Intent if the developer wasn’t explicit. The Action property
expresses what the Intent is for (the Intent.ACTION_VIEW action
indicates that the data is to be displayed to the user, for example).
The data property is an optional URI and could point to a file,
contact, web page, phone number, and so on. Intents also potentially
have a collection of key/value pairs called extras, as well as flags,
components, and other more advanced features.
Each of these IPC mechanisms uses Intents in some capacity and is
probably somewhat familiar to most Android developers. However,
because using these safely is key to Android security,
1) Intents are messages which components can send and receive. It is a universal mechanism of passing data between processes. With help of the intents one can start services or activities, invoke broadcast receivers and so on.
2) Bundles are entities the data is passed through. It is similar to the serialization of an object, but much faster on android. Bundle can be got from intent via the getExtras() method.
3) Binders are the entity which allows activities and services obtain a reference to another services. It allows not simply send messages to services but directly invoke methods on them.
For more info look at:
Grasping Android's IPC mechanism
Android’s Securable IPC Mechanisms
Related
Binders are used for Inter(not intra) Process Communication/Remote Method Invocation so why/how the communication between Activity and Service is possible via binders where there are no different processes involvement.
When the very first component of an app is initialized a request to the kernal is triggeres (I think via ActivityManagerService(Native)-AMS > AMN) (running in system_process) for a new process fork if no corresponding process exists. Once the process is created then any other component (say a Service) belongs to the same process unless specified. Now this service wants to communicate to an Activity of the same process. Then why IBinder is one of the option?
Doesn't that violate the underlying principle on which Binders are created?
Interprocess communication is done by writing data to some source such as a socket or pipe, and another process reading it. There's no reason that the same process can't read and write though- it's less efficient than other means of passing data within a process, but it still works.
As for why would you use it in the same process- the alternative would be for Android to have two ways of communicating with services, one way for if they're the same process, and one way for it they aren't. That would complicate things for the framework for minimal gain. It would greatly complicate things for Services where the client may or may not be in the same process (say a service that is used by both the client app and other apps by the same company). So they just use the same method for both.
Binders are intended to be a construct of an IPC - this is true but, if used in the same process to communicate between an Activity and a Service, there is no performance cost, i.e, it is converted into a direct call instead of an RPC.
Don't assume that Binders will only be used for IPC but it is a general purpose communication mechanism in Android.
You won't find this documented normally anywhere. I found it in one of the discussion between Commonsware and Dianne Hackborne(designer of the binder concept in the Android team).
What is the advantage of using Binder for IPC over (Semaphores , Message Queue, PIPES) in Android stack?
Old question (and likely unmonitored by the poster), but worth answering:
A) All filesystem-based or filesystem-representable IPC mechanisms (notably pipes), can't be used because of a lack of a world-writable directory, where all processes can mkfifo/create the filesystem/socket representation of their IPC port (/dev/socket notwithstanding, which is used for system processes, e.g. rile, zygote, and their ilk).
B) None of the suggested mechanisms have the capability of "service location" which is required for Android. In UNIX, there's an RPC portmapper, and Android needs similar functionality. Enter: The ServiceManager, which can use binder to register as a context manager, to register/lookup service handles on the fly
C) There is an extensive need for serialization - be it intents, or other messages. Binder provides the parcel abstraction, which can be used for data marshaling by the Parcel.java.
D) SysV has other issues than Mr. Lambada's answer which are more paramount, notably race conditions, and lack of authorization.
E) Message queues and pipes can't pass descriptors. UNIX Domain sockets may, but can't be used because of (A) (again, unless you're root/system, like zygote, rild, installd..)
F) Binder is really lightweight, and has built-in authorization mechanisms. It also has nifty features like waking up the recipient process, as well as memory sharing, which the other mechanisms simply don't have. (and remember, no mmap(2), because of the file problem in (A) for named mappings).
and - let's not forget
G) Binder was started at Palm (ah, nostalgia) (q.v. OpenBinder). Ex-palmers got to Android, and brought their code in with them.
From the ndk's docs/system/libc/SYSV-IPC.html file:
Android does not support System V IPCs, i.e. the facilities provided by the following standard Posix headers:
<sys/sem.h> /* SysV semaphores */
<sys/shm.h> /* SysV shared memory segments */
<sys/msg.h> /* SysV message queues */
<sys/ipc.h> /* General IPC definitions */
The reason for this is due to the fact that, by design, they lead to global kernel resource leakage.
For example, there is no way to automatically release a SysV semaphore allocated in the kernel when:
a buggy or malicious process exits
a non-buggy and non-malicious process crashes or is explicitly killed.
Killing processes automatically to make room for new ones is an important part of Android's application lifecycle implementation. This means
that, even assuming only non-buggy and non-malicious code, it is very likely that over time, the kernel global tables used to implement SysV IPCs will fill
up.
At that point, strange failures are likely to occur and prevent programs that use them to run properly until the next reboot of the system.
Binders are used to to communicate over process boundaries since different processes don't share a common VM context => no more direct access to each others Objects (memory). Both parties within the same process (usually things that are within the same app) means (imho) that you should not use Binders since they slow down / complexify things unnecessary.
Binders are usually not used directly but rather via the "Service" or the "Messenger" classes. While communication with a Service is done via a full api of functions, the communication with a Messenger has to use "Message"s. Messengers a lot simpler to implement.
Apart from using Binders you can use anything that is available from any VM instance like "LocalSocket"s, Files, ContentProviders, Intents, ...
Binders are not ideal for transferring large data streams (like audio/video) since every object has to be converted to (and back from) a Parcel. All the conversion takes time. Much better in that case would be a LocalSocket for example.
Binders are used to enable remote procedure calls. You could implement RPC using the synchronization tools you mention but you would also need to write a lot of code to make it come together... with a Binder (normally only used within an Android Service) you have much less code to write; barely more than your actual remote functions.
Will any one please tell me what are all the IPC mechanisms that are present in Android.
To my knowledge are:
Intents
Binders
IPC is inter-process communication. It describes the mechanisms used by different types of android components to communicate with one another.
1) Intents are messages which components can send and receive. It is a universal mechanism of passing data between processes. With help of the intents one can start services or activities, invoke broadcast receivers and so on.
2) Bundles are entities of data that is passed through. It is similar to the serialization of an object, but much faster on android. Bundle can be read from intent via the getExtras() method.
3) Binders are the entities which allow activities and services to obtain a reference to another service. It allows not simply sending messages to services but directly invoking methods on them.
There are three types of IPC mechanism in Android:
Intents (along with Bundles)
Binders
ASHMEM (Anonymous Shared Memory) - The main difference between Linux shared memory and this shared memory is, in Linux other processes can't free the shared memory but here if other processes require memory this memory can be freed by Android OS.
All the answers are good and concise in this post. But I would like to light upon which IPC mechanism should we use. First of all IPC means Inter Process communication where two applications or processes will communicate with each other by passing some data between them. Since android is meant for embedded and small devices, we should not use serialization for IPC, rather we can use BINDERs which internally uses parcels. Parcel is a sort of light weight serialization by using shared memory concept.
There are many differences between Binder IPC and Serialization IPC:
1. Serialization is very heavy to use in embedded devices, communication will be very slow.
2. Binders uses Parcels to make IPC very fast.
3. Binders internally uses Shared memory concept which uses less memory while sharing data between two processes.
Bottom Line: Binders uses less memory, and quite fast as it uses parcels. Serialization is very heavy, takes time to send and receive data, and also it takes more memory compared to binders.
Note: To pass data between activities, services, and receivers use only Bundles. Don't go for either serialization or binders. Binders are specifically used only for binder services where 2 processes will communicate.
Hope this Helps :)
As written on Android Developers page, IPC mechanisms in Android include:
Intents (with Bundles included)
Binders or Messengers with a Service
BroadcastReceivers
There are three types of the IPC mechanisms:
handler
implementing binder
AIDL
The tree specific inter-process communications in Android are:
AIDL which is a two way with concurrent operation.
Messanger aa a two way but not concurrent
Broadcast as a one way
Also, you can use sockets but it's not recommended.
Another solution that worked for me was using the Internal files:
https://developer.android.com/training/data-storage#filesInternal
Write from one process, close file, read from another.
My team is working on an Android project which consists of several Android applications which exchange data (on the same phone). The idea is to have several applications which are collecting some data and send this data to the main application. The main challenge here is to do the exchange as cheap as possible in terms of CPU load & battery usage.
As far as I know, there are two ways to achieve inter-process communications:
Intents & activities - one activity catches the intents of another
Remote methods (through AIDL)
I wonder which of these is more efficient in the following scenarios:
Very frequent messages/method calls with very little data sent/traffic (e.g. just passing a bunch of primitives)
Less frequent messages/method calls with large traffic chunks (e.g. collect data and periodically send a few KB/MB of data)
Very frequent messages/method calls with large data chunks exchanged
I would appreciate any help, either in terms of comparison or a reference/link to a benchmark.
I think for 1) you'd be best with a remote service and for 2) and 3) you'd be better off writing to files or a database. Intents are more for infrequent interprocess communication and starting apps and services.
You could also try to use native code to create a shared memory as an alternative option. Check out this link for details:
http://www.androidenea.com/2010/03/share-memory-using-ashmem-and-binder-in.html
I suggest you use the Unix domain sockets mechanism to address scenario 3). The high frequency will make the use of files/databases complicated, and according to this answer, using Android's IPC is not recommended performance wise, since every object has to be converted to (and back from) a Parcel which takes time.. You can also use Unix pipes but it has some restrictions:
How to create named pipe (mkfifo) in Android?
https://groups.google.com/forum/#!topic/android-ndk/lD-V7Nxe5y4
How to use unix pipes in Android
I've been working with Android for well over a year now, but I still have trouble determining when different types of messaging/communication between processes/threads should be used. I'm mainly talking about broadcasting Intents, using AIDL for services, using Handlers to send messages and socket communication.
Many of these tools can be used to accomplish similar tasks, but which is better suited to particular situations?
This is a pretty open ended question, but let me take a shot at describing how I see the intra/inter application communication working best.
One of the key aspects of Android messaging is the concept of all application components being loosely bound. Because all applications run in a separate process, and one 'app' may actually consist of several applications (responsible for providing different Activities or Services), the messaging techniques are all based around the idea of marshaling messages across process boundaries.
Intents
The preferred technique for messaging, always try to use an Intent whenever possible. It is the most 'native' way to transfer messages within Android.
Advantages
Using Intents for messaging maintains the loose binding of application components, letting you transfer messages seamlessly between several applications. Intents are used heavily within the core system to start Activities and Services, and to broadcast and receive system events.
Using extras Bundles you can include key/value pairs of primitives as payload data within Intents to easily pass information from one application component to another - even if those components are running in different processes.
Disadvantages
Because Intents are designed to go between processes, the extras payload only supports primitive types. If you need to send an object using an Intent you'll need to deconstruct it into primitives at one end and reconstruct it at the other.
Application Class
If you only want to communicate within a single application running in a single process this is a handy solution.
Advantages
By extending the Application class (and implementing it as a Singleton) you get an object that will exist whenever any of your application components exist, providing a centralized place to store and transfer complex object data between application components.
Disadvantages
This technique limits your messaging to components within a single application.
Service Binding, IPC, and AIDL
Binding to a service lets you access its methods and exchange objects with it. AIDL is a way of defining how to serialize an object into OS primitives so that it can be marshalled across process boundaries if the Service you're binding to is running in a separate application.
Advantages
When you bind to a Service you have access to it as though it was an object within the calling class. That means you can execute methods on the Service and exchange rich objects with it.
Note that if you're binding to a Service in a different application process you'll need to create the AIDL definitions that tell Android how to seralize / deserialize any objects you want to pass between applications.
Disadvantages
Creating the AIDL classes for IPC is a bit of extra work, and binding creates additional dependencies between Services and Activities which can make it harder for the kernel to clean up resources when other applications are being starved.
Marshelling messages across process boundaries is expensive though. So if you're not executing methods on a Service, using binding and IPC is probably overkill - see if you can achieve the same thing using Intents.
Sockets
If you're resorting to sockets to communicate within or between applications running on a single device, it's either because there's no other way or you've missed a trick somewhere. If your messages are leaving the device then sockets are a good, fast, alternative. If you're staying on the device chances are Intents or IPC is going to be a better option.
It all depends on the use case and kind of your application. If the application is running all the time better to go with AIDL approach as it is most secure way to communicate. If application need not run all the time, then you can go with either broadcast intent or pending intent approach to communicate between applications.
My 2 cents
I have not used local sockets.
Seems like overkill since you have to
generate and parse the data.
Intents are for either things that
other apps might want to do (launch
me in a compose window or to pick
something out)
AIDL/Parcels/Handlers for having a
GUI talk to a headless process that
is running constantly. Depending on
the app a lot of the actual data
transfer might happen using a content
provider but I tend to have some data
that needs to be transferred outside
of that channel.
This is a good article I found helpful in trying to find a substitute for Cocoa's NSUserDefaults class:
http://developer.android.com/guide/appendix/faq/framework.html