I'm about to port a large C++ project (some sort of Library Project, it contains absolutely no GUI) to Android. It's actually a Visual C++ project, but it will be ported to Linux as intermediate step. I know that Android is not a "full" Linux and does not claim to provide all POSIX functions, but I also know there are a lot of "POSIXish functions" on Android by using the NDK.
Now my actual question is:
Which are the biggest/most important functions that are NOT available on Android compared with the full POSIX set? So that I can keep that in mind when doing the porting from Visual C++ to Linux GCC.
I tried to find something on Google, but found nothing really helpful, just here and there some stuff that mentioned that there are some POSIX functions on Android...
Bionic a recode by Google. It is small but optimized for Android.
The only big thing I know of that it lacks is indeed the pthread_cancel() function.
My experience is that if you port it successfully to GNU/Linux, without pthread_cancel() calls, then you should be mostly ok.
BTW, what kind of library are you trying to build? What does it uses? Network, threads...
PS: Even Linux is not fully POSIX.
Bionic Wikipedia page
https://en.wikipedia.org/wiki/Bionic_(software)#Differences_from_POSIX
Also has some interesting info:
Although bionic aims to implement all of C11 and POSIX, there are still (as of Oreo) about 70 POSIX functions missing[8] from libc. There are also POSIX functions such as the endpwent/getpwent/setpwent family that are inapplicable to Android because it lacks a passwd database. As of Oreo, libm is complete.
Some functions deliberately do not conform to the POSIX or C standards for security reasons, such as printf which does not support the %n format string.[9]
Official Bionic in tree documentation quote
https://android.googlesource.com/platform/bionic/+/37ad9597839c70a7ec79578e5072df9c189fc830/docs/status.md
Run ./libc/tools/check-symbols-glibc.py in bionic/ for the current list of POSIX functions implemented by glibc but not by bionic. Currently (2017-10):
aio_cancel
aio_error
aio_fsync
aio_read
aio_return
aio_suspend
aio_write
lio_listio
pthread_cancel
pthread_mutex_consistent
pthread_mutex_getprioceiling
pthread_mutex_setprioceiling
pthread_mutexattr_getprioceiling
pthread_mutexattr_getprotocol
pthread_mutexattr_getrobust
pthread_mutexattr_setprioceiling
pthread_mutexattr_setprotocol
pthread_mutexattr_setrobust
pthread_setcancelstate
pthread_setcanceltype
pthread_testcancel
wordexp
wordfree
libm
Current libm symbols: https://android.googlesource.com/platform/bionic/+/master/libm/libm.map.txt
0 remaining missing POSIX libm functions.
The most obvious feature missing is pthread_cancel().
This blog has some additional details: http://codingrelic.geekhold.com/2008/11/six-million-dollar-libc.html
Good overview of Bionic: https://android-platform.googlegroups.com/attach/0f8eba5ecb95c6f4/OVERVIEW.TXT?gda=HWJaO0UAAAB1RXoVyyH5sRXFfLYnAq48KOFqr-45JqvtfiaR6gxIj4Qe8cBqW3BaWdSUPWi_jHqO3f1cykW9hbJ1ju6H3kglGu1iLHeqhw4ZZRj3RjJ_-A&view=1&part=4
shared memory is also something you might find differently implemented in android. was hit hard while trying to work with shm_open and shm_unlink on android kernel. Android implements asynchronous shared memory (ashmem).
Related
i used the code Webview.loadurl("www.google.com")
and I know that
using the dynamic analysis tool, in the libc.so module, the connection() function is used.
*connection format is connection(sockfd, addr, addrlen)
parameter:addr include "www.google.com" URL
I would like to know how this module and WebKit work.
enter image description here
Libc is the standard system library, including standard C library, POSIX, and other Linux and possibly android extensions).
So nearly all libraries need libc to work: they use it to call the system. Very seldom libraries call directly the kernel (because libc provide most of the call, but few special use calls).
Libc is very complex. You may read C books and POSIX. For implementation, you can Google: you will find a lot. Some complete, some made for tiny systems. Often they are open source, so you can check the implementation. Possibly try the small libraries (for embedded systems): the code is much more readable. Avoid glibc (now near the reference): it should be compiled for too many systems, so the code is made compatible with old and broken compilers, at cost of readability (but use glibc documentation).
I have a pretty large project written in C++ for Windows, with some MSVC-specific fancy things, like __declspec(property), usage of SEH, extra template and macro flexibility, intrinsics and etc.
In a nutshell it consists of various applications and a shared library (lib), which is a pretty large "algorithmic" code. It's written entirely in plain C++ (MSVC's version of it, as I said), not dependent on any 3rd-party, all the code is "hand-made", no stdlib, STL, Boost or etc.
Now this code needs to be ported to mobile platforms, namely Android and iOS.
The code should be platform-independent. No explicit dependence on OS, besides the very basic things, like heap memory allocation. It does depend on some Windows-specific things: SEH, TLS, but those are the things I can sacrifice, if they can't be replaced.
And I'm thinking about how to deal with it. Cleaning the whole code from MSVC-specific stuff is possible, but not convenient. I'd prefer to keep them, and definitely I don't want to keep several codebases for different platforms.
So, what are the options at my disposal?
As I understood, there are C++ compilers for Android (part of NDK), but they are probably standard C++ compliant. Anyway, iOS development is based on Objective-C, which is a superset of plain C.
This led me to an idea to "compile" the existing MSVC-specific C++ code into a plain C. There is an option in MSVC compiler to generate "listing" files, containing the assembler code. I guess if there's an option to create appropriate listings containing C-code. Or alternatively MSVC-compliant 3rd party C++ -> C converters.
Any thoughts?
So, what are the options at my disposal?
Write portable C or C++ code.
I've got C and C++ libraries that run on Android, iOS, Windows Phone, Windows, BSD, OS X and Linux. The library is written once and it runs everywhere.
While the "core library" is portable C/C++, the next layer up is not. That's where the libraries integrate with the platform. For example, the iOS test suite driver has a Cocoa/CocoaTouch UI on Apple platforms, and an MFC test suite drive on Windows and Windows Mobile. And the test suite drivers on Linux are command line because I don't waste time with GTK or Qt.
The routines to seed the random number generators are platform specific. I consider them a core function, so its in the core library and guarded by platform specific #defines.
Don't make the mistake of re-implementing your core library on every platform it runs on. That means you will need 4x to 8x the development cycles to duplicate the code and behavior. And there will always be small, hidden behavioral bugs that you waste countless hours tracking down.
And I'm thinking about how to deal with it. Cleaning the whole code from MSVC-specific stuff is possible, but not convenient.
Yes, do this. Pay the tax once and enjoy the benefits for the remainder of the code's life.
This led me to an idea to "compile" the existing MSVC-specific C++ code into a plain C.
No, I would not do this. Remove the platform specific stuff from the core library. Make the core library portable.
MSVC++ to C compilers come in the "if you have to ask, you can't afford it" category. Just too small a market.
A more realistic chance would be to wait what Microsoft is doing. They're seriously looking into targeting additional mobile platforms with MSVC 2015.
TLS is probably the easiest, as that is standard C++ (thread_local). SEH should be mapped to regular C++ exception handling, which means you need to trap pointer bugs before they happen. MSVC isn't exactly known for extra template flexibility, it's in fact rather inflexible. x86 intrinsics obviously are right out on ARM.
I am embarking on some Android NATIVE coding (e.g. C++, not Java), and need to use the fairly undocumented sp<> ("Strong Pointer") refcount'd pointer class.
As far as I can tell, the Android sp<> template looks VERY much like the more familiar BOOST shared_ptr<> template. Standard refcount mechanism. They do NOT appear to be part-for-part compatible. For instance, Strong Pointers do NOT appear to be threadsafe. What other gotchas are there between the two?
A wider question would be: why is there no online reference for the NDK? Are they having enough diskspace problems on developer.android.com that they cannot fit it there? Grumble.
Android's sp<> is undocumented because it is part of the platform, and its implementation might change between platform revisions. You should not use it in NDK code, unless you copy all of the headers and corresponding source files into your own project.
It is intentionally not thread-safe for performance reason: actually doing thread-safe ref-counting requires adding memory barrier instructions which slow down the operation significantly on ARM (not so much on x86 and x86_64 though). Even Chrome uses two different classes to implement ref-counting for this reason (i.e. base::RefCounted and base::RefCountedThreadSafe).
As to other gotchas, I can't really tell, but I guess the implementation of weak pointers is also different from Boost. In all cases, if you don't understand what this code does, don't use it, it's not meant for general consumption.
Forenote: This is an extension of the thread started on /r/haskell
Lets start with the facts:
Android is one awesome Operating System
Haskell is the best programming language on the planet
Therefore, clearly, combining them would make Android development that much better. So essentially I would just like to know how I can write Haskell programs for the Android OS. My question is:
How can I get a Haskell program to execute/run on the Android OS?
How you do it is by first getting a Haskell compiler which can target C with the android NDK which comes with a GCC port for ARM architectures. JHC can trivially do this with a very small inf style file which describes the platform (word size, c-compiler, etc) I've done this with the Wii homebrew dev kit and it was quite easy. However jhc still has some stability issues with complex code such as using a monad transformer stack with IO but jhc has been improving a lot over the last 6 months. There is only one person working on JHC I just wished more people could help him.
The other option is to build an "unregistered" port of GHC targeting the ndk gcc, this is a lot more involved process because GHC is not a true cross-compiler at the moment and you need to understand the build system what parts you need to change. Another option is NHC which can cross-compile to C, like GHC you need to build nhc targeting a C compiler, NHC does not have many Haskell extensions like GHC.
Once you have Haskell compiler targeting NDK GCC, you will need to write bindings to either the android NDK JNI glue code framework (added since android 2.3) or you must write JNI glue code between Java-C-Haskell, the former option is the easier solution and if I remember correctly might actually be backwards compatible with previous versions of Android below 2.3.
Once you have this you must build Haskell code as shared library or static library which gets linked into the NDK java glue code (which itself is a shared library). As far as I'm aware you can not officially run native executables on android. You could probably do it with a rooted phone, thus I assume this means you can not distribute native executables on the app store even when the NDK gcc port can generate native executables just fine. This also probably kills the option for using LLVM unless you can get the NDK JNI working with LLVM.
The biggest hurdle isn't so much of getting a Haskell compiler for android (which is still a big hurdle) the biggest problem is that some one needs to write binding APIs for NDK libraries which is a huge task and the situation is worse if you need to write android UI code because there are no NDK APIs for this part of the android SDK. If you want to do android UI code in Haskell somebody will have to write Haskell bindings to Java through JNI/C. Unless there is a more automated process to writing binding libraries (I know there are some, they are just not automated enough for me) then chances of some one doing it are quite low.
L01man: Is there a tutorial about how to do this? For the
first part, I understand I have to download JHC. What do I have to
write in the inf file and how to use it?
Please note before I answer this question I haven't used jhc for quite sometime since I originally wrote this and newer versions have been released since so I do not know how stable jhc currently is when it comes to code generation of more complex Haskell programs. This is a warning to anyone before you consider making a large Haskell program with JHC, you should do some small tests before you go full on.
jhc does have a manual http://repetae.net/computer/jhc/manual.html and a section on setting-up cross-compilation and .ini file with options: http://repetae.net/computer/jhc/manual.html#crosscompilation.
L01man: The second part is an alternative to the first. I don't know how to do what you said in the
third.
Before you begin you should have some knowledge of C and be comfortable with using the Haskell foreign function interface (FFI) and tools such as hs2c. You should also be familiar with using the Android NDK and building .apk with shared libraries. You will need to know these to interface between C-Haskell, Java/C-Haskell and develop Haskell programs for Android that you can officially distribute/sell on the market store.
L01man: I understand that its goal is to create a binding for the
Android API. But... does the 4th part says we can't make .apk with
Haskell?
.apk is just an app package file format and is built with the tools that come with the Android SDK (not NDK), this has very little to do building the binaries itself. Android packages can contain native shared libraries, this what your Haskell program will be and the native shared/static libraries are generated via the Android NDK.
A language that has recently come to my attention is Eta.
Eta's compiler is a fork of GHC 7.10 which has a JVM backend. It is possible to use the generated JAR files to write Android apps and even use its Foreign Function Interface to call native Android Java libraries.
Brian McKenna has written a blog post about how to configure an Android Studio project to use an Eta library.
There is https://github.com/neurocyte/android-haskell-activity demonstrating Haskell code running.
I once came across the same Reddit thread, but it was old, and comments were closed. I sent a message to the OP, but am not sure whether it reached the recipient. My suggestion here (may work for older Androids where native activities were not possible).
I (developed in Haskell some time ago, but currently switched to Smalltalk) am currently developing a port of Squeak VM to Android. The way I am doing this is similar to what might be dealt with in a haskell-on-android project: a lump of C code which needs to be called from Java part of the application (basically all that can be done in Android is to handle various events; an application cannot poll for events itself and does not have any event loop). In my case the code is generated by the Squeak VM building tools, in the case of haskell on android this will be output from GHC of JHC or whatever front end used. This repo may be worth looking at:
http://gitorious.org/~golubovsky/cogvm/dmg-blessed/trees/master/platforms/android/project
Under "src" there is the Java code which provides for user events interception and sending them to the native code (see the CogView class). The C code of the VM itself is not completely there (see squeakvm.org, the Cog branch for that), but one may get the idea. One also might look under http://gitorious.org/~golubovsky/cogvm/dmg-blessed/trees/master/platforms/android/vm which is the C frontend to the interpreter (including user events handling, some timekeeping, etc.)
Hope this helps.
Dmitry
There is https://github.com/conscell/hugs-android a port of HUGS Haskell interpreter to Android.
I think the general answer should come from source->source transformations, since loading specially compiled shared objects seems to be a bit of a kludge (involving ghc->c and a c->java step in the answers above). This question thus falls under the heading of Haskell on the JVM, which has been tried (with one step as a Java intermediate representation) and discussed at length. You could use frege if the libraries you need compile there. The only remaining steps would be the beginnings of the Android framework API translated into IO() actions and maybe a wrapper for building the manifest xml and apk.
I want to port an aplication written in C++ to android. Converting the application from C++ to Java will take a lot of work that I would prefer to use on making the application better for that platform instead of fixing convertion bugs and solving refactoring problems.
The NDK seems a good route to take but actually I don't want to miss a platform(if it is a considerable % of the market) just because the NDK doesn't or won't support it.
Android claims to support MIPS, ARM, X86 and others ... but actually all the implementations I have seen are only on ARM (or arm compatible).
I checked that on this site:
http://www.pdadb.net/
Would it be a bad desision to use the NDK?
Are there any non ARM devices that run or will run Android?
Where can I find more information about this?
Thanks in advance!
At this point the problem is not that you would not lose market share due to CPU architecture, as there are very few non-ARM Android devices at the moment, the problem is that you may lose market share due to requiring users to run Android 2.3 or later, which you would have to use in order to create a fully native application with access to the window, sensor, and input subsystems.
Avoiding rewriting code is a good goal but you would likely have to rewrite portions of the code anyway due to Android's dissimilar Window and life-cycle APIs. Now you would have to rewrite some important parts of the code in C++ rather than in Java.
You could try a hybrid approach where you write most of the UI in Java, then make calls to your existing C++ code.
Are you making a game? Then you'll probably want to deal with these issues and press on with the NDK. If not, try implementing as much of the program as possible in Java and use the NDK for the complex, tested parts of your code that need to be fast.
The documentation gives the following:
The latest release of the NDK supports these ARM instruction sets:
ARMv5TE (including Thumb-1 instructions)
ARMv7-A (including Thumb-2 and VFPv3-D16 instructions, with optional support for NEON/VFPv3-D32 instructions)
Future releases of the NDK will also support:
x86 instructions (see CPU-ARCH-ABIS.HTML for more information)
Would it be a bad desision to use the NDK?
For algorithms, the NDK is fine. For games, the NDK is fine. For implementing an ordinary app, the NDK will not be terribly helpful.
Are there any non ARM devices that run or will run Android?
Google TV runs on x86 (Atom).
Use this improved NDK: http://developer.mips.com/android/download-android-ndk/