Recently I upgraded the NDK and now my app crashes with missing symbol mkfifo:
E/dalvikvm(2031): dlopen("/data/app-lib/...mylib.so") failed: Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "mkfifo" referenced by "mylib.so"...
The older platforms mkfifo was defined inline in sys/stat.h
static __inline__ int mkfifo(const char *__p, mode_t __m) {
return mknod(__p, (__m & ~S_IFMT) | S_IFIFO, (dev_t)0);
}
But in platform version 21 it was changed to just an extern decleration:
extern int mkfifo(const char*, mode_t);
So that explains the missing symbol exception... my question is how do I fix it?
This happens if you build against the android-21 platform headers. Set APP_PLATFORM in jni/Application.mk to an older version, to build using the old headers, to make sure you only link to functions available earlier.
(Before android-21, the C library features and headers didn't really change significantly, so for the normal C library functions, it doesn't matter if you build targeting android-3 or android-20.)
This has been reported and is intentional behavior, see e.g. https://code.google.com/p/android/issues/detail?id=73725.
If you don't need to use new features from android-21, just build using older headers. (It doesn't matter that you're targeting an older platform version if you want to try to build for e.g. arm64-v8a or x86_64 that didn't exist before; ndk-build would build the 32 bit parts using the older target, and the 64 bit ones using the oldest target that supports them.)
In case you want to try to use new features from the android-21 platform conditionally if running on such a platform, you probably need to use dlopen and dlsym to load it conditionally anyway, so then you need to duplicate the other definitions from the new headers as well, to allow you to build using the older platform headers.
I've tried mstorsjo's fix and it seems to work, however I was a bit concerned that from the link he posted it seems that Google doesn't really think this is a good idea. As a result I did a bit more digging and it seems like the 'proper' fix is to ship multiple APKs, with (at least) one targeting android-20 and below, and another targeting android-21 and above.
The problem arises from a change in the NDK to force the use of the 'fPIE' option when performing an NDK build. From the NDK 10d release notes:
Introduced the requirement, starting from API level 21, to use -fPIE -pie when building. In API levels 16 and higher, ndk-build uses PIE when building. This change has a number of implications, which are discussed in Developer Preview Issue 888. These implications do not apply to shared libraries.
If you look at the Developer Preview Issue 888, it states the following:
Bionic commit 76e289c026f and its predecessor, 2aebf5429bb, require all dynamically linked native executables to be built with -fPIE -pie. This has a number of possibly-unintended side effects, including:
Old/unmaintained apps that worked correctly under KitKat may now fail to run under "L". This could affect even simple apps that do not use the network, handle untrusted data, or target the new "L" SDK API.
Actively maintained apps that wish to target "L" must ship PIE executables or static executables. If an app wants to target ICS or below, it must also ship non-PIE executables or static executables. Even a simple "Hello world" program built with -fPIE -pie will segfault on ICS and GB.
Obviously you may prefer to go with the previous solution but just thought it was worth noting.
Related
I'm compiling a 3rd-party Java library for Android that uses JNI. I read the relevant pages on adding C++ support on developer.android but I'm still confused about a couple of issues regarding C++ STL runtime that I was hoping I could clear up here:
1- My library has no control over the app it will be embedded in, so I don't know if there will be other libraries that might use a static/shared STLs. If I use a static C++ runtime with ANDROID_STL=c++_static, is it safe, or should I have to worry about another library that could be using something like gnustl_static which might conflict with mine?
2- If I use a shared C++ runtime with ANDROID_STL=c++_shared, is it a guarantee that a specific element in the STL will use the libc++ runtime or could it be possible to use gnustl if it doesn't exist? For example, If I was using std::string with a shared c++ runtime (c++_shared) in an app that has another library of gnustl_static, will my std::string implementation be taken from libc++ or gnustl?
Ideally, I'd like to have a very stripped down version of a static c++ runtime with (c++_static) that only includes std::vector, std::string and std::map. I was actually planning to use something like -ffunction-sections as described here and #768.
Please advise and thank you.
Environment Details
Pkg.Desc = Android NDK
Pkg.Revision = r15c
Android Studio = 3.1.2
system: cmake Host OS: Arch Linux ($ uname -r % 4.18.5-arch1-1-ARCH)
Compiler: Clang++
STL: c++_static/c++_shared
Your concern is a very real one. But if handled properly, you can find a robust way out.
The warnings about using single C++ runtime across all libraries in the app (and the whole idea to define C++ support in NDK as APP_STL vs. most other flags such as LOCAL_CFLAGS or LOCAL_SHARED_LIBRARIES, are relevant for the native libraries that are connected. JNI libraries that never communicate directly (except through their corresponding Java layers) can use different C++ runtimes. Another point is that normal build will only package one C++ runtime shared lib into the APK. Note that versioning is also a potential hazard: if a developer who adds your library uses a different NDK release, there might be collisions or unexpected side effects when his version of STL runtime works with your code.
Therefore, to achieve maximum flexibility, your library should use a static C++ runtime. This may effect the size of the binary, but if, as you say, you use only a limited subset of STL, this extra will be rather small.
The bottom line, you will have minimum to worry about if build your shared library with libc++_static.
There's many shared libs(*.so) in my android project. Some of them were built by others, I don't have the source code.
I'm using NDK r10e, and I want to upgrade the NDK version to r13b.
If I don't change the makefile, just build part of the shared libs with NDK-r13b, others built with NDK-r10e were not changed. Is there any problem with capability of the android program?
NDK-r10e use clang-3.5
NDK-r13b use clang-3.8
following configurations are the same:
APP_ABI := armeabi-v7a
APP_PLATFORM := android-19
APP_STL := gnustl_shared
It's generally a good idea, but isn't always required. We try to preserve compatibility across NDK versions since, yes, moving across a compatibility boundary is difficult when you don't have the source to your dependencies, but sometimes bugs can't be fixed without breaking compatibility.
It's generally a good idea because doing so will have the least surprises. We only ever test a version of the NDK against itself. It will also save you a lot of time trying to figure what went wrong when there are compatibility breaks.
A couple examples (by no means a complete list) of things that can go wrong:
The public API surface of the NDK has changed over time. The APIs still exist on the device so that legacy apps continue working, but sometimes APIs are removed from the NDK so that new apps don't use them (there are a handful of reasons this might happen). In these cases, an old library that uses a removed symbol won't be able to be linked against a new library/executable using an NDK that doesn't have those APIs (this actually depends on the behavior of the linker; it's always true for static libraries, I think it's only arm64 for shared libraries).
For upgrading from r10 in particular, there was an ABI break in libc++ (required to avoid nasty interactions with the system) that means that libraries built with r10 and libraries built with r11+ are not compatible. You're using gnustl, so that problem won't affect you, but it's worth mentioning for others that land here.
Android already has the PCRE shared library ("/system/lib/libpcre.so"), but it's compiled without Unicode support, so I've built my own version of PCRE for Android. My native code is linked to the PCRE shared library. However, when I run my app, it uses the system's version of PCRE, rather than the one I've built, even though my APK does include my version of PCRE. How do I make it use my version of PCRE rather than the system's? Generally in Linux I use "LD_LIBRARY_PATH" or "RPATH", but on Android I don't know how to use them, if that's at all possible.
You can load("your-path-to-lib") before you load the main library. With this API you can load a native library at arbitrary location. But the easiest way is to rely on the default behavior of build and installer, which will pack the native libraries (named libsomething.so) that it finds in libs/<ABI> folders, into the APK file, and unpack the ABI variant that matches the target into
getContext().getApplicationInfo().nativeLibraryDir
(this was added in API level 9)
If the library is pre-loaded, you cannot have your library side-by-side with the system one, due to a bug that invloved older versions of Android. Still, you may succeed to unload it manually, using dlclose():
handle = dlopen("<libname>", RTLD_NOLOAD);
dlclose(handle);
dlclose(handle); // twice, because dlopen() increments ref count
You will probably do these manipulations in a separate small dlclose_helper.so. Load this helper load before you load the main library, which needs the private version of <libname>.
A fix was introduced for API level 23 that lets us finally load both dir1/libx.so and dir2/libx.so (see the official doc).
Note that for API level 24, new restrictions have also been introduced that limits access to system libraries (you can only load white-listed ones).
I believe the only way is to rename the library (libpcre_myapp, for example).
Note that renaming just the file probably is not sufficient, but changing the SO_NAME ELF property.
I'm working on an Android app that has heavy use of the NDK. On the Java side of things, we target SDK 19 with a min SDK of 16. Is there a way to do something similar on the NDK side?
Right now our Application.mk file has APP_PLATFORM := android-16. Is there a way to target platform 19 but still have compatibility back to 16 on the NDK side?
It's doable, but it is not too easy.
For the java code, as you know, you can set any higher target SDK version and use such features, as long as you make sure that those codepaths only are executed on the newer devices - simple.
For the native code, you can in principle set a higher APP_PLATFORM than your baseline, and try to do the same, but there's a few details you need to keep track of:
You can't unconditionally link to functions from the newer platform, you need to load them dynamically. That is, instead of calling functions directly and adding the libraries to LOCAL_LDLIBS, you need to load the functions via dlopen and dlsym instead, to make sure that the binary is loadable on the older versions. (Or alternatively, you can build separate shared libraries, where one shared library can be loaded on all platform, while another one only can be loaded on newer platforms.)
Some bionic libc functions have changed (mainly in android-21, but some minor ones also changed before that) - functions that did exist before but have changed symbol name. One of the more common functions that has changed is rand - prior to android-21, rand was an inline function that actually called lrand48(), so your binary ended up depending on lrand48 which existed in the older android versions' libc.so, while they didn't have any rand there. In android-21, a lot of such functions have been added, and the inline functions removed, so if you build with APP_PLATFORM := android-21, your binary will end up depending on the function rand which didn't exist before. See https://stackoverflow.com/a/27093163/3115956 and https://stackoverflow.com/a/27338365/3115956 for more details about this.
Keep in mind that you don't need to set APP_PLATFORM to the same as the target SDK on the java side, you only (may) need to set it if you want to selectively use newer features on newer firmware versions.
Due to the second issue you might not want to set a higher APP_PLATFORM at all. If you use dlopen (so you don't actually need the .so files to link against), you can manage pretty easily by copying those new headers from the newer platform version into your own project, and building with the older APP_PLATFORM.
Take a look at https://developer.android.com/ndk/guides/stable_apis.html
For example, if you don't use OpenGL ES 3.0 and OpenGL ES 3.1 APIs and don't link libGLESv3, then your app has compatibility to API level 14.
Should/does NDK9 work with android API19? (though it was released with API18).
Full story:
I was building an Android App using kivy, python-for-android and buildozer.
compiling with MDK9 (ie 9d) and api19 result in error:
E/AndroidRuntime( 1773): java.lang.UnsatisfiedLinkError: Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "wait4" referenced by "libpython2.7.so"...
compiling with NDK9 (ie 9d) and API18 works. :)
Just in case anyone stumbles across this. I had the same issue, but building python from scratch using the platform tools, and with NDK r10 and r10b.
It's because google removed an inline wait4() declaration in the NDK platform android-19. The original problem was that wait4() was exposed in the headers but not declared anywhere, so if you tried to use wait4 on older NDKs, you'd probably crash (like you do now).
So they went in and added it to libc.so in API 18 I believe, but libc.so is shipped with the O/S, so devices with O/S older than 18 would not have wait4, so they patched it by adding an inlined wait4() method in NDK platforms up through android-18, then took it out in android-19 I'm not fully sure why yet, seems like it could have made more sense to leave it alone at that point, especially since it's considered an obsolete function. I was told I should not build against android-19 if I wanted the app to run on devices older than API 18, but others say to always use the latest NDK to match your build target. Here is a link to the problem.
https://code.google.com/p/android/issues/detail?id=19854
In my case, I went into the cpython Modules/posixmodule.c file, and added:
#if defined(__ANDROID__)
#undef HAVE_WAIT4
#endif
And in my case that's fine, because none of my python modules use wait4. Indeed, Linux deems the wait4 command obsolete (http://linux.die.net/man/2/wait4). You should instead use the waitpid.
So, even if you download some 3rd party python module that does use wait4, you have 2 options. 1) change that module to use waitpid, or 2) update the Modules/posixmodule.c file inside the "ifdef HAVE_WAIT4" section, and replace the wait4 call with waitpid. The downside is you lose the returned resource usage information, which waitpid doesn't provide, so then if your module needs that, you'd have to add something to retrieve resource usage for that pid separately.
Alternatively, you could remove "wait4" from your configure script if you never plan to use it, and any modules you add that needed it would break, at which point the new engineer working on the problem would have to rediscover all of this over again.
Android SDK are released more often that NDK. It happened more than once that if you use a too recent SDK, the NDK will not have the .h for it. Now i'm not sure this would be related to your issue at all.