I'm porting a big chunk of native code with several interdependent libraries. Loading the code fails since Android linker only looks dependencies in /system/lib/ and not in the application install directory at /data/data/com.packagename.nnn/lib.
As a first workaround I loaded all .so's from Java with System.load() in correct dependency order. This solution isn't satisfactory for me mostly because the set of libraries changes from build to build due to plugin architecture. Also the UI shouldn't need to know that much about native libs.
I've found that Android does not support RPATH or setting LD_LIBRARY_PATH for an app. The only workaround I found is building the libraries with fully qualified SONAMEs.
However some of the dependencies come pre-built.
Is it possible compile fully qualified NEEDED tag into my own library even when the needed .so does not have fully qualified SONAME?
Or is it possible to modify existing .so and replace its SONAME or NEEDED with fully qualified one without recompiling?
Mozilla Fennec loads a bunch of shared objects from a custom cache directory under their /data/data/package/ directory, you might take a look at their source.
Related
I have a modularized application architecture with 3 module layers.
A core library module at the bottom and an application library module and my application on top of it. Each builds it's own native shared library. But not only the java code but also the C++ code in them also depends on each other.
So my "application.so" must link to "library.so" and "core.so" and on the other hand "library.so" must link "core.so" just like you expect from any layered architecture.
I can see in the file system that the shared so libraries all end up in the same build directory but i can't find a way to make them link each other. "find_library" is not going to work. And just naming them in "target_link_libraries" is not going to work either.
EDIT: With
buildFeatures {
prefabPublishing true
}
prefab {
infosqueezer {
headers "src/main/cpp/include"
}
}
i was able to generate an aar. But i'm not able to include it into the the other module. Both debug and release aar's are generated in the "library/build/output/aar/" directory. The modules are in the same project so i just need to reference by file somehow.
I'm using
implementation project(path: ':library')
but it does not pickup the so file. Also the AAR does not contain any "libraryConfig.cmake" or "library-config.cmake" that the find_package command would need to find the link library.
Also the whole prefab process seems to be terrible. I just want to split the in project source code to reduce compile time and dependencies. Prefab is adding so much and surely a good idea to distribute indpendent libraries but overkill for internal libraries.
All i need is just a way to reference the generated "library.so" file in another module.
I know it's Android but do they really make everything so terrible complicated?
There is no supported way to depend on native libraries from other modules in the same project with externalNativeBuild (at least in part because it will lead to worse build performance, see below).
As described your project should have a single gradle module for its native code if you want the best build performance (and want to be on a supported usage model).
If you really want these to be separate, the only way to do this that works with AGP is for them to be separate projects. I don't think you should do that.
Also the whole prefab process seems to be terrible.
This isn't what it's meant for. Screw drivers make terrible hammers.
I just want to split the in project source code to reduce compile time and dependencies.
Splitting gradle projects into more modules usually makes compile times worse, not better. I can't think of any reason that it would improve build performance for native projects, and it can easily make it much slower since invoking CMake is not cheap and splitting modules up multiplies that cost.
Issue
Built APK looks wrong and prevents my application from loading its native library because it cannot find a dependency when calling dlopen(): dlopen failed: library "libboost_filesystem.so.1.68.0" not found.
jniLibs content
My jniLibs directory, for the target platform, contains the following files:
libboost_filesystem.so.1.68.0 is the "real" shared object.
libboost_filesystem.so is a symlink to libboost_filesystem.so.1.68.0
APK Content
After building, the APK contains a libboost_filesystem.so which now is the binary object (not a symlink).
It seems like Android build system followed the symlink, grabbed content of the "pointed-to" file, but used the name of the symlink instead.
I have tried to remove the symlink from the jniLibs folder, but doing that it seems that the xxx.so.VERSION files are then ignored.
Question
How can I embed my "full name" shared object into the jniLibs without the Android build system messing with it ?
No you can't. You should avoid versioning the so file. See https://stackoverflow.com/a/45058227/192373 for instructions.
It's also quite natural that Android does not support this technique, because your native libraries belong to your APK and no version conflict can occur.
Consider linking boost filesystem statically to avoid extra lookup.
I'm making an Android app with help of the NDK. One of the shared libraries I'm using depends on ICU, which is another library that I'm trying to explicitly include too.
The problem is that my device (like apparently many others) has an old version of ICU pre-installed. This means that whenever I try to load my shared library, the system tries to load the system version instead of my own, more recent version.
Android seems to ignore any RPATHs, which would otherwise let libraries specific where to look for dependencies. As far as I understand RPATH is essentially hard-coded to /vendor/lib and /system/lib on Android.
I've seen a few workarounds for the issue, but none of them seem to work:
Explicitly load all the libraries (including dependencies) in dependency order.
Although the ICU libraries appear to load fine (via their absolute paths), I still get a cannot locate symbol error when attempting to load my shared library. I've quadruple-checked (via nm and readelf) that the missing symbol does exist in my (newly-compiled) ICU .so files.
Change the SONAME of the ICU dynamic libs to something project-specific and depend on them instead.
This would possibly work, but is kind of a last result because it would involve recompiling a lot of library code with non-trivial dependencies. It also feels more like a hack. A simple name change of the libraries doesn't work for the same reason as 1. – although the ICU symbols get loaded, they are not recognised when the dependent library tries to access them.
My question is to anyone who has tried something like this before or knows their way around linking a bit more than I do: how did you get it working? Is there a way to force Android to get its proverbial hotsauce together and actually load the correct libraries, or otherwise use the symbols which I've already successfully loaded?
I am using a number of static pre-built static libraries in my native android application and everything works fine. Now I want to switch one of my static libraries to be .so. I was successfully able to build .so library by replacing BUILD_STATIC_LIBRARY with BUILD_SHARED_LIBRARY in its android.mk and adding required dependencies.
I was also able to build my application by replacing corresponding PREBUILT_STATIC_LIBRARY with PREBUILT_SHARED_LIBRARY in its android.mk. The resulting application now fails to start. I cannot even get to point where debugger attaches to the application.
Besides that what I do not understand is how the build system knows that the function should be imported from the library. My so library should export one function, but I did not declare it as dllexport/import or something. Still there are no unresolved symbols in my application (when I remove my prebuilt library from the list, the unresolved symbol appears as expected).
The other question is that I see there are two .so files generated. One big file in obj/local/$(TARGET_ARCH_ABI) folder and another small one in libs/$(TARGET_ARCH_ABI). When declaring my prebuilt library I reference the second one in libs folder.
I did try to search stackoverflow for answers and found quite a few related posts:
loading library (.so file) in android
NDK - How to use a generated .so library in another project
How to use .so file in Android code to use the native methods
How to use libffmpeg.so in Android project?
but I do not see how these posts related to my problem since I can successfully build and even link my application.
You need to load the libraries in reverse dependency order in the java code. You previously probably have something like this:
System.loadLibrary("mylib");
Now if your prebuilt library (that was previously a static library, now a shared library) is named dependencylib, you need to change the code for loading the libraries into this:
System.loadLibrary("dependencylib");
System.loadLibrary("mylib");
As for your question how the linker can figure it out; when linking libmylib.so, it looks for all undefined symbols in all the other libraries you specified (i.e. in libdependencylib.so, and in libc.so and other system libraries). As long as all undefined symbols are found somewhere, the linker is ok. Then at runtime, when libmylib.so is loaded, it does the same routine again; all undefined symbols are looked up in the list of symbols loaded in the current process. On linux, you normally don't need to manually mark symbols as dllexport as you do on windows - all non-static symbols are exported by default.
There may be two reasons why the app fails to start after the change of STATIC -> SHARED.
The prebuilt library is not installed. With your device connected, run adb ls -l /data/your.package.name/lib/. Do you see the library there?
The prebuilt library is not loaded. In your main Java class, try
static {
System.loadLibrary("prebuiltname");
System.loadLibrary("yourlib");
}
This is a static constructor, the safest place to load JNI library.
If you are on linux you will see exported symbols using nm -D. example nm -D libzip.so:
...
0000000000009dc0 T zip_unchange
0000000000009dd0 T zip_unchange_all
0000000000009e30 T zip_unchange_archive
0000000000009e60 T _zip_unchange_data
If you want to control visibility of your functions use __attribute__ ((visibility ("default"))) and command line -fvisibility=hidden. More information here.
Now I want to switch one of my static libraries to be .so. I was successfully able to build .so library by replacing BUILD_STATIC_LIBRARY with BUILD_SHARED_LIBRARY in its android.mk and adding required dependencies.
I don't think you can do it if its a C++ library. From <doc>/CPLUSPLUS-SUPPORT.html:
Please keep in mind that the static library variant of a given C++
runtime SHALL ONLY BE LINKED INTO A SINGLE BINARY for optimal
conditions.
What this means is that if your project consists of a
single shared library, you can link against, e.g., stlport_static, and
everything will work correctly.
On the other hand, if you have two
shared libraries in your project (e.g. libfoo.so and libbar.so) which
both link against the same static runtime, each one of them will
include a copy of the runtime's code in its final binary image. This
is problematic because certain global variables used/provided
internally by the runtime are duplicated.
This is likely to result in code that doesn't work correctly, for example:
* memory allocated in one library, and freed in the other would leak
or even corrupt the heap.
* exceptions raised in libfoo.so cannot be caught in libbar.so (and may
simply crash the program).
* the buffering of cout not working properly
This problem also happens if you want to link an executable and a shared
library to the same static library.
In other words, if your project requires several shared library modules,
then use the shared library variant of your C++ runtime.
From above, it means everything needs to link against the same C++ standard runtime shared object.
I wish to back port the Android RTP APIs introduced in version 3.1(Honeycomb) to earlier versions. I downloaded the source of version 4.0 and found that it these APIs had both java and native code. In order to build the native code with the NDK, certain shared libraries are required.
According the Android.mk file, these are libnativehelper, libcutils, libutils, and libmedia. Though the source of all of these are present in the source code, building them was difficult. Each required many other shared libraries. For eg, libmedia requires these shared libraries: libui, libcutils, libutils, libbinder, libsonivox, libicuuc, libexpat, libcamera_client, libstagefright_foundation, libgui and libdl.
So my question is, is there some way of obtaining the original 4 shared libs? Does it involve building the entire source?
Say I need to build a piece of native code which is going to use standard Android shared libraries such as libutils, libcutlis, libmedia. I would perform following steps:
Install AOSP repository with target version.
Add my source code to appropriate directories under ./frameworks/base. In your case it might be easier to create a separate folder and put proper Android.mk of course.
You might get compile errors if required functions from those standard shared libraries are not present in the previous version.
When you build the code as part of AOSP it will build required libraries and link them for you automatically.
P.S. To accomplish that you're better to use a Linux-based build host.
using cygwin terminal, build native part i.e. jni folder. To build using cygwin, goto jni folder using cygdrive command. Then type ndk-build. After successful completion, shared libraries i.e. .so files will be created in libs folder.
I can understand your problem, you can pull the libraries from /system/lib of device or emulator. But you need a system permission. But you can do it by installing application.
Otherwise build your source code on linux platfor. Building process is very easy, just using 2 or 3 command. First time it is needed long time to build. After that you need very short time to build, it will build only according to the timestamp of modified code.
Please have a look here