I downloaded an APK from Play Store that contains native code binaries. In the APK file there is an lib/x86 folder that supposedly contains a library file containing native procedures, normally a .so extension. Since the code is in x86, is it possible to write a Java program to invoke the library on the desktop? Even if you dont have the source code for that library. The NDK function just has to accept parameters and return a value. For example, can we write
class AppNativeLoader
{
public static native void generateRand(int seed);
static
{
System.loadLibrary( "AndroidNDKLib" );
}
}
public class WCallTest
{
public static void main( String[ ] args )
{
long seed = System.currentTimeMillis();
if(args.length > 0) {
seed = Long.valueOf(args[0]);
}
long rand = AppNativeLoader.generateRand(seed);
System.out.println(rand);
}
}
NOTE: This is just an example. The actual environment differs. Using JRE 7 on RHEL, I extracted the x86 .so and placed it in the same directory as the .class file. I still get an UnSatisfiedLinkerError. Anything amiss? Assuming there are no callbacks and the function doesn't utilize and Android APIs, is this possible?
EDIT: I opened the lib in IDA Pro and I saw the following dependencies
.plt:0000B100 ; Needed Library 'liblog.so'
.plt:0000B100 ; Needed Library 'libz.so'
.plt:0000B100 ; Needed Library 'libc.so'
.plt:0000B100 ; Needed Library 'libm.so'
.plt:0000B100 ; Needed Library 'libstdc++.so'
.plt:0000B100 ; Needed Library 'libdl.so'
These should be available in my desktop environment, no?
Not all Linux environments are identical (even crossing distribution boundaries is not guaranteed to work). NDK binaries are built against Bionic and a handful of other Android specific libraries, whereas your RedHat system uses glibc and a bunch of other things available from the RedHat repositories.
tl;dr you can't run Android binaries on desktop Linux.
You can try downloading the needed shared libraries from here (make sure to choose the correct API version, and an architecture matching the architecture of the NDK shared library, to find out which shared libraries you need you can simply use ldd).
Then, to easily access the methods exposed by the shared lib, you can decompile the java code of the app using jadx, and then write your own code around the JNI classes.
Then, to compile your java code, you can use any version of the JDK.
Then, to execute it, you'll have to use a version of JRE matching the architecture of the NDK shared library (in your case, you'll have to download the 32-bit JRE).
However, this is not guaranteed to work: I am currently getting segfaults in the NDK shared library I'm trying to use on my PC, and since most NDK binaries are stripped, debugging is going to be a nightmare.
Related
We are working on an Android and iOS (but we want to just get Android working for now) application that needs to rely on some native drivers (.so, .a and some .ini files). The company providing those drivers also provides a Xamarin project to showcase how the drivers are used. They seem to be storing those driver and other files as assets (.ini files under Assets/Files and .so files under Assets/lib/arm64-v8a and Assets/lib/armeabi-v7a respectively) and extracting the former using the following code:
void ExtractAssets()
{
var assets = ApplicationContext.Assets;
var paths = assets.List("Files");
foreach (var path in paths)
{
// Read the compressed file and extract
string readPath = Path.Combine("Files", path);
byte[] buffer = new byte[32000];
int bytesRead = 0;
using (var inStream = assets.Open(readPath))
using (var outStream = ApplicationContext.OpenFileOutput(path, Android.Content.FileCreationMode.Private))
{
do
{
bytesRead = inStream.Read(buffer, 0, buffer.Length);
if (bytesRead > 0)
{
outStream.Write(buffer, 0, bytesRead);
}
}
while (bytesRead > 0);
}
System.Diagnostics.Debug.WriteLine(string.Format("Extracted '{0}' to '{1}'", readPath, path));
}
}
Since Xamarin is becoming obsolete and MAUI is the next big thing, we decided to build our project on that. My colleague and I have spent the entire afternoon on a sample project and we still do not manage to simply load a libhello-jni.so file (we use this one) at runtime to call a stringFromJNI(). We tried:
Using an Android Native Binding project — we never really got what that's about and how mappings are supposed to work
Trying to include the libhello-jni.so file as an asset and load it using JavaNative.LoadLibrary("hello-jni"); and this had various unsuccessful results, it either did not find the library at all or it was complaining about the lib being built for x86 time processor instead of x64.
Can anyone please help us with some sample code allowing us to just simply bind a native .so file in a MAUI project, or at least provide us with a good guide? Thank you and sorry for the long post.
Not sure if this is exactly what you are going for or not, but we use bindings to a C library that we develop for various platforms, including Android. The way to correctly place native libraries into the final APK is via the EmbeddedNativeLibrary type in .csproj as demonstrated here. This will include your native library alongside the other native libraries that the toolchain is going to create (like the .NET runtime, garbage collection native lib, etc).
One thing changed in .NET 6, however. This affects us as an SDK distributor, because we need to include all necessary files inside of Nuget packages, so it may not affect you. In Xamarin, when you embed native libraries, the native libraries are literally stitched into your assembly and no further files are needed. In .NET 6, now they are all placed into a separate .aar file which also needs to be available when building the final application.
If you do everything correctly, you will see your native library inside of the APK in the lib/<arch> directory. Our native library is not JNI, and thus we use P/Invoke so we don't need a LoadLibrary call but I am reasonably certain that the call should succeed with the libraries in this directory.
I'm working on an Android project which uses a Java class that is a wrapper on a C++ library. The C++ library is a company internal library and we have access to its source code, but in the Android project it is only dynamically linked, so it is used only in the form of headers (.h) and shared objects (.so). Having access to the library source code, is it possible to specify to Android Studio the path to the source code so I can step inside the library using the debugger?
The debugger works, I can step inside the Java_clory_engine_sdk_CloryNative_nativeInit function, but I would also like to further debug the library corresponding to the Clory::Engine class which, as I mentioned, is an internal library we have source code access to.
For example, Clory::Engine::instance is part of the library and I would like to specify to Android Studio the location of the CloryEngine.cpp file so I can step inside Clory::Engine::instance with the debugger, thus debugging this static member function.
I am using Android Studio 3.1.4.
Is this possible?
EDIT:
The clory-sdk.gradle file specifies the CMakeLists.txt file which configures the C++ layer.
externalNativeBuild {
cmake {
path "CMakeLists.txt"
}
}
So I am using an internal application which uses the Clory SDK. Inside the app.gradle file I use:
dependencies {
...
compile project(':clory-sdk-core')
compile project(':clory-sdk')
...
}
so I don't think we're using the aars for the app.gradle project. The aars are shipped to the client, but we are using app.gradle project to test our little SDK functionalities before doing that. The JNI layer is inside clory-sdk-core project.
EDIT 2:
Here is the CMakeLists.txt which handles the JNI layer:
cmake_minimum_required(VERSION 3.4.1)
set(CMAKE_AUTOMOC ON)
set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_BUILD_TYPE Debug)
add_library(
clory-lib
SHARED
# JNI layer and other helper classes for transferring data from Java to Qt/C++
src/main/cpp/clory-lib.cpp
src/main/cpp/JObjectHandler.cpp
src/main/cpp/JObjectResolver.cpp
src/main/cpp/JObjectCreator.cpp
src/main/cpp/DataConverter.cpp
src/main/cpp/JObjectHelper.cpp
src/main/cpp/JEnvironmentManager.cpp
)
find_library(
log-lib
log
)
target_compile_options(clory-lib
PUBLIC
-std=c++11
)
# Hardcoded for now...will fix later...
set(_QT_ROOT_PATH /Users/jacob/Qt/5.8)
if(${ANDROID_ABI} MATCHES ^armeabi-v7.*$)
set(_QT_ARCH android_armv7)
elseif(${ANDROID_ABI} MATCHES ^x86$)
set(_QT_ARCH android_x86)
else()
message(FATAL_ERROR "Unsupported Android architecture!!!")
endif()
set(CMAKE_FIND_ROOT_PATH ${_QT_ROOT_PATH}/${_QT_ARCH})
find_package(Qt5 REQUIRED COMPONENTS
Core
CONFIG
)
target_include_directories(clory-lib
PUBLIC
${CMAKE_CURRENT_LIST_DIR}/src/main/cpp
)
set(_CLORYSDK_LIB_PATH ${CMAKE_CURRENT_LIST_DIR}/src/main/jniLibs/${ANDROID_ABI})
target_link_libraries(clory-lib
${log-lib}
-L${_CLORYSDK_LIB_PATH}
clorysdk
Qt5::Core
)
The library clorysdk is actually our internal library I was talking about, which contains e.g. Clory::Engine::instance I would like to step into with the debugger. It was built with qmake and is built in debug mode (CONFIG+=debug was added in the effective qmake call).
EDIT 3:
In the LLDB session which has opened after it hit the Java_clory_engine_sdk_CloryNative_nativeInit breakpoint, I got the following:
(lldb) image lookup -vrn Clory::Engine::instance
2 matches found in /Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so:
Address: libclorysdk.so[0x0001bb32] (libclorysdk.so..text + 8250)
Summary: libclorysdk.so`Clory::Engine::instance(Clory::Engine::Purpose)
Module: file = "/Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so", arch = "arm"
Symbol: id = {0x0000005e}, range = [0xcb41eb32-0xcb41ebc0), name="Clory::Engine::instance(Clory::Engine::Purpose)", mangled="_ZN4Clory2Engine8instanceENS0_7PurposeE"
Address: libclorysdk.so[0x0001b82c] (libclorysdk.so..text + 7476)
Summary: libclorysdk.so`Clory::Engine::instance(Clory::RuntimeConfiguration const&, Clory::Engine::Purpose)
Module: file = "/Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so", arch = "arm"
Symbol: id = {0x000000bd}, range = [0xcb41e82c-0xcb41e970), name="Clory::Engine::instance(Clory::RuntimeConfiguration const&, Clory::Engine::Purpose)", mangled="_ZN4Clory2Engine8instanceERKNS_20RuntimeConfigurationENS0_7PurposeE"
(lldb) settings show target.source-map
target.source-map (path-map) =
First of all, there was no CompileUnit section in the result of the command image lookup -vrn Clory::Engine::instance. How is this possible to have no source-map defined(second lldb command) if the libclorysdk.so was built in Debug mode? Is it possible to explicitly set it so that the debugger would search there for the library's source files?
EDIT 4:
After searching more I found out that the process of creating the APK actually strips the *.so libraries from their debugging symbols. libclorysdk.so built in debug mode has about 10MB while the libclorysdk.so file which I extracted after unarchiving the generated *.apk file is just 350KB.
As stated here, running greadelf --debug-dump=decodedline libclorysdk.so on the debug version outputs references to the source files, but if the command is run on the *.apk extracted library, it outputs nothing.
Is there a way to stop Android Studio from stripping the *.sos? I tried How to avoid stripping for native code symbols for android app but didn't have any effect, *.apk file is the same size as before and debugging the native libraries still doesn't work.
I'm using Gradle 3.1.4.
EDIT 5:
The stripping solution works, but in my case, it needed a Clean & Build before hitting the breakpoints in the library. Deploying *.sos which are not stripped is allowing you to have debugging sessions and step inside the native libraries.
Note:
If the libraries are built using the Qt for Android toolchain, the *.sos deployed to $SHADOW_BUILD/android-build are also stripped(where $SHADOW_BUILD is the build directory usually starting with build-*). So in order to debug those you should copy them from outside the android-build directory where each *.so is generated.
The debug info records the location of the source files when they were built.
(lldb) image lookup -vrn Clory::Engine::instance
The CompileUnit line shows the source file. Suppose it says:
"/BuildDirectory/Sources/Clory/CloryEngine.cpp"
Let's assume you have the source on your machine here:
"Users/me/Sources/Clory"
So you can tell lldb: find the source file rooted at /BuildDirectory/Sources/Clory in Users/me/Sources/Clory instead.
(lldb) settings set target.source-map /BuildDirectory/Sources/Clory Users/me/Sources/Clory
You can use these commands in the lldb console of Android Studio or put into a .lldbinit file for general use.
If there no debug symbols available, you might have to build the referenced library in debug mode.
Either with -DCMAKE_BUILD_TYPE=DEBUG:
defaultConfig {
externalNativeBuild {
cmake {
arguments "-DANDROID_TOOLCHAIN=gcc", "-DCMAKE_BUILD_TYPE=DEBUG"
cppFlags "-std=c++14 -fexceptions -frtti"
}
}
}
externalNativeBuild {
cmake {
path file('src/main/cpp/CMakeLists.txt')
}
}
Or add this to the CMakeLists.txt of the library:
set(CMAKE_BUILD_TYPE Debug)
See the CMake documentation and Symbolicating with LLDB.
Elsewhere it explains (lldb) settings set target.source-map /buildbot/path /my/path:
Remap source file path-names for the debug session. If your source files are no longer located in the same location as when the program was built --- maybe the program was built on a different computer --- you need to tell the debugger how to find the sources at their local file path instead of the build system's file path.
There's also (lldb) settings show target.source-map, to see what is mapped.
(lldb) set append target.source-map /buildbot/path /my/path seems rather suitable, in order not to overwrite existing mappings.
I have an Android native library (C++ code base) called:
libserverapp.so
And I cannot get the Android build to find it:
"DllNotFoundException: serverapp"
I am using an internal build system, but when I parse the output of the build process, I can see many calls of the form:
android-ndk/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/arm-linux-androideabi-g++.exe -march=armv7-a
After building my Android app, I inspect the resulting APK (renaming to .zip and extracting), and can see my library file here:
lib/armeabi-v7a/libserverapp.so
I can confirm that "ARMv7" is the target architecture in the Android Player settings, and I access the library, in C#, via:
[DllImport("serverapp", CallingConvention = CallingConvention.Cdecl)]
private static extern void run_sim(StringBuilder matchInput, StringBuilder results, int randomSeed);
I have built a Windows DLL of the C++ code, to use in the Editor, and everything works great. However, when I move to Android, the .so cannot be found. The import settings for libserverapp.so are:
Platform: Android; CPU: ARMv7; Path: Assets/Plugins/Android/libserverapp.so; Type: Native
Given that the final APK includes the .so where I expect it to be (lib/armeabi-v7a/), I assume my Unity settings are correct? Also, I am not using IL2CPP for the Android build.
Finally, when I do an object dump of the library file (using arm-linux-androideabi-objdump.exe), the file format of the library file is "elf32-littlearm".
I feel that the issue here is simply finding the .so, not the functionality within it. Any ideas on what's going on here?
Thanks!
I ended up solving the problem. I mentioned that was using an internal build system. Well, there seems to be a bug in it. I ported things over to official Android NDK makefiles, and then it "just worked". So in this case, the library could be found, but its contents weren't valid.
Swiften is a XMPP client library and my objective was to build it for a ARM-embedded target running Linux.
I hacked my way to a successful cross-compile with little knowledge of SCons. I'll lay out my hack here with the hope that someone can point me to a maintainable solution using the two makery files, SConscript.boot and SConstruct.
I had two tasks (neither accomplished satisfactorily):
Successfully switching the tool-chain from native-compile to cross-compile
Ensuring that OpenSSL libraries were successfully linked (not supplied by the swiftim project; they has to be installed and built in the 3rdParty folder).
Switching the tool-chain from native-compile to cross-compile for ARM
My ARM cross tool-chain components, gcc, g++, ld, etc are located here.
/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/
I couldn't find a way to tell scons to use the cross tool-chain (from the above location) instead of the native tool (in the usual place, /usr/bin). Prefacing the invocation (./scons Swiften) with the fully-qualified values for the environment variables, CC and CXX didn't work (while not recommended, its alluded to in one place).
Scons would only pick up the native tool-chain even after many ad hoc changes to the makery.
So, as a hack, I had to change the native tool-chain to point to the cross tool-chain.
/usr/bin/gcc -> /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/bin/arm-linux-gnueabihf-gcc-4.7.3*
/usr/bin/g++ -> /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/bin/arm-linux-gnueabihf-g++*
The first compile-break for ARM was fixed by adding the line below to the default portion of the build script, SConscript.boot.
env.Append(CPPDEFINES = ["_LITTLE_ENDIAN"])
The next compile-break has to do with the OpenSSL header files not being found. To fix the location issue, I had to introduce the line below into SConscript.boot
vars.Add(PackageVariable("openssl", "OpenSSL location", "/home/auro-tripathy/swiftim/swift/3rdParty/OpenSSL/openssl-1.0.1c/"))
Linking with OpenSSL
For the sample Switften programs to link with the OpenSSL libraries, I had to move libssl.a and libcrypto.a (built separately) from the location they were built to the toolchain library-location like so.
mv ~/swiftim/swift/3rdParty/OpenSSL/openssl-1.0.1c/libcrypto.a /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/lib/gcc/arm-linux-gnueabihf/4.7.3/.
Help
Not understanding of the working of scons, I've made some hacks to get it to work.
I’d like some help to:
Introduce a new target called ARM-embedded, just like other targets; iPhone, android, etc
Clean way to integrate OpenSSL into the build .
Update
Per dirkbaechle, retried the script below and it works
export CC=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/gcc
export CXX=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/g++
./scons Swiften
Brady's answer is correct, regarding how you'd do it in plain SCons. I'd just like to mention that the top-level SConstruct of Swiften already provides arguments like "cc=" and "cxx=" for using local toolchains.
You might want to inspect the ouput of scons -h for a complete list of available options.
In addition, the SConscript for the OpenSSL build expects the sources to be located in the relative folder named "openssl", not "openssl-1.0.1c" as in your case. Maybe that's where your build problems are mainly coming from.
I left a comment above regarding the cross-compilation. Its already been answered in the link provided, but basically you just need to set the appropriate construction variables: CC, CXX, LINK, etc.
As for a "Clean way to integrate OpenSSL into the build" this can be performed simply by adding library and include paths appropriately as follows replacing the quoted values appropriately:
(without having to copy/move the original files)
# This sets the location of the OpenSSL Include paths
env.Append(CPPPATH="path/to/openssl/includes")
# This sets the location of the OpenSSL Libraries
env.Append(LIBPATH="path/to/openssl/libraries")
# These are the OpenSSL libraries to be linked into the binary
env.Append(LIBS=["OpenSSL_lib", "OpenSSL_lib2"])
The choice of compiler, and additional flags, can all be set in Swift's config.py file. A snippet from config.py using a custom compiler and flags is below (the one I use on one of my dev boxes):
cc = link = "/usr/local/llvm-git/bin/clang"
cxx = "/usr/local/llvm-git/bin/clang++"
bothflags = " -std=c++11 -stdlib=libc++ -nostdinc++"
cxxflags = bothflags + " -I/usr/local/libcxx/include -Wno-deprecated"
linkflags = bothflags + " -L/usr/local/libcxx/lib"
This should work for cross-compiling in the same manner.
To use a bundled openssl, you should just be able to extract into 3rdParty/OpenSSL, and add openssl_force_bundled = True to your config.py. You should not need to fiddle with setting include paths to this yourself. It's conceivable that this is tied to a particular openssl release as I've not compiled a bundled openssl since 1.0.0a, but if it doesn't work with the current version it's probably a bug that ought to be fixed. You could also cross-compile openssl yourself and use openssl='/path/to/openssl', but that's a little more of a nuisance for you.
I currently developing app for Android that provides various audio settings. I use android system prepared by someone else, and it provides (I see in source and compiled files) some methods that could be useful for me. For example there are (kernel/drivers/audio/audio.c) some methods to change bands (for equalizer). In compiled system there is audio_setting.so file in some audio dir on kernel. Is there a possibility to use this methods (library) in my application NDK? I don't want to compile my app with whole system, rather to dynamically add this lib.
edit:
It was simplier than I thought. I used:
void *some_lib;
bundlewrapper = dlopen("some/path/some_lib.so", RTLD_LAZY);
if ( some_lib!= NULL ) {
LOGV("Loaded lib\n");
// use methods from lib
}
Sure, you can use any code on the system.
Obviously if it is non-JNI code you'll have to call it from your own JNI code or wrapper.
In your Android.mk file you will need to add the extra lib in LOCAL_LDLIBS