Develop Reference

Android NDK Firebase Crashlytics Symbolication ( C++ ) [duplicate]

I'm working on an Android project which uses a Java class that is a wrapper on a C++ library. The C++ library is a company internal library and we have access to its source code, but in the Android project it is only dynamically linked, so it is used only in the form of headers (.h) and shared objects (.so). Having access to the library source code, is it possible to specify to Android Studio the path to the source code so I can step inside the library using the debugger?
The debugger works, I can step inside the Java_clory_engine_sdk_CloryNative_nativeInit function, but I would also like to further debug the library corresponding to the Clory::Engine class which, as I mentioned, is an internal library we have source code access to.
For example, Clory::Engine::instance is part of the library and I would like to specify to Android Studio the location of the CloryEngine.cpp file so I can step inside Clory::Engine::instance with the debugger, thus debugging this static member function.
I am using Android Studio 3.1.4.
Is this possible?
EDIT:
The clory-sdk.gradle file specifies the CMakeLists.txt file which configures the C++ layer.
externalNativeBuild {
cmake {
path "CMakeLists.txt"
}
}
So I am using an internal application which uses the Clory SDK. Inside the app.gradle file I use:
dependencies {
...
compile project(':clory-sdk-core')
compile project(':clory-sdk')
...
}
so I don't think we're using the aars for the app.gradle project. The aars are shipped to the client, but we are using app.gradle project to test our little SDK functionalities before doing that. The JNI layer is inside clory-sdk-core project.
EDIT 2:
Here is the CMakeLists.txt which handles the JNI layer:
cmake_minimum_required(VERSION 3.4.1)
set(CMAKE_AUTOMOC ON)
set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_BUILD_TYPE Debug)
add_library(
clory-lib
SHARED
# JNI layer and other helper classes for transferring data from Java to Qt/C++
src/main/cpp/clory-lib.cpp
src/main/cpp/JObjectHandler.cpp
src/main/cpp/JObjectResolver.cpp
src/main/cpp/JObjectCreator.cpp
src/main/cpp/DataConverter.cpp
src/main/cpp/JObjectHelper.cpp
src/main/cpp/JEnvironmentManager.cpp
)
find_library(
log-lib
log
)
target_compile_options(clory-lib
PUBLIC
-std=c++11
)
# Hardcoded for now...will fix later...
set(_QT_ROOT_PATH /Users/jacob/Qt/5.8)
if(${ANDROID_ABI} MATCHES ^armeabi-v7.*$)
set(_QT_ARCH android_armv7)
elseif(${ANDROID_ABI} MATCHES ^x86$)
set(_QT_ARCH android_x86)
else()
message(FATAL_ERROR "Unsupported Android architecture!!!")
endif()
set(CMAKE_FIND_ROOT_PATH ${_QT_ROOT_PATH}/${_QT_ARCH})
find_package(Qt5 REQUIRED COMPONENTS
Core
CONFIG
)
target_include_directories(clory-lib
PUBLIC
${CMAKE_CURRENT_LIST_DIR}/src/main/cpp
)
set(_CLORYSDK_LIB_PATH ${CMAKE_CURRENT_LIST_DIR}/src/main/jniLibs/${ANDROID_ABI})
target_link_libraries(clory-lib
${log-lib}
-L${_CLORYSDK_LIB_PATH}
clorysdk
Qt5::Core
)
The library clorysdk is actually our internal library I was talking about, which contains e.g. Clory::Engine::instance I would like to step into with the debugger. It was built with qmake and is built in debug mode (CONFIG+=debug was added in the effective qmake call).
EDIT 3:
In the LLDB session which has opened after it hit the Java_clory_engine_sdk_CloryNative_nativeInit breakpoint, I got the following:
(lldb) image lookup -vrn Clory::Engine::instance
2 matches found in /Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so:
Address: libclorysdk.so[0x0001bb32] (libclorysdk.so..text + 8250)
Summary: libclorysdk.so`Clory::Engine::instance(Clory::Engine::Purpose)
Module: file = "/Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so", arch = "arm"
Symbol: id = {0x0000005e}, range = [0xcb41eb32-0xcb41ebc0), name="Clory::Engine::instance(Clory::Engine::Purpose)", mangled="_ZN4Clory2Engine8instanceENS0_7PurposeE"
Address: libclorysdk.so[0x0001b82c] (libclorysdk.so..text + 7476)
Summary: libclorysdk.so`Clory::Engine::instance(Clory::RuntimeConfiguration const&, Clory::Engine::Purpose)
Module: file = "/Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so", arch = "arm"
Symbol: id = {0x000000bd}, range = [0xcb41e82c-0xcb41e970), name="Clory::Engine::instance(Clory::RuntimeConfiguration const&, Clory::Engine::Purpose)", mangled="_ZN4Clory2Engine8instanceERKNS_20RuntimeConfigurationENS0_7PurposeE"
(lldb) settings show target.source-map
target.source-map (path-map) =
First of all, there was no CompileUnit section in the result of the command image lookup -vrn Clory::Engine::instance. How is this possible to have no source-map defined(second lldb command) if the libclorysdk.so was built in Debug mode? Is it possible to explicitly set it so that the debugger would search there for the library's source files?
EDIT 4:
After searching more I found out that the process of creating the APK actually strips the *.so libraries from their debugging symbols. libclorysdk.so built in debug mode has about 10MB while the libclorysdk.so file which I extracted after unarchiving the generated *.apk file is just 350KB.
As stated here, running greadelf --debug-dump=decodedline libclorysdk.so on the debug version outputs references to the source files, but if the command is run on the *.apk extracted library, it outputs nothing.
Is there a way to stop Android Studio from stripping the *.sos? I tried How to avoid stripping for native code symbols for android app but didn't have any effect, *.apk file is the same size as before and debugging the native libraries still doesn't work.
I'm using Gradle 3.1.4.
EDIT 5:
The stripping solution works, but in my case, it needed a Clean & Build before hitting the breakpoints in the library. Deploying *.sos which are not stripped is allowing you to have debugging sessions and step inside the native libraries.
Note:
If the libraries are built using the Qt for Android toolchain, the *.sos deployed to $SHADOW_BUILD/android-build are also stripped(where $SHADOW_BUILD is the build directory usually starting with build-*). So in order to debug those you should copy them from outside the android-build directory where each *.so is generated.

The debug info records the location of the source files when they were built.
(lldb) image lookup -vrn Clory::Engine::instance
The CompileUnit line shows the source file. Suppose it says:
"/BuildDirectory/Sources/Clory/CloryEngine.cpp"
Let's assume you have the source on your machine here:
"Users/me/Sources/Clory"
So you can tell lldb: find the source file rooted at /BuildDirectory/Sources/Clory in Users/me/Sources/Clory instead.
(lldb) settings set target.source-map /BuildDirectory/Sources/Clory Users/me/Sources/Clory
You can use these commands in the lldb console of Android Studio or put into a .lldbinit file for general use.

If there no debug symbols available, you might have to build the referenced library in debug mode.
Either with -DCMAKE_BUILD_TYPE=DEBUG:
defaultConfig {
externalNativeBuild {
cmake {
arguments "-DANDROID_TOOLCHAIN=gcc", "-DCMAKE_BUILD_TYPE=DEBUG"
cppFlags "-std=c++14 -fexceptions -frtti"
}
}
}
externalNativeBuild {
cmake {
path file('src/main/cpp/CMakeLists.txt')
}
}
Or add this to the CMakeLists.txt of the library:
set(CMAKE_BUILD_TYPE Debug)
See the CMake documentation and Symbolicating with LLDB.
Elsewhere it explains (lldb) settings set target.source-map /buildbot/path /my/path:
Remap source file path-names for the debug session. If your source files are no longer located in the same location as when the program was built --- maybe the program was built on a different computer --- you need to tell the debugger how to find the sources at their local file path instead of the build system's file path.
There's also (lldb) settings show target.source-map, to see what is mapped.
(lldb) set append target.source-map /buildbot/path /my/path seems rather suitable, in order not to overwrite existing mappings.

Unity: DllNotFoundException (Unity 2018.2; Android)

I have an Android native library (C++ code base) called:
libserverapp.so
And I cannot get the Android build to find it:
"DllNotFoundException: serverapp"
I am using an internal build system, but when I parse the output of the build process, I can see many calls of the form:
android-ndk/toolchains/arm-linux-androideabi-4.6/prebuilt/windows/bin/arm-linux-androideabi-g++.exe -march=armv7-a
After building my Android app, I inspect the resulting APK (renaming to .zip and extracting), and can see my library file here:
lib/armeabi-v7a/libserverapp.so
I can confirm that "ARMv7" is the target architecture in the Android Player settings, and I access the library, in C#, via:
[DllImport("serverapp", CallingConvention = CallingConvention.Cdecl)]
private static extern void run_sim(StringBuilder matchInput, StringBuilder results, int randomSeed);
I have built a Windows DLL of the C++ code, to use in the Editor, and everything works great. However, when I move to Android, the .so cannot be found. The import settings for libserverapp.so are:
Platform: Android; CPU: ARMv7; Path: Assets/Plugins/Android/libserverapp.so; Type: Native
Given that the final APK includes the .so where I expect it to be (lib/armeabi-v7a/), I assume my Unity settings are correct? Also, I am not using IL2CPP for the Android build.
Finally, when I do an object dump of the library file (using arm-linux-androideabi-objdump.exe), the file format of the library file is "elf32-littlearm".
I feel that the issue here is simply finding the .so, not the functionality within it. Any ideas on what's going on here?
Thanks!

I ended up solving the problem. I mentioned that was using an internal build system. Well, there seems to be a bug in it. I ported things over to official Android NDK makefiles, and then it "just worked". So in this case, the library could be found, but its contents weren't valid.

Cross-Compiling Swiften library for ARM-embedded with SCons makery

Swiften is a XMPP client library and my objective was to build it for a ARM-embedded target running Linux.
I hacked my way to a successful cross-compile with little knowledge of SCons. I'll lay out my hack here with the hope that someone can point me to a maintainable solution using the two makery files, SConscript.boot and SConstruct.
I had two tasks (neither accomplished satisfactorily):
Successfully switching the tool-chain from native-compile to cross-compile
Ensuring that OpenSSL libraries were successfully linked (not supplied by the swiftim project; they has to be installed and built in the 3rdParty folder).
Switching the tool-chain from native-compile to cross-compile for ARM
My ARM cross tool-chain components, gcc, g++, ld, etc are located here.
/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/
I couldn't find a way to tell scons to use the cross tool-chain (from the above location) instead of the native tool (in the usual place, /usr/bin). Prefacing the invocation (./scons Swiften) with the fully-qualified values for the environment variables, CC and CXX didn't work (while not recommended, its alluded to in one place).
Scons would only pick up the native tool-chain even after many ad hoc changes to the makery.
So, as a hack, I had to change the native tool-chain to point to the cross tool-chain.
/usr/bin/gcc -> /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/bin/arm-linux-gnueabihf-gcc-4.7.3*
/usr/bin/g++ -> /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/bin/arm-linux-gnueabihf-g++*
The first compile-break for ARM was fixed by adding the line below to the default portion of the build script, SConscript.boot.
env.Append(CPPDEFINES = ["_LITTLE_ENDIAN"])
The next compile-break has to do with the OpenSSL header files not being found. To fix the location issue, I had to introduce the line below into SConscript.boot
vars.Add(PackageVariable("openssl", "OpenSSL location", "/home/auro-tripathy/swiftim/swift/3rdParty/OpenSSL/openssl-1.0.1c/"))
Linking with OpenSSL
For the sample Switften programs to link with the OpenSSL libraries, I had to move libssl.a and libcrypto.a (built separately) from the location they were built to the toolchain library-location like so.
mv ~/swiftim/swift/3rdParty/OpenSSL/openssl-1.0.1c/libcrypto.a /opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/lib/gcc/arm-linux-gnueabihf/4.7.3/.
Help
Not understanding of the working of scons, I've made some hacks to get it to work.
I’d like some help to:
Introduce a new target called ARM-embedded, just like other targets; iPhone, android, etc
Clean way to integrate OpenSSL into the build .
Update
Per dirkbaechle, retried the script below and it works
export CC=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/gcc
export CXX=/opt/toolchain/gcc-linaro-arm-linux-gnueabihf-4.7-2013.01-20130125_linux/arm-linux-gnueabihf/bin/g++
./scons Swiften

Brady's answer is correct, regarding how you'd do it in plain SCons. I'd just like to mention that the top-level SConstruct of Swiften already provides arguments like "cc=" and "cxx=" for using local toolchains.
You might want to inspect the ouput of scons -h for a complete list of available options.
In addition, the SConscript for the OpenSSL build expects the sources to be located in the relative folder named "openssl", not "openssl-1.0.1c" as in your case. Maybe that's where your build problems are mainly coming from.

I left a comment above regarding the cross-compilation. Its already been answered in the link provided, but basically you just need to set the appropriate construction variables: CC, CXX, LINK, etc.
As for a "Clean way to integrate OpenSSL into the build" this can be performed simply by adding library and include paths appropriately as follows replacing the quoted values appropriately:
(without having to copy/move the original files)
# This sets the location of the OpenSSL Include paths
env.Append(CPPPATH="path/to/openssl/includes")
# This sets the location of the OpenSSL Libraries
env.Append(LIBPATH="path/to/openssl/libraries")
# These are the OpenSSL libraries to be linked into the binary
env.Append(LIBS=["OpenSSL_lib", "OpenSSL_lib2"])

The choice of compiler, and additional flags, can all be set in Swift's config.py file. A snippet from config.py using a custom compiler and flags is below (the one I use on one of my dev boxes):
cc = link = "/usr/local/llvm-git/bin/clang"
cxx = "/usr/local/llvm-git/bin/clang++"
bothflags = " -std=c++11 -stdlib=libc++ -nostdinc++"
cxxflags = bothflags + " -I/usr/local/libcxx/include -Wno-deprecated"
linkflags = bothflags + " -L/usr/local/libcxx/lib"
This should work for cross-compiling in the same manner.
To use a bundled openssl, you should just be able to extract into 3rdParty/OpenSSL, and add openssl_force_bundled = True to your config.py. You should not need to fiddle with setting include paths to this yourself. It's conceivable that this is tied to a particular openssl release as I've not compiled a bundled openssl since 1.0.0a, but if it doesn't work with the current version it's probably a bug that ought to be fixed. You could also cross-compile openssl yourself and use openssl='/path/to/openssl', but that's a little more of a nuisance for you.

Android NDK Eclipse CDT won't allow me to use substr method of string

Even a simple code like this gives the error
string test= "hello";
string part = test.substr(2, 3);
Invalid arguments Candidates are: stlpmtx_std::basic_string,stlpmtx_std::allocator> substr(?, ?)
Even casting to size_t did not not help
string part = test.substr((size_t)2, (size_t)3);
But both pieces of code working when running it as a console application using gcc 4.6 on Linux.
But when using Androind NDK it gives me the error.
If I close the file in the editor and close Eclipse, and then open Eclipse and build the project it compiles. But if I have the file open it throws up the error.
I am using APP_STL:= stlport_static in the Application.mk make file
Yes I did include <string>
Edit: I just looked at the times of .so file, it seems to be compiling even though Eclipse is showing the errors.

I found the solution I need to have to have stlport system include folder in the Paths and Symbols section of project properties -> C/C++ General
path_To_NDK/sources/cxx-stl/system/include
I only had the
path_To_NDK/sources/cxx-stl/stlport/stlport

Using Boost in android ndk with windows

I am trying to use boost library with Android ndk in Eclipse with Windows. I tried to follow this tutorial
I am stuck in the step with "bjam" command in cygwin.
bjam --without-python --without-serialization toolset=gcc-android4.4.3 link=static runtime-link=static target-os=linux --stagedir=android
Error: bjam command not found.
What is bjam? Also I used the boost 1.53 along ndk r8e. Can someone help me with this please?

Android NDK is no longer dependent on Cygwin, so you can build Boost with the NDK from within Windows command prompt (cmd).
In order to make Boost.Build find the NDK, edit boost\tools\build\v2\user-config.jam file and append the following text:
import os ;
androidNDKRoot = C:/android-ndk-r8e ; # put the relevant path
using gcc : android :
$(androidNDKRoot)/toolchains/arm-linux-androideabi-4.7/prebuilt/windows/bin/arm-linux-androideabi-g++ :
<compileflags>--sysroot=$(androidNDKRoot)/platforms/android-9/arch-arm
<compileflags>-mthumb
<compileflags>-Os
<compileflags>-fno-strict-aliasing
<compileflags>-O2
<compileflags>-DNDEBUG
<compileflags>-g
<compileflags>-lstdc++
<compileflags>-I$(androidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.7/include
<compileflags>-I$(androidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.7/libs/armeabi/include
<compileflags>-D__GLIBC__
<compileflags>-D_GLIBCXX__PTHREADS
<compileflags>-D__arm__
<compileflags>-D_REENTRANT
<archiver>$(androidNDKRoot)/toolchains/arm-linux-androideabi-4.7/prebuilt/windows/bin/arm-linux-androideabi-ar
<ranlib>$(androidNDKRoot)/toolchains/arm-linux-androideabi-4.7/prebuilt/windows/bin/arm-linux-androideabi-ranlib
;
Certainly, instead of c:/android-ndk-r8e you have to put the right location of the NDK on your PC.
Besides, you can select more recent platform API, instead of android-9.
Also note that the NDK supplies several tool-chains, and the above settings point to gcc-4.7. If you prefer to build boost with some other tool-chain, change arm-linux-androideabi-4.7 to the relevant path.
After you've put the configuration in user-config.jam, open cmd, cd to the directory where Boost resides, and invoke bootstrap. Then invoke b2 like this (for example):
b2 --without-python --without-serialization threading=multi link=static runtime-link=static toolset=gcc-android target-os=linux threadapi=pthread --stagedir=android stage
UPDATE: As of 11/2015, older NDK toolchains seem to have issues with the newer Boost versions, causing compiler crash, so consider using a more recent compiler. To do this, just change every 4.7 occurrence in the above script to 4.9. Also, it's worth compiling with a more recent Android API (eg. andoroid-9 -> andoroid-16 or so).