Develop Reference

Android NDK Firebase Crashlytics Symbolication ( C++ ) [duplicate]

I'm working on an Android project which uses a Java class that is a wrapper on a C++ library. The C++ library is a company internal library and we have access to its source code, but in the Android project it is only dynamically linked, so it is used only in the form of headers (.h) and shared objects (.so). Having access to the library source code, is it possible to specify to Android Studio the path to the source code so I can step inside the library using the debugger?
The debugger works, I can step inside the Java_clory_engine_sdk_CloryNative_nativeInit function, but I would also like to further debug the library corresponding to the Clory::Engine class which, as I mentioned, is an internal library we have source code access to.
For example, Clory::Engine::instance is part of the library and I would like to specify to Android Studio the location of the CloryEngine.cpp file so I can step inside Clory::Engine::instance with the debugger, thus debugging this static member function.
I am using Android Studio 3.1.4.
Is this possible?
EDIT:
The clory-sdk.gradle file specifies the CMakeLists.txt file which configures the C++ layer.
externalNativeBuild {
cmake {
path "CMakeLists.txt"
}
}
So I am using an internal application which uses the Clory SDK. Inside the app.gradle file I use:
dependencies {
...
compile project(':clory-sdk-core')
compile project(':clory-sdk')
...
}
so I don't think we're using the aars for the app.gradle project. The aars are shipped to the client, but we are using app.gradle project to test our little SDK functionalities before doing that. The JNI layer is inside clory-sdk-core project.
EDIT 2:
Here is the CMakeLists.txt which handles the JNI layer:
cmake_minimum_required(VERSION 3.4.1)
set(CMAKE_AUTOMOC ON)
set(CMAKE_INCLUDE_CURRENT_DIR ON)
set(CMAKE_BUILD_TYPE Debug)
add_library(
clory-lib
SHARED
# JNI layer and other helper classes for transferring data from Java to Qt/C++
src/main/cpp/clory-lib.cpp
src/main/cpp/JObjectHandler.cpp
src/main/cpp/JObjectResolver.cpp
src/main/cpp/JObjectCreator.cpp
src/main/cpp/DataConverter.cpp
src/main/cpp/JObjectHelper.cpp
src/main/cpp/JEnvironmentManager.cpp
)
find_library(
log-lib
log
)
target_compile_options(clory-lib
PUBLIC
-std=c++11
)
# Hardcoded for now...will fix later...
set(_QT_ROOT_PATH /Users/jacob/Qt/5.8)
if(${ANDROID_ABI} MATCHES ^armeabi-v7.*$)
set(_QT_ARCH android_armv7)
elseif(${ANDROID_ABI} MATCHES ^x86$)
set(_QT_ARCH android_x86)
else()
message(FATAL_ERROR "Unsupported Android architecture!!!")
endif()
set(CMAKE_FIND_ROOT_PATH ${_QT_ROOT_PATH}/${_QT_ARCH})
find_package(Qt5 REQUIRED COMPONENTS
Core
CONFIG
)
target_include_directories(clory-lib
PUBLIC
${CMAKE_CURRENT_LIST_DIR}/src/main/cpp
)
set(_CLORYSDK_LIB_PATH ${CMAKE_CURRENT_LIST_DIR}/src/main/jniLibs/${ANDROID_ABI})
target_link_libraries(clory-lib
${log-lib}
-L${_CLORYSDK_LIB_PATH}
clorysdk
Qt5::Core
)
The library clorysdk is actually our internal library I was talking about, which contains e.g. Clory::Engine::instance I would like to step into with the debugger. It was built with qmake and is built in debug mode (CONFIG+=debug was added in the effective qmake call).
EDIT 3:
In the LLDB session which has opened after it hit the Java_clory_engine_sdk_CloryNative_nativeInit breakpoint, I got the following:
(lldb) image lookup -vrn Clory::Engine::instance
2 matches found in /Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so:
Address: libclorysdk.so[0x0001bb32] (libclorysdk.so..text + 8250)
Summary: libclorysdk.so`Clory::Engine::instance(Clory::Engine::Purpose)
Module: file = "/Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so", arch = "arm"
Symbol: id = {0x0000005e}, range = [0xcb41eb32-0xcb41ebc0), name="Clory::Engine::instance(Clory::Engine::Purpose)", mangled="_ZN4Clory2Engine8instanceENS0_7PurposeE"
Address: libclorysdk.so[0x0001b82c] (libclorysdk.so..text + 7476)
Summary: libclorysdk.so`Clory::Engine::instance(Clory::RuntimeConfiguration const&, Clory::Engine::Purpose)
Module: file = "/Users/jacob/.lldb/module_cache/remote-android/.cache/6EDE4F0A-0000-0000-0000-000000000000/libclorysdk.so", arch = "arm"
Symbol: id = {0x000000bd}, range = [0xcb41e82c-0xcb41e970), name="Clory::Engine::instance(Clory::RuntimeConfiguration const&, Clory::Engine::Purpose)", mangled="_ZN4Clory2Engine8instanceERKNS_20RuntimeConfigurationENS0_7PurposeE"
(lldb) settings show target.source-map
target.source-map (path-map) =
First of all, there was no CompileUnit section in the result of the command image lookup -vrn Clory::Engine::instance. How is this possible to have no source-map defined(second lldb command) if the libclorysdk.so was built in Debug mode? Is it possible to explicitly set it so that the debugger would search there for the library's source files?
EDIT 4:
After searching more I found out that the process of creating the APK actually strips the *.so libraries from their debugging symbols. libclorysdk.so built in debug mode has about 10MB while the libclorysdk.so file which I extracted after unarchiving the generated *.apk file is just 350KB.
As stated here, running greadelf --debug-dump=decodedline libclorysdk.so on the debug version outputs references to the source files, but if the command is run on the *.apk extracted library, it outputs nothing.
Is there a way to stop Android Studio from stripping the *.sos? I tried How to avoid stripping for native code symbols for android app but didn't have any effect, *.apk file is the same size as before and debugging the native libraries still doesn't work.
I'm using Gradle 3.1.4.
EDIT 5:
The stripping solution works, but in my case, it needed a Clean & Build before hitting the breakpoints in the library. Deploying *.sos which are not stripped is allowing you to have debugging sessions and step inside the native libraries.
Note:
If the libraries are built using the Qt for Android toolchain, the *.sos deployed to $SHADOW_BUILD/android-build are also stripped(where $SHADOW_BUILD is the build directory usually starting with build-*). So in order to debug those you should copy them from outside the android-build directory where each *.so is generated.

The debug info records the location of the source files when they were built.
(lldb) image lookup -vrn Clory::Engine::instance
The CompileUnit line shows the source file. Suppose it says:
"/BuildDirectory/Sources/Clory/CloryEngine.cpp"
Let's assume you have the source on your machine here:
"Users/me/Sources/Clory"
So you can tell lldb: find the source file rooted at /BuildDirectory/Sources/Clory in Users/me/Sources/Clory instead.
(lldb) settings set target.source-map /BuildDirectory/Sources/Clory Users/me/Sources/Clory
You can use these commands in the lldb console of Android Studio or put into a .lldbinit file for general use.

If there no debug symbols available, you might have to build the referenced library in debug mode.
Either with -DCMAKE_BUILD_TYPE=DEBUG:
defaultConfig {
externalNativeBuild {
cmake {
arguments "-DANDROID_TOOLCHAIN=gcc", "-DCMAKE_BUILD_TYPE=DEBUG"
cppFlags "-std=c++14 -fexceptions -frtti"
}
}
}
externalNativeBuild {
cmake {
path file('src/main/cpp/CMakeLists.txt')
}
}
Or add this to the CMakeLists.txt of the library:
set(CMAKE_BUILD_TYPE Debug)
See the CMake documentation and Symbolicating with LLDB.
Elsewhere it explains (lldb) settings set target.source-map /buildbot/path /my/path:
Remap source file path-names for the debug session. If your source files are no longer located in the same location as when the program was built --- maybe the program was built on a different computer --- you need to tell the debugger how to find the sources at their local file path instead of the build system's file path.
There's also (lldb) settings show target.source-map, to see what is mapped.
(lldb) set append target.source-map /buildbot/path /my/path seems rather suitable, in order not to overwrite existing mappings.

Building Verilator (C++) with CMake built-in NDK

I tried with this example, but nothing happens:
cmake_minimum_required(VERSION 3.8)
project(cmake_simulator)
set(CMAKE_SYSTEM_NAME Android)
set(CMAKE_SYSTEM_VERSION 21)
set(CMAKE_ANDROID_ARCH_ABI x86)
set(CMAKE_ANDROID_NDK /home/icarolima/Android/Sdk/ndk/21.3.6528147)
set(CMAKE_ANDROID_STL_TYPE gnustl_static)
set(CMAKE_TOOLCHAIN_FILE /home/icarolima/Android/Sdk/ndk/21.3.6528147/build/cmake/android.toolchain.cmake)
find_package(verilator HINTS $ENV{VERILATOR_ROOT} ${VERILATOR_ROOT})
if (NOT verilator_FOUND)
message(FATAL_ERROR "Verilator was not found. Either install it, or set the VERILATOR_ROOT environment variable")
endif()
# Create a new executable target that will contain all your sources
add_library(simulator SHARED simulator.cpp)
# Add the Verilated circuit to the target
verilate(simulator
INCLUDE_DIRS "."
SOURCES top.sv
VERILATOR_ARGS -Wno-CASEINCOMPLETE -Wno-WIDTH -Wno-COMBDLY -cc +1800-2012ext+sv)
For example, if I change the CMAKE_ANDROID_ARCH_ABI to anything else, nothing happens. It is like CMake is ignoring the NDK part of the code.
But If I change the project to another location, different things happen:
cmake_minimum_required(VERSION 3.8)
set(CMAKE_SYSTEM_NAME Android)
set(CMAKE_SYSTEM_VERSION 21)
set(CMAKE_ANDROID_ARCH_ABI x86)
set(CMAKE_ANDROID_NDK /home/icarolima/Android/Sdk/ndk/21.3.6528147)
set(CMAKE_ANDROID_STL_TYPE gnustl_static)
project(cmake_simulator)
set(CMAKE_TOOLCHAIN_FILE /home/icarolima/Android/Sdk/ndk/21.3.6528147/build/cmake/android.toolchain.cmake)
find_package(verilator HINTS $ENV{VERILATOR_ROOT} ${VERILATOR_ROOT})
if (NOT verilator_FOUND)
message(FATAL_ERROR "Verilator was not found. Either install it, or set the VERILATOR_ROOT environment variable")
endif()
# Create a new executable target that will contain all your sources
add_library(simulator SHARED simulator.cpp)
# Add the Verilated circuit to the target
verilate(simulator
INCLUDE_DIRS "."
SOURCES top.sv
VERILATOR_ARGS -Wno-CASEINCOMPLETE -Wno-WIDTH -Wno-COMBDLY -cc +1800-2012ext+sv)
The error:
CMake Error at /home/icarolima/Android/Sdk/cmake/3.10.2.4988404/share/cmake-3.10/Modules/Platform/Android/Determine-Compiler-NDK.cmake:97 (message):
Android: No toolchain for ABI 'x86' found in the NDK:
/home/icarolima/Android/Sdk/ndk/21.3.6528147
I have no experience with CMake, I think that the problem is the order of the things. Can anyone help me?

Setting all of these variables (such as CMAKE_SYSTEM_NAME, CMAKE_SYSTEM_VERSION, CMAKE_ANDROID_ARCH_ABI, etc.) should happen in the toolchain file. You may certainly experience some nasty CMake behavior by putting these in the CMakeLists.txt file itself. There is even a sample toolchain file in the CMake documentation you linked here.
Also, the CMAKE_TOOLCHAIN_FILE variable should be set on the command line when you call cmake, not in the CMake file itself. This reduces your CMakeLists.txt file to something like this:
cmake_minimum_required(VERSION 3.8)
project(cmake_simulator)
find_package(verilator HINTS $ENV{VERILATOR_ROOT} ${VERILATOR_ROOT})
if (NOT verilator_FOUND)
message(FATAL_ERROR "Verilator was not found. Either install it, or set the VERILATOR_ROOT environment variable")
endif()
# Create a new executable target that will contain all your sources
add_library(simulator SHARED simulator.cpp)
# Add the Verilated circuit to the target
verilate(simulator
INCLUDE_DIRS "."
SOURCES top.sv
VERILATOR_ARGS -Wno-CASEINCOMPLETE -Wno-WIDTH -Wno-COMBDLY -cc +1800-2012ext+sv)
Then, you should call cmake, specifying the toolchain file to use, like this:
cmake -DCMAKE_TOOLCHAIN_FILE=/home/icarolima/Android/Sdk/ndk/21.3.6528147/build/cmake/android.toolchain.cmake ..

So, just to clarify, the way I solved it can be seen here: Dockerfile, and here: sandbox_template.
Thanks for the answers #squareskittles!

Fetch and build libpng-android from CMake

I'm developing a C++ library for Android, and I have as dependency the libpng. I'm using the modified library for Android https://github.com/julienr/libpng-android.git, which has a build.sh file in order to compile for the different architectures. Or it can be compiled using ndk-build.
I'm Fetching the library from CMake using FetchContent, this effectively download the sources files, but it doesn't build properly. I'm setting the BUILD_COMMAND but it doesn't work.
FetchContent_Declare( png
GIT_REPOSITORY https://github.com/julienr/libpng-android.git
GIT_TAG master
UPDATE_DISCONNECTED TRUE
STEP_TARGETS update
BUILD_COMMAND "${NDK_PATH}/ndk-build.cmd NDK_PROJECT_PATH=${png_SOURCE_DIR}"
FetchContent_GetProperties(png)
if (NOT png_POPULATED)
FetchContent_Populate(png)
add_subdirectory("${png_SOURCE_DIR}" "${png_BINARY_DIR}" EXCLUDE_FROM_ALL)
endif()
It seems that the BUILD_COMMAND do nothing.
So, How can I fetch and build the libpng-android from CMake, and then import that result as an imported library?

To complete the Tsyvarev comment with references, according to the doc...
The following options are explicitly prohibited (they are disabled by the
FetchContent_Populate() command):
CONFIGURE_COMMAND
BUILD_COMMAND
INSTALL_COMMAND
TEST_COMMAND
ref: https://cmake.org/cmake/help/latest/module/FetchContent.html#command:fetchcontent_getproperties (just the paragraph above)

Thanks to #Tsyvarev and #Mizux I can follow the correct way to do it.
FetchContent_Declare( libpng
GIT_REPOSITORY https://github.com/julienr/libpng-android.git
GIT_TAG master
UPDATE_DISCONNECTED TRUE
STEP_TARGETS update)
FetchContent_GetProperties(libpng)
if (NOT libpng_POPULATED)
FetchContent_Populate(libpng)
add_subdirectory("${libpng_SOURCE_DIR}" ${libpng_BINARY_DIR} EXCLUDE_FROM_ALL)
endif()
exec_program("${CMAKE_ANDROID_NDK}/ndk-build.cmd"
${libpng_BINARY_DIR}
ARGS NDK_PROJECT_PATH=${libpng_SOURCE_DIR} APP_ABI=${ANDROID_ABI})
and then linking the libraries
target_link_libraries( project
PRIVATE
${libpng_SOURCE_DIR}/obj/local/${ANDROID_ABI}/libpng.a
)

Issue in Linking a C Library compiled with CMake in Android.

I am building a daemon on Android Platform. For which I have cross compiled the existing code, which is present in Linux to Android. Please consider my below 2 cases -
I cross compile the code using Cmake with Android NDK-15 and push manually to the board on which Android Code is available. The path in which we push is /system/lib. Then if I try to execute my daemon (which is also pushed manually to /system/bin ) everything works fine.
But,
2 The cross compiled library have been pushed into the AOSP code by writing an Android.mk file written in vendor specific folder. I have used BUILD_PREBUILTS method and was successfully able to push the libraries to /system/lib. Now, when I try to execute my application (Compiled with CMake ) I get a linker error.
Below is the error which I get ( Reference Example ) -
Below is the folder structure -
Total Number of folders - 3
a. abc folder which produces output libabc.so and links to libdef.so and
libghi.so
b. def folder which produces output libdef.so
c. ghi folder which produces output libghi.so
When I push all the 3 libraries (libabc.so, libdef.so, libghi.so ) directly in /system/lib and my executable (drive_test) which links all the 3 libraries, I have no issue. But, when I push it along with AOSP build I get the following error on execution of the executable (drive_test) -
a. Unable to link ../abc/def.so file.
I am unable to debug how to solve this linker issue. I am not getting whether it is because of CMake (or) any other issue.

If you're trying to make a system daemon (something that goes in /system/bin), don't use the NDK. Just use the AOSP build system.
In mydaemon/Android.bp (make sure to add this path to the root Android.bp file):
cc_library {
name: "libmylib",
srcs: ["mylib.cpp"],
}
cc_binary {
name: "mydaemon",
srcs: ["mydaemon.cpp"],
shared_libs: ["libmylib"],
}
The build system will deal with getting libraries and binaries into the right place for you.

Android CMake: Could NOT find OpenSSL

I'm pretty new to Android with NDK / CMake. However, I'm trying to embed a native CMake library into an Android Application. However, this library depends on OpenSSL.
That's why I downloaded a precompiled version of OpenSSL for Android.
However, when I try to sync the project I get the following error:
Could NOT find OpenSSL, try to set the path to OpenSSL root folder
in the system variable OPENSSL_ROOT_DIR (missing: OPENSSL_LIBRARIES)
(found version "1.1.0f")
Here is my (minimal) project structure
<app-name>
-app
- src
- main
- cpp
- library
- CMakeLists.txt
CMakeLists.txt
- distribution
- openssl
- armeabi
- include
- openssl
...
- lib
libcrypto.a, libssl.a
In my build.gradle I've defined the following:
externalNativeBuild {
cmake {
path 'src/main/cpp/CMakeLists.txt'
}
}
The /app/src/main/cpp/CmakeLists.txt looks as follows:
cmake_minimum_required(VERSION 3.4.1)
set(distribution_DIR ${CMAKE_SOURCE_DIR}/../../../../distribution)
set(OPENSSL_ROOT_DIR ${distribution_DIR}/openssl/${ANDROID_ABI})
set(OPENSSL_LIBRARIES "${OPENSSL_ROOT_DIR}/lib")
set(OPENSSL_INCLUDE_DIR ${OPENSSL_ROOT_DIR}/include)
message("OPENSSL_LIBRARIES ${OPENSSL_LIBRARIES}")
message("OPENSSL_INCLUDE_DIR ${OPENSSL_INCLUDE_DIR}")
find_package(OpenSSL REQUIRED)
add_subdirectory(library)

find_package(...) searches for libraries in a few standard locations (read here - search for "Search paths specified in cmake"). In your case, it fails because it can't find OpenSSL on the machine you're trying to cross-compile the code for Android.
I know I also had various attempts on linking OpenSSL with my native c++ Android code, and the only way I managed it make it work, was the following:
SET(distribution_DIR ${CMAKE_SOURCE_DIR}/../../../../distribution)
SET(OPENSSL_ROOT_DIR ${distribution_DIR}/openssl/${ANDROID_ABI})
SET(OPENSSL_LIBRARIES_DIR "${OPENSSL_ROOT_DIR}/lib")
SET(OPENSSL_INCLUDE_DIR ${OPENSSL_ROOT_DIR}/include)
SET(OPENSSL_LIBRARIES "ssl" "crypto")
#<----Other cmake code/defines here--->
LINK_DIRECTORIES(${OPENSSL_LIBRARIES_DIR})
ADD_LIBRARY(library #your other params here#)
TARGET_INCLUDE_DIRECTORIES(library PUBLIC ${OPENSSL_INCLUDE_DIR})
TARGET_LINK_LIBRARIES(library ${OPENSSL_LIBRARIES})
I know I also tried to make find_package work correctly, by playing with some of it configuration properties, such as CMAKE_FIND_ROOT_PATH, and some other approaches, but I couldn't get it done.
Don't know if the solution I provided is the best approach, cmake-wise. Maybe someone has a better way of doing it, but alas, it solved my problem at the time.
Hope this helps

The reason OpenSSL is not found is because all the find_*() commands also rely on the variable CMAKE_SYSROOT variable, which is used to prefix paths searched by those commands. This is used when cross-compiling to point to the root directory of the target environment.
A solution is to add the path where OpenSSL is located to CMAKE_FIND_ROOT_PATH; the documentation of find_library() states:
The CMake variable CMAKE_FIND_ROOT_PATH specifies one or more
directories to be prepended to all other search directories. This
effectively “re-roots” the entire search under given locations.
This solution works for me:
set(OPENSSL_ROOT_DIR "/path/to/openssl/for/android")
list(APPEND CMAKE_FIND_ROOT_PATH "${OPENSSL_ROOT_DIR}")
find_package(OpenSSL)

For me it worked to import openssl via gradle and then linking the fetched lib like described here

Either append NO_CMAKE_FIND_ROOT_PATH to your find_* command, e.g.
find_package(OpenSSL REQUIRED NO_CMAKE_FIND_ROOT_PATH)
Or set CMAKE_FIND_ROOT_PATH_MODE_* variables (like this) to NEVER. The following options fixed the Android and iOS builds on macOS for me:
-DOPENSSL_USE_STATIC_LIBS=TRUE \
-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=NEVER \
-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=NEVER