Develop Reference

CMake cannot find libraries and package when building using Android toolchain [duplicate]

I build a C++ project depending on the Boost library using CMake (3.4.1). Host platform is Linux, targets are that host and cross-build Android NDK.
I'm only using Boost header files and I just downloaded/extracted the boost folder (and I don't have a /usr/include/boost directory).
In my CMakeLists.txt file I declare the dependency to Boost like this:
find_package(Boost 1.57 REQUIRED)
And I configure my build like this:
BOOST_ROOT=/path/to/boost cmake ../src
Which actually works as expected for my native build.
When I now configure a build exactly the same way (only specifying some more environment variables and a CMAKE_TOOLCHAIN_FILE) CMake gives me:
BOOST_ROOT=/path/to/boost JAVA_HOME=/bla/bla/bla \
ANDROID_NDK=/bla/bla/bla \
ANDROID_SDK=/bla/bla/bla \
ANT=/usr/bin/ant \
cmake ../src -DCMAKE_TOOLCHAIN_FILE=/bla/bla/android.toolchain.cmake
CMake Error at /usr/share/cmake/Modules/FindBoost.cmake:1247 (message):
Unable to find the requested Boost libraries.
Unable to find the Boost header files. Please set BOOST_ROOT to the root
directory containing Boost or BOOST_INCLUDEDIR to the directory containing
Boost's headers.
Call Stack (most recent call first):
CMakeLists.txt:4 (find_package)
So I believe I did almost the same to build for the Android target but the very same method that finds Boost for the host-build doesn't work here.
I tried to set Boost_DIR, BOOSTROOT and BOOST_INCLUDEDIR all with the same effect. Also I've deleted all content in the build directory before trying anything new.
What can be possible reasons for this behavior? I've already tried to print BOOST_ROOT directly in the FindBoost.cmake script like this:
message("BOOST_ROOT: $ENV{BOOST_ROOT}")
With the expected behavior (writing BOOST_ROOT: /path/to/boost).
Of course I can cheat now and just link the boost folder into the include folder of the cross compiler but that's not nice of course and I want to find out what's going on.

When cross-compiling, the toolchain file normally sets the variable CMAKE_FIND_ROOT_PATH. Combined with the CMAKE_FIND_ROOT_PATH_MODE_LIBRARY variable set to ONLY, CMAKE_FIND_ROOT_PATH variable is used as effective chroot for find_library calls, so only libraries under the given prefix(es) are searched.
Analogue variables exist to adjust the behavior for find_path (used for searching include paths) and find_program.
THe toolchain file you use actually sets CMAKE_FIND_ROOT_PATH at line 1521:
set( CMAKE_FIND_ROOT_PATH "${ANDROID_TOOLCHAIN_ROOT}/bin"
"${ANDROID_TOOLCHAIN_ROOT}/${ANDROID_TOOLCHAIN_MACHINE_NAME}"
"${ANDROID_SYSROOT}"
"${CMAKE_INSTALL_PREFIX}"
"${CMAKE_INSTALL_PREFIX}/share" )
and below sets CMAKE_FIND_ROOT_PATH_MODE_* variables to ONLY. So you need to have Boost installed under one of these directories, and give hints (like BOOST_ROOT) relative to it.
Note, that Boost should be built for the target platform (Android NDK in you case), not for the platform where you cross-compile (Linux).

Is there a way to prevent CMake from enforcing the toolchain layout when using CMAKE_ANDROID_STANDALONE_TOOLCHAIN?

I am using CMake 3.8 or newer
I am using a custom toolchain file to configure for Android
Using the NDK is not an option
As the title suggests, I have a custom Android toolchain I'd like to use, rather than the NDK. The problem is that when setting CMAKE_SYSTEM_NAME to Android and specify CMAKE_ANDROID_STANDALONE_TOOLCHAIN=<path-to-my-toolchain>, CMake seems to enforce the layout of said toolchain. It's like the user is not trusted to completely configure their own Android toolchain?
Currently, I am forced to set CMAKE_SYSTEM_NAME to Generic or Linux in my toolchain file to get past the configuration stage, and I do not really like that solution.
Is it really not possible to use a standalone Android toolchain without having CMake enforce the layout (without modifying the CMake Android modules, of course)?
CMake error output
The first thing it complains about is not having a sysroot:
CMake Error at /usr/local/share/cmake-3.8/Modules/Platform/Android-Determine.cmake:74 (message):
Android: The standalone toolchain directory specified by CMAKE_ANDROID_STANDALONE_TOOLCHAIN:
/home/<path-to-toolchain>
does not contain a sysroot with a known layout. The file:
/home/<path-to-toolchain>/sysroot/usr/include/android/api-level.h
does not exist.
If I fix the error above, the next in line is this:
CMake Error at /usr/local/share/cmake-3.8/Modules/Platform/Android/Determine-Compiler-Standalone.cmake:16 (message):
Android: No '*-gcc' compiler found in CMAKE_ANDROID_STANDALONE_TOOLCHAIN:
This error is caused by the fact that CMake expects a bin-folder with a gcc toolchain in the main toolchain folder.

Looking at the CMake implementation, it would seem that it does make assumptions about the toolchain layout. Apart from the requirement stated in the CMake docs that there must be a sysroot subdirectory below the nominated CMAKE_ANDROID_STANDALONE_TOOLCHAIN directory, the implementation assumes there will be matches for ${CMAKE_ANDROID_STANDALONE_TOOLCHAIN}/bin/*-gcc${_ANDROID_HOST_EXT}, where _ANDROID_HOST_EXT will be .exe on Windows and empty otherwise. It then continues on with a bit more logic, but if the toolchain is gcc-based, it should be fine.
Looking at the implementation a bit more deeply, if you set the ANDROID_STANDALONE_TOOLCHAIN environment variable rather than the CMAKE_ANDROID_STANDALONE_TOOLCHAIN CMake variable, it bypasses the sysroot check (more accurately, the check for the api-level.h file at a path under the sysroot). If you do this, then after that, CMake will still try to determine the API level by first looking for <sysroot>/bin/clang and querying it if found, or else looking for the api-level.h file again. If your toolchain doesn't have <sysroot>/bin/clang, then you can set the undocumented and internal CMake variable _ANDROID_STANDALONE_TOOLCHAIN_API to the API to prevent CMake from looking for and requiring api-level.h. You wouldn't normally want to rely on something internal like this, since it could change with any future CMake release, but that would be the price of what you are wanting to do.
If you don't want to rely on internal details, then setting CMAKE_SYSTEM_NAME to Linux and manually setting up the toolchain details like any other non-Android build is likely to be your best option. You will then need to set up all relevant compiler and linker flags, etc. just like any other cross-compiling situation, since it won't trigger any of CMake's automatic handling for Android.

How to add support for custom android compiler in cmake?

Am using a Customized Android Compiler. That compiles cpp code and generates the .so file. From Command Prompt it is working fine but when I try to Build using Cmake then Cmake always picks wither Clang or GNU Compiler for Android compilation even after setting the compiler name in toolchain file.
My Analysis and Findings are as below.
I am working towards creating a toolchain file for an ANDROID compiler
I understand that the flow in CMAKE, when CMAKE_SYSTEM_NAME is specified in Android is different when compared to specifying CMAKE_SYSTEM_NAME as IOS or LINUX
Please validate my current understanding when a toolchain file or command-line option sets CMAKE_SYSTEM_NAME to "Android"
CMakeDetermineSystem.cmake loads this file: Android-Determine.cmake
Next is the platform-specific initialization step: CMakeSystemSpecificInitialize.cmake which loads Android-Initialize.cmake to select the sysroot.
A "determine" step also runs for each language when it is first enabled in a new build tree:
Android-Determine-C.cmake
Android-Determine-CXX.cmake
The language files go here:
Determine-Compiler.cmake
Determine-Compiler-NDK.cmake
The latter file is where we parse a bunch of information from the NDK.
The results persist in CMakeFiles/$v/CMake${lang}Compiler.cmake for future runs.
Next is the language-specific initialization step. For Example – in case {lang} is C : CMakeCInformation.cmake
That loads one of these:
Android-GNU-C.cmake
Android-Clang-C.cmake
which loads one of these:
Android-GNU.cmake
Android-Clang.cmake
Determine-Compiler-NDK.cmake is where cmake looks for versions of clang or gcc or llvm toolchains and sets the appropriate compiler.
Our Requirement and Problem statement
When I tried manually setting a c and cxx compiler from a toolchain file, cmake would not allow me to point to my custom compilers. Ideally, I would want my toolchain file to tell cmake the libraries to include/link and the compilers to use. But once System has been determined as android, cmake goes through the above sequence of events and expects toolchains to be specified for either clang, gcc or llvm compilers.
I believe I would probably need to make changes in the following files to accommodate cmake to use custom compilers.
Determine-Compiler-NDK.cmake - We would need to add support for an "NewCompiler Toolchain" which would contain android libraries and compilers.
New Android-{NewCompiler}-C.cmake, Android-{ NewCompiler }-CXX.cmake, Android-{ NewCompiler }.cmake and another appropriate toolchain file to set target architecture and compiler flags.
Any comments on the process or how to go about this would be appreciated.

Toolchain files can set the path to compilers on the host, but
target-platform-wide information like the set of libraries and
include directories to use typically belongs in platform information
modules (e.g. Platform/Android-... and the like).
Since your compiler doesn't come with the NDK, perhaps you could look
at using the standalone toolchain mode.

How to link against a shared library without inclusion in final package?

I have been working with a project that needs to link against two shared libraries from other applications, without actually including them in the final package. I was using Android Studio with a common way of executing an external ndk-build but I am now using the experimental gradle NDK support.
From what I understand from the information here, using the jniLibs source set will package the binary with the final application, which is not what I want. I would like to link against the libraries during compile time, and expect that my own module is loaded in an environment where they are available.
My old workaround was to add the line: TARGET_NO_UNDEFINED_LDFLAGS := within the Android.mk, but I'm not aware of any way to something equivalent with the new NDK support.
Currently, I am including the jni directory which contains the libraries alongside my source by adding -L"<directory>" to ldFlags, and linking against them as I would with a system library, as below (currently, I am only building for ARMv7):
android.ndk {
moduleName = "ropecraftpe"
ldLibs.addAll(['log', 'name of one library', 'name of other library'])
ldFlags.add('-L"' + file('src/main/jni').absolutePath + '"')
cppFlags.add('-std=c++11')
stl = 'gnustl_shared'
abiFilters.addAll(['armeabi-v7a'])
}
This works for the most part - oddly enough, the linker does not complain about calls to static functions or globals, but does complain when attempting to use a constructor defined in one of the libraries: Error:(12) undefined reference to 'ClassName::ClassName(std::string const&, etc.)'. This class is defined in a header file included from my own C++ code, and the constructor is definitely present in the linked .so.
How does one go about linking this project successfully?

I have made a workaround; my solution was to add the libraries that I wished to link against to the NDK's system library directory.
This is located under ANDROID_NDK/platforms/android-XX/arch-XX/usr/lib/. This appears to work fine, as I wanted the libraries to be treated in the same way as system libraries (not included as prebuilts, but still linked against).
As an addition, one can place the header files under the include directory there.

What is the standalone toolchain?

I am trying to understand what a standalone toolchain means.
Following are my findings.
A toolchain which is ready to use with all the configuration that is the system headers and libraries in the correct path . For Android it will also have the API headers in the path where the toolchain can look it up. Why the term "standalone"? This probably will be different that the regular toolchain in the sense that the R T will need to be configured and made ready for android use withe sysroot and libc header paths given etc.
Please comment .
Hmm, I was compiling for android and one process was running a script called make-standalone-toolchain.sh a standalone toolchain is created .I was going through this script to understand what this is doing. Not really good at shell scripting. But made out certain things. ""Generate a customized Android toolchain installation that includes a working sysroot. The result is something that can more easily be used as a standalone cross-compiler, e.g. to run configure and make scripts." --toolchain arch ndk-dir package-dir system platform variables are set Compute source sysroot
SRC_SYSROOT="$NDK_DIR/platforms/$PLATFORM arch-$ARCH" Copying sysroot headers and libraries... libstdc++ headers and libraries... prebuilt binaries.all into a temporary folder then a copying from Tmp dir to install
dir creating a tar ie a package file to add the tmpdir wanted to know what exactly is happening here or a link or suggestion where to look.but ofcourse dont want to read very elaborate manuals.
~

This blog posting may answer your question:
http://clseto.mysinablog.com/index.php?op=ViewArticle&articleId=3480794
There is a recommended way to build native C/C++ program on Android: List your source files in a script (Android.mk) and run 'ndk-build'. It may not be a problem if you are writing new programs but if you already have a working build script (like those open source softwares) which can build you program nicely on Linux, it would be a headache to migrate your script to Android.mk. All you need in this situation is a C/C++ cross compiler and then replace the variables in your script (such as CC, CXX, AR, AS, RANLIB, ...) to something like 'arm-linux-androideabi-gcc', 'arm-linux-androideabi-g++', ...
Fortunatley, inside the 'Android NDK Dev Guide', there is a section 'Standalone Toolchain' which just describes what we need....

First of all, the best guide for stand alone toolchains in Android is here: https://developer.android.com/ndk/guides/standalone_toolchain.
I have used it several times for different devices and platform.
You need to download NDK and then run the script 'make-standalone-toolchain.sh' with a few parameters (as said in the link above) that will determine the API levels of your apps, the architecture of the device etc.
The output of the script will be a directory that you can use as a toolchain in order to cross compile native C/C++ code to run on Android devices. You need to put in your Makefile the path to the toolchain directory and add the architecture suffix for the binaries inside (for example 'arm-eabi-'). Something like:
CROSS_COMPILE = /path-to-toolchain-dir/bin/arm-eabi-
There should be files like '/path-to-toolchain-dir/bin/arm-eabi-gcc' in your toolchain directory.
Anyway, this will tell the Makefile to use your toolchain's binaries in order to compile the C/C++ native code and create the compatible executables for your target machine.
For example, this is the commands I used to create a stand alone tool chain for a certaion Android device:
./make-standalone-toolchain.shj --arch=arm --platform=android-21 --install-dir=<dest-dir> --toolchain=arm-linux-androideabi-4.9