While building my android project, cmake runs several instances of clang++.exe that eat up my machine's resources. Is there a way to limit this? Would such a solution lead to longer build times?
Here is a screenshot of my resource map: resource status
I have the following package versions:
android studio: 4.0.1
ndk: 21.3.6528147
cmake:3.10.2
Your screenshot indicates you are generating for the Ninja build system... so...
If building through the CMake executable, documented here, you would use:
cmake --build . -- -j 8
which would build with up to 8 CPUs in parallel. (Change 8 to however many instances you'd like.)
If just building using Ninja directly, you would use something like:
ninja -j 8
to achieve the same effect.
Related
I have been trying to build a c++ library for android using cmake cross compilation.
The library itself is a trivial test I made purely for testing the androind build process.
I have been using the cmake gui (v 3.25.0) on a windows machine.
I use a visual studio 2019 generator in cmake and specify a toolchain file.
Unfortunately, I seem to get the same error whatever I try. I tried searching for similar problems and trouble shooting, but have so far been unable to make any progress. If anyone with more experience could lend a hand, I would be very grateful!
I have attempted setting many different variables in the toolchain file in an attempt to resolve the problem, but the basic version of what I am working with is:
set(CMAKE_SYSTEM_NAME Android)
set(CMAKE_ANDROID_ARCH_ABI armeabi-v7a)
set(CMAKE_ANDROID_NDK C:/Microsoft/AndroidNDK64/android-ndk-r16b/)
The error I run into happens whenever I configure cmake. The outut I get from cmake looks something like this:
Android: Targeting API '27' with architecture 'arm', ABI 'armeabi-v7a', and processor 'armv7-a'
Android: Selected Clang toolchain 'arm-linux-androideabi-clang' with GCC toolchain 'arm-linux-androideabi-4.9'
The C compiler identification is Clang 5.0.300080
The CXX compiler identification is Clang 5.0.300080
Detecting C compiler ABI info
Detecting C compiler ABI info - failed
Check for working C compiler: C:/Microsoft/AndroidNDK64/android-ndk-r16b//toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe
Check for working C compiler: C:/Microsoft/AndroidNDK64/android-ndk-r16b//toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe - broken
CMake Error at C:/Program Files/CMake/share/cmake-3.25/Modules/CMakeTestCCompiler.cmake:70 (message):
The C compiler
"C:/Microsoft/AndroidNDK64/android-ndk-r16b//toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: C:/Users/username/Documents/Code Projects/Android Test/build/CMakeFiles/CMakeScratch/TryCompile-zqgcqm
Run Build Command(s):C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/MSBuild/Current/Bin/MSBuild.exe cmTC_b0f1d.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=16.0 /v:m && Microsoft (R) Build Engine version 16.11.2+f32259642 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
ANDROID_HOME=C:\\Microsoft\AndroidSDK\25
ANT_HOME=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Apps\apache-ant-1.9.3
JAVA_HOME=C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot
NDK_ROOT=C:\\Microsoft\AndroidNDK64\android-ndk-r16b
testCCompiler.c
clang.exe : warning : argument unused during compilation: '-mthumb' [-Wunused-command-line-argument] [C:\Users\username\Documents\Code Projects\Android Test\build\CMakeFiles\CMakeScratch\TryCompile-zqgcqm\cmTC_b0f1d.vcxproj]
clang.exe : warning : argument unused during compilation: '-mfpu=vfpv3-d16' [-Wunused-command-line-argument] [C:\Users\username\Documents\Code Projects\Android Test\build\CMakeFiles\CMakeScratch\TryCompile-zqgcqm\cmTC_b0f1d.vcxproj]
clang.exe : warning : argument unused during compilation: '-mfloat-abi=softfp' [-Wunused-command-line-argument] [C:\Users\username\Documents\Code Projects\Android Test\build\CMakeFiles\CMakeScratch\TryCompile-zqgcqm\cmTC_b0f1d.vcxproj]
CLANGCOMPILE : error : unknown target CPU 'armv7-a' [C:\Users\username\Documents\Code Projects\Android Test\build\CMakeFiles\CMakeScratch\TryCompile-zqgcqm\cmTC_b0f1d.vcxproj]
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:4 (project)
Configuring incomplete, errors occurred!
I initially used used the NDK version found in the android tools for visual studio found at: "C:/Microsoft/AndroidNDK64/android-ndk-r16b/"
as an NDK version.
I have also tried directly downloading the latest version of the NDK and pointing to that with CMAKE_ANDROID_NDK in the toolchain file, though this did not seem to make any change.
I have tried directly using the toolchain.cmake files found at "NDK/build/cmake/android.toolchain.cmake"
for both the NDK versions described above. This too gave the same error.
Any ideas what could be causing this or how to fix?
Ok, so I was misunderstanding a few things about cross-compiling to android.
First, if I want to be able to use the static library (e.g in android studio), I will need to generate and build a solution for more than one android ABI.
The ABI's to build for are: armeabi-v7a, arm64-v8a, x86_64, x86. I then need to check for each android ABI in the CMakeLists.txt file of Android studio (e.g. elseif(ANDROID_ABI STREQUAL arm64-v8a)) and swap out the location of the library to import to the build folder of that particular ABI.
As far as I am aware, it is unfortunately not possible to generate for all these ABI's as seperate platforms within the same visual studio solution. Given that, cmake must be run 4 seperate times to different build folders, each time targeting a different ABI and generator platform. To save the hassle of doing this manually, creating a batch (.bat) script to automate the process seems the best way to go. This will involve using cmake from the command line instead of the GUI version. The script can also be made to build the generated projects to save opening each one in visual studio and building there.
The batch script I have made to generate and build cross-compilation projects for android is as follows:
#echo OFF
set BUILD_DIR=build
set ANDROID_NDK=C:\Microsoft\AndroidNDK\android-ndk-r23c
set GENERATOR="Visual Studio 17 2022"
set CMAKE_GENERATOR=-G %GENERATOR%
set CMAKE_TOOLCHAIN_FILE=-DCMAKE_TOOLCHAIN_FILE=%ANDROID_NDK%\build\cmake\android.toolchain.cmake
set CMAKE_SYSTEM_NAME=-DCMAKE_SYSTEM_NAME=Android
set EXTRA_CMAKE_ARGS=-DBUILD_SHARED_LIBS=true -DANDROID_TOOLCHAIN=clang -DANDROID_STL=c++_static
CALL :build_android armeabi-v7a ARM 16
CALL :build_android arm64-v8a ARM64 21
CALL :build_android x86_64 x64 21
CALL :build_android x86 x86 16
EXIT /B %ERRORLEVEL%
:build_android
set ABI_VERSION=%~1
set GENERATOR_PLATFORM=%~2
set MINIMUM_API_LEVEL=%~3
set CMAKE_ANDROID_ARCH_ABI=-DANDROID_ABI=%ABI_VERSION%
set ABI_BUILD_DIR=%BUILD_DIR%\%ABI_VERSION%
set CMAKE_GENERATOR_PLATFORM=-A %GENERATOR_PLATFORM%
set CMAKE_BUILD_DIR=-B %ABI_BUILD_DIR%
set CMAKE_MIN_API=-DANDROID_PLATFORM=android-%MINIMUM_API_LEVEL%
set CMAKE_ARGS=%CMAKE_BUILD_DIR% %CMAKE_ANDROID_ARCH_ABI% %CMAKE_GENERATOR% %CMAKE_GENERATOR_PLATFORM% %CMAKE_SYSTEM_NAME% %CMAKE_TOOLCHAIN_FILE% %CMAKE_MIN_API%
echo building for android ABI: %ABI_VERSION%
echo cmake arguments = %CMAKE_ARGS%
echo:
cmake %CMAKE_ARGS% %EXTRA_CMAKE_ARGS%
echo:
cmake --build %ABI_BUILD_DIR% --target ALL_BUILD
echo:
echo:
EXIT /B 0
This worked for my simple test library, but I guess depending on the project the arguments to the cmake commands may need to be tweaked.
I am working on a native Android project using NDK r12b. This NDK (and it looks like all others since) ships with a prebuilt GNU linker version 2.25. We have recently tried to pull up lots of our project's submodules which has introduced a build error that looks like it is caused by this bug in ld.
build error output:
/opt/android-ndk-r12b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: internal error in do_relocate_sections, at /usr/local/google/buildbot/src/android/gcc/toolchain/build/../binutils/binutils-2.25/gold/reloc.cc:953
collect2: error: ld returned 1 exit status
I thought I would attempt to update the linker but have had no luck finding prebuilt binaries. So I went down the rabbit hole of trying to compile ld/binutils myself. Either my Google skills are failing me, or documentation for doing this is truly rare or presumes the user has a lot of introductory knowledge that I don't have.
When building binutils 2.29.1 with target=arm-linux my application build errors out with this result:
/opt/android-ndk-r12b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: unrecognised emulation mode: armelf_linux_eabi
Supported emulations: armelf_linux armelf armelfb armelfb_linux
collect2: error: ld returned 1 exit status
For reference, the -V output of our current version of ld is:
me#linux-vm:/opt/android-ndk-r12b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin$ ./ld-2.25 -V
GNU gold (binutils-2.25-0666073 2.25.51.20141117) 1.11
Supported targets:
elf64-littleaarch64
elf64-bigaarch64
elf32-littleaarch64
elf32-bigaarch64
elf64-tradlittlemips
elf32-tradlittlemips-nacl
elf64-tradbigmips
elf32-tradlittlemips-nacl
elf32-tradlittlemips
elf32-tradlittlemips-nacl
elf32-tradbigmips
elf32-tradlittlemips-nacl
elf32-tilegx-be
elf64-tilegx-be
elf32-tilegx-le
elf64-tilegx-le
elf32-bigarm
elf32-bigarm-nacl
elf32-littlearm
elf32-littlearm-nacl
elf64-powerpcle
elf64-powerpc
elf32-powerpcle
elf32-powerpc
elf64-sparc
elf32-sparc
elf32-x86-64
elf32-x86-64-freebsd
elf32-x86-64-nacl
elf64-x86-64
elf64-x86-64-freebsd
elf64-x86-64-nacl
elf32-i386
elf32-i386-freebsd
elf32-i386-nacl
Supported emulations:
aarch64_elf64_le_vec
aarch64_elf64_be_vec
aarch64_elf32_le_vec
aarch64_elf32_be_vec
elf64-tradlittlemips
elf32-tradlittlemips-nacl
elf64-tradbigmips
elf32-tradlittlemips-nacl
elf32-tradlittlemips
elf32-tradlittlemips-nacl
elf32-tradbigmips
elf32-tradlittlemips-nacl
elf32tilegx_be
elf64tilegx_be
elf32tilegx
elf64tilegx
armelfb
armelfb_nacl
armelf
armelf_nacl
elf64lppc
elf64ppc
elf32lppc
elf32ppc
elf64_sparc
elf32_sparc
elf32_x86_64
elf32_x86_64_nacl
elf_x86_64
elf_x86_64_nacl
elf_i386
elf_i386_nacl
So apparently I'm missing some important configuration parameters. I also tried using an older (r8e) Android NDK's build/tools/build-gcc.sh script to build the entire compiler toolchain but with a newer binutils version. This resulted in an unknown build error:
me#linux-vm:/opt/android-ndk-r8e/build/tools$ ./build-gcc.sh --gmp-version=5.0.5 --mpfr-version=3.1.1 --mpc-version=1.0.1 --binutils-version=2.26
$(pwd)/src $(pwd) arm-linux-androideabi-4.7
To follow build in another terminal, please use: tail -F /tmp/ndk-me/build/toolchain/config.log
Using C compiler: gcc -m32
Using C++ compiler: g++ -m32
Sysroot : Copying: /opt/android-ndk-r8e/platforms/android-9/arch-arm --> /tmp/ndk-me/build/toolchain/prefix/sysroot
Configure: arm-linux-androideabi-4.7 toolchain build
Building : arm-linux-androideabi-4.7 toolchain [this can take a long time].
Error while building toolchain. See /tmp/ndk-me/build/toolchain/config.log
Last entries of config.log:
ar cru libintl.a bindtextdom.o dcgettext.o dgettext.o gettext.o finddomain.o loadmsgcat.o localealias.o textdomain.o l10nflist.o explodename.o dcigettext.o dcngettext.o dngettext.o ngettext.o plural.o plural-exp.o localcharset.o relocatable.o localename.o log.o osdep.o intl-compat.o
ranlib libintl.a
make[1]: Leaving directory `/tmp/ndk-me/build/toolchain/libbfd-binutils-2.26/intl'
At this point I'm just spinning my tires and trying to brute force success with different combinations of configuration parameters and source package versions. It seems as simple as adding emulation support for armelf_linux_eabi, but like I said, I have failed to find any documentation or guides that show how to do this. Surely there is a GNU wizard out there somewhere who could point me down a better path? Any help is appreciated!
I would try --enable-targets=all first. It is a bit of a big hammer, but maybe it helps you to avoid figuring out the exact target triplets you need (my guess would be arm-unknown-linux-eabi).
I'm passing in preprocessor directives via CMakeLists.txt for the build of a native android library using android NDK.
add_definitions(-DMY_DIRECTIVE=1)
It would be great to double check that those preprocessor directives are actually finding their way into the calls to the compiler (llvm ?)
But the gradle build output doesn't seem to include the calls to the compiler, I just get:
Building C object CMakeFiles/my_project.dir/home/me/projects/my_proj/src/my_native.c.o
Is there a means to make the gradle output more verbose such that I can see the actual compiler calls and check those preprocessor directives are present?
The answer is to understand that Gradle utilises CMake to build the android NDK component (shared library) of an android project, and CMake utilises Ninja as a build system to handle the calls to the compiler. The compiler used by android NDK now defaults to LLVM->Clang.
So in order to actually see the Clang calls you have to find the build.ninja files for each target of your android project.
In my case I am only building for an armeabi-v7a target architecture. Therefore the relevant build.ninja files are found in:
/home/me/projects/my_proj/app/.externalNativeBuild/cmake/debug/armeabi-v7a
/home/me/projects/my_proj/app/.externalNativeBuild/cmake/release/armeabi-v7a
cd to either directory and run:
ninja -v
i.e. the -v option is the key to see all the calls to the clang compiler that the native build (android NDK) part of your android project generates.
Note if you have installed CMake via the android package manager, you may find that ninja is not installed in a location that is on your PATH. For me the ninja binary is located as follows:
~/Android/Sdk/cmake/3.6.3155560/bin/ninja
(same directory as cmake binary)
Therefore for me to see all the clang compiler calls for my android project's debug armeabi-v7a build I have to run:
cd /home/me/projects/my_proj/app/.externalNativeBuild/cmake/debug/armeabi-v7a
~/Android/Sdk/cmake/3.6.3155560/bin/ninja -v
Note if ninja tells you ninja: no work to do.
Then run:
~/Android/Sdk/cmake/3.6.3155560/bin/ninja clean
Relevant ninja documentation is -> https://ninja-build.org/manual.html#_extra_tools
I have a really simple helloworld.cpp program
#include <iostream>
using namespace std;
int main ()
{
cout << "Hello World!";
return 0;
}
And I'm trying to compile it for android x86 with the cross-compiler from the toolchain:
/Users/me/android-ndk-r8/toolchains/x86-4.4.3/prebuilt/darwin-x86/bin/i686-android-linux-g++ helloworld.cpp -L "/Users/me/android-ndk-r8/sources/cxx-stl/stlport/libs/x86/" -lstlport_static
However, I get errors:
helloworld.cpp:2:20: error: iostream: No such file or directory
Any idea why?
Check the documentation.html file included with the NDK, under "Standalone Toolchain". It says that if you invoke the compiler in this way you won't be able to "use any C++ STL". However it is possible, as the documentation explains, if you first create a "customized" toolchain installation, using something like the following command:
$NDK/build/tools/make-standalone-toolchain.sh --platform=android-8 --install-dir=/tmp/my-android-toolchain --arch=x86
where $NDK is the path to your NDK directory. Note the --arch=x86 which means that the toolchain is prepared specifically for the x86 Android. This prepares what you need in one directory, including the STL headers and folders. You should then be able to use -lstdc++ to link against the STL (static version), i.e. something like:
/tmp/my-android-toolchain/bin/i686-android-linux-g++ helloworld.cpp -lstdc++
For a more complete explanation, please see the NDK documentation.
The NDK documentation is not entirely accurate, at least not currently. In fact, it states when using the prebuilt toolchain "You won't be able to use any C++ STL (either STLport or the GNU libstdc++) with it.", but this is out of date. I created a small hello world program using the include with the same error. It can be solved without creating your own toolchain though, which is nice if you don't want to have to add one more step to your configuration process and allows you to always use the latest SDK platform without creating a new toolchain every time.
The NDK ships with the source code for several versions of standard C++ libraries: GAbi++, STLport, and GNU STL. Each flavor comes with prebuilt shared and static libs as well. My example below will use stlport.
To use the stand-alone toolchain at its installed location, you can do something like this:
export CXX='$NDK_ROOT/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-g++ --sysroot="$NDK_ROOT/platforms/android-19/arch-arm"'
This, for example, would set your CXX compiler to compile ARM on the OS X system using SDK platform level 19. This much you probably already knew. Also, you would want to export your CC, CPP, LD, AR, and RANLIB if you use it. I also personally create an envar for READELF.
To add support for C++ libs, you could do something like follows:
$CXX helloworld.cpp -I$NDK_ROOT/sources/cxx-stl/stlport/stlport -L$NDK_ROOT/sources/cxx-stl/stlport/libs/armeabi -lstlport_shared
Note that this will link the libstlport_shared.so which will now be required at runtime, so you may need to add a relative path to the command above to support that, depending on your APK structure. If you want to just test this simple case, you can just run this on the emulator as follows:
adb push a.out /data
adb push $NDK_ROOT/sources/cxx-stl/stlport/libs/armeabi/libstlport_shared.so /data
adb shell
# su
# cd /data
# chmod 777 a.out
# ./a.out
To get rid of the headache of dealing with shared library paths, you can also statically link the C++ library in by changing "-lstlport_shared" to "-lstlport_static". There are some consequences of doing this, as explained in the NDK Dev Guide. The biggest issue is due to having the static library linked in multiple places, causing:
- memory allocated in one library, and freed in the other would leak or even corrupt the heap.
- exceptions raised in libfoo.so cannot be caught in libbar.so (and may simply crash the program).
- the buffering of std::cout not working properly
A useful tool is also included to see what dependencies your program has, the readelf tool.
To see what other shared libraries your program requires, you can run the following:
$NDK_ROOT/toolchains/arm-linux-androideabi-4.8/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-readelf -a a.out | grep NEEDED
Other cool standard tools:
addr2line - convert a stack trace address to a line of code
nm - Displays the symbol table
objdump - displays info on an object
i call one of the functions from gnustl after it runs fine from prebuilt aosp arm-linux-androideabi-gcc --std=c++11 and after crashing error i cant get a backtrace from logs or reason, my hope is turning to crossbuilt qemu-linux-user to run host compiled i686 binary on the arm, difficulty is interacting with crosshost libs aapt from adt always crashes on any other than platform specific libs, unless kernel module packaged update is possible...
I wand build project in windows, I get same Errors:
bash C:\linux\android-ndk1\ndk-build V=1
cygwin warning:
MS-DOS style path detected: C:\dev\android\workspace4_android\FFWall
Preferred POSIX equivalent is: /cygdrive/c/dev/android/workspace4_android/FFWall
CYGWIN environment variable option "nodosfilewarning" turns off this warning.
Consult the user's guide for more details about POSIX paths:
http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
Android NDK: ERROR:jni/Android.mk:bambuser-libavcore: LOCAL_SRC_FILES points to a missing file
/cygdrive/c/linux/android-ndk1/build/core/prebuilt-library.mk:43: *** Android NDK: Aborting . Stop.
Android NDK: Check that jni/ffmpeg-android/build/ffmpeg/armeabi/lib/libavcore.so exists or that its path is correct
Another way building:
C:\art\android-ndk\ndk-build V=1
Cannot run program "C:\art\android-ndk\ndk-build": Launching failed
Error: Program "C:/art/android-ndk/ndk-build" is not found in PATH
PATH=[C:\Art\android-ndk;C:/Program Files/Java/jre7/bin/client;C:/Program Files/Java/jre7/bin;C:/Program Files/Java/jre7/lib/i386;C:\Program Files\PC Connectivity Solution\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\MATLAB\R2011a\runtime\win32;C:\Program Files\MATLAB\R2011a\bin;C:\Program Files\MATLAB\R2007b\bin;C:\Program Files\MATLAB\R2007b\bin\win3;C:\dev\android\android-sdk-windows\tools;C:\cygwin\bin;C:/art/android-ndk/;C:\linux\android-ndk-r7b;C:\dev\android\eclipse]
Why this happened?
From: http://developer.android.com/sdk/ndk/overview.html#reqs
Required development tools
For all development platforms, GNU Make 3.81 or later is required. Earlier versions of GNU Make might work but have not been tested.
A recent version of awk (either GNU Awk or Nawk) is also required.
For Windows, Cygwin 1.7 or higher is required. The NDK will not work with Cygwin 1.5 installations.
Meaning, that you will have to install Cygwin. After that you can call the ndk-build like:
$ProjectPath> /cygdrive/c/myNdkPath/ndk-build
Please also check the "Getting Started" part of:
http://developer.android.com/sdk/ndk/index.html