I am trying to compile a program using Qt and OpenCV for arm64-v8a
I created a kit to use an arm64-v8a version of Qt, but I get tons of undefined reference errors
I have compiled Qt using the following parameters :
./configure -xplatform android-clang \
--disable-rpath -nomake tests -nomake examples \
-android-ndk <ndk-r16b-root> -android-sdk <sdk-root> -android-ndk-host linux-x86_64 \
-android-toolchain-version 4.9 -android-ndk-platform android-21 -android-arch arm64-v8a \
-skip qttranslations -skip qtserialport -skip qtwebengine -no-warnings-are-errors \
-prefix <install-dir> -opensource -confirm-license
Then I added 2 compilers, the first for C and the second for C++ :
<ndk-r16b-root>/toolchains/llvm/prebuilt/linux-x86_64/bin/clang
with ABI : <custom> arm-linux-android-elf-64-bit
<ndk-r16b-root>/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++
with ABI : <custom> arm-linux-android-elf-64-bit
I cloned the auto-detected kit for this version of Qt and used the following settings :
Device type : Android Device
Device : Run on Android (default for Android)
Sysroot : <ndk-r16b-root>/platforms/android-21/arch-arm64
Compilers : the 2 above
Android GDB server : <ndk-r16b-root>/prebuilt/android-arm64/gdbserver/gdbserver
Qt version : the one I just compiled
Qt mkspec : android-clang
Other fields are left empty/by default
My .pro file looks like this :
QT += quick multimedia multimediawidgets
CONFIG += c++11
# The following define makes your compiler emit warnings if you use
# any feature of Qt which as been marked deprecated (the exact warnings
# depend on your compiler). Please consult the documentation of the
# deprecated API in order to know how to port your code away from it.
DEFINES += QT_DEPRECATED_WARNINGS
# You can also make your code fail to compile if you use deprecated APIs.
# In order to do so, uncomment the following line.
# You can also select to disable deprecated APIs only up to a certain version of Qt.
#DEFINES += QT_DISABLE_DEPRECATED_BEFORE=0x060000 # disables all the APIs deprecated before Qt 6.0.0
HEADERS += <my-headers>
SOURCES += <my-sources>
RESOURCES += qml/qml.qrc
INCLUDEPATH += OpenCV-android-sdk/sdk/native/jni/include
LIBS += -L"OpenCV-android-sdk/sdk/native/3rdparty/libs/arm64-v8a"\
-L"OpenCV-android-sdk/sdk/native/libs/arm64-v8a"\
-llibtiff\
-llibjpeg\
-llibjasper\
-llibpng\
-lIlmImf\
-llibwebp\
-lopencv_core\
-ltbb\
-lopencv_flann\
-lopencv_imgproc\
-lopencv_highgui\
-lopencv_features2d\
-lopencv_calib3d\
-lopencv_ml\
-lopencv_objdetect\
-lopencv_video\
-lopencv_photo\
-lopencv_videostab\
-ltegra_hal\
-lopencv_imgcodecs\
-lopencv_shape\
-lopencv_stitching\
-lopencv_superres\
-lopencv_videoio
# Additional import path used to resolve QML modules in Qt Creator's code model
QML_IMPORT_PATH =
# Additional import path used to resolve QML modules just for Qt Quick Designer
QML_DESIGNER_IMPORT_PATH =
# Default rules for deployment.
qnx: target.path = /tmp/$${TARGET}/bin
else: unix:!android: target.path = /opt/$${TARGET}/bin
!isEmpty(target.path): INSTALLS += target
DISTFILES += \
android/AndroidManifest.xml \
android/gradle/wrapper/gradle-wrapper.jar \
android/gradlew \
android/res/values/libs.xml \
android/build.gradle \
android/gradle/wrapper/gradle-wrapper.properties \
android/gradlew.bat
ANDROID_PACKAGE_SOURCE_DIR = $$PWD/android
The OpenCV-android-sdk folder comes from here using 3.2.0 Android pack.
When I try to compile, I get tons of undefined references like the followings :
OpenCV-android-sdk/sdk/native/libs/arm64-v8a/libopencv_core.a(matrix.cpp.o): In function `cv::SparseMat::newNode(int const*, unsigned long)':
matrix.cpp:(.text._ZN2cv9SparseMat7newNodeEPKim+0x4b8): undefined reference to `std::__throw_length_error(char const*)'
OpenCV-android-sdk/sdk/native/libs/arm64-v8a/libopencv_core.a(matrix.cpp.o): In function `std::vector<cv::Mat, std::allocator<cv::Mat> >::_M_fill_insert(__gnu_cxx::__normal_iterator<cv::Mat*, std::vector<cv::Mat, std::allocator<cv::Mat> > >, unsigned long, cv::Mat const&)':
matrix.cpp:(.text._ZNSt6vectorIN2cv3MatESaIS1_EE14_M_fill_insertEN9__gnu_cxx17__normal_iteratorIPS1_S3_EEmRKS1_[_ZNSt6vectorIN2cv3MatESaIS1_EE14_M_fill_insertEN9__gnu_cxx17__normal_iteratorIPS1_S3_EEmRKS1_]+0xcd4): undefined reference to `std::__throw_length_error(char const*)'
OpenCV-android-sdk/sdk/native/libs/arm64-v8a/libopencv_core.a(matrix.cpp.o): In function `std::vector<cv::UMat, std::allocator<cv::UMat> >::_M_fill_insert(__gnu_cxx::__normal_iterator<cv::UMat*, std::vector<cv::UMat, std::allocator<cv::UMat> > >, unsigned long, cv::UMat const&)':
matrix.cpp:(.text._ZNSt6vectorIN2cv4UMatESaIS1_EE14_M_fill_insertEN9__gnu_cxx17__normal_iteratorIPS1_S3_EEmRKS1_[_ZNSt6vectorIN2cv4UMatESaIS1_EE14_M_fill_insertEN9__gnu_cxx17__normal_iteratorIPS1_S3_EEmRKS1_]+0xbb0): undefined reference to `std::__throw_length_error(char const*)'
The actual linker command used (where errors happen) is :
<ndk-r16b-root>/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ -D__ANDROID_API__=26 -target aarch64-none-linux-android -gcc-toolchain <ndk-r16b-root>/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64 -Wl,--exclude-libs,libgcc.a --sysroot=<ndk-r16b-root>/platforms/android-26/arch-arm64/ -Wl,-soname,libTest.so -Wl,-rpath-link=<newly-compiled-Qt-root>/lib -Wl,--no-undefined -Wl,-z,noexecstack -shared -o libTest.so myfilter.o qml_main_qml.o qmlcache_loader.o -L<ndk-r16b-root>/sources/cxx-stl/llvm-libc++/libs/arm64-v8a -LOpenCV-android-sdk/sdk/native/3rdparty/libs/arm64-v8a -LOpenCV-android-sdk/sdk/native/libs/arm64-v8a -llibtiff -llibjpeg -llibjasper -llibpng -lIlmImf -llibwebp -lopencv_core -ltbb -lopencv_flann -lopencv_imgproc -lopencv_highgui -lopencv_features2d -lopencv_calib3d -lopencv_ml -lopencv_objdetect -lopencv_video -lopencv_photo -lopencv_videostab -ltegra_hal -lopencv_imgcodecs -lopencv_shape -lopencv_stitching -lopencv_superres -lopencv_videoio -L<newly-compiled-Qt-root>/lib -lQt5Quick -lQt5MultimediaWidgets -lQt5Multimedia -lQt5Widgets -lQt5Gui -lQt5Qml -lQt5Network -lQt5Core -lGLESv2 -lc++ -llog -lz -lm -ldl -lc
Edit: tried to compile OpenCV myself, does not work better.
Edit2: I just saw Petesh's answer there: https://stackoverflow.com/a/12920944/9987834
Could this be because my version of Qt and OpenCV have been compiled using different standard libraries?
Edit3: Cannot manage to compile OpenCV correctly, I get errors like this one:
skipping incompatible /usr/local/lib/libopencv_core.a when searching for -lopencv_core
Edit4: For some reason, OpenCV stopped compiling at all, giving me errors like these:
In file included from /path/to/opencv-3.2.0/modules/core/src/algorithm.cpp:43:
In file included from /path/to/opencv-3.2.0/modules/core/src/precomp.hpp:84:
In file included from /path/to/opencv-3.2.0/modules/core/include/opencv2/core/hal/intrin.hpp:297:
/path/to/opencv-3.2.0/modules/core/include/opencv2/core/hal/intrin_neon.hpp:285:49: error: expected
unqualified-id
template <typename T> static inline float16x4_t vld1_f16(const T* ptr)
^
<ndk-r16b-root>/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/5.0.300080/include/arm_neon.h:7745:24: note:
expanded from macro 'vld1_f16'
#define vld1_f16(__p0) __extension__ ({ \
^
[ 31%] Built target libwebp
[ 31%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/command_line_parser.cpp.o
In file included from /path/to/opencv-3.2.0/modules/core/src/algorithm.cpp:43:
In file included from /path/to/opencv-3.2.0/modules/core/src/precomp.hpp:84:
In file included from /path/to/opencv-3.2.0/modules/core/include/opencv2/core/hal/intrin.hpp:297:
/path/to/opencv-3.2.0/modules/core/include/opencv2/core/hal/intrin_neon.hpp:778:8: error: unknown type
name 'v_float16x4'; did you mean 'v_float32x4'?
inline v_float16x4 v_load_f16(const short* ptr)
^~~~~~~~~~~
v_float32x4
My cake (gui) output is here : https://pastebin.com/ZBrtizUi
Does anyone have an idea why?
Even if it is a late reply, my comment might help another user updating a deprecated Android project...
Here is my typed CMake command (configured variables are detailled in the linked logfiles):
cmake ${OPENCV_CMAKEFILE} -DCMAKE_TOOLCHAIN_FILE=${ANDROID_TOOLCHAIN_FILE} -DANDROID_NDK=${ANDROID_NDK_PATH} -DANDROID_NATIVE_API_LEVEL=android-25 -DBUILD_JAVA=OFF -DBUILD_ANDROID_EXAMPLES=ON -DBUILD_ANDROID_PROJECTS=ON-DANDROID_STL=c++_static -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX:PATH=${OPENCV_INSTALL_PATH} -DANDROID_ABI=arm64-v8a -DCMAKE_BUILD_TYPE=Release
1. With your settings, when FP16 support is enabled using OpenCV 4.5.3, CMake logs indicates that:
the test 'HAVE_CPU_NEON_SUPPORT' is performed and validated (executing the check file 'cmake/checks/cpu_neon.cpp')
the test 'HAVE_CPU_FP16_SUPPORT' is performed and validated (executing the check file 'cmake/checks/cpu_fp16.cpp')
and the resulting identified CPU/HW features baseline is identified as "NEON FP16" (see the linked logfile).
Finally, the libraries and example applications are successfully built.
2. With your settings, when FP16 support is enabled using OpenCV 3.2.0, CMake logs indicates that:
the test 'CXX_HAS_MFPU_NEON' is performed and validated
the test 'C_HAS_MFPU_NEON' is performed and validated
the check file 'cmake/checks/fp16.cpp' is ignored.
Finally, the libraries and example applications are not successfully built (with the same errors you described). I linked typed commands and both CMake configuration logs and compilation error logs.
I finally disabled FP16 support and rebuilt OpenCV 3.2.0 successfully. Prebuilts are accessible here.
From my investigation, it appears that some progress on FP16 compiler support for OpenCV has been integrated (reviewed and merged) following the release 3.2.0. This merge from June 21 is the latest work on this matter, a comment from #alalek introduces some previous pull requests.
Related
I have created a shared library using Qt Creator and I have added the Android SDK, Android NDK and Android Qt kit. Then I compiled my library for Android successfully. I even tested it in an Android application successfully.
As I am not using the Qt libraries, my library does not depend on huge Qt libraries. But unexpectedly, here is my dependencies:
[matin#Lenovo-X1-Fedora ~]$ ndk-depends libMatinChess.so
WARNING: Could not find library: libgnustl_shared.so
libMatinChess.so
libz.so
libstdc++.so
libm.so
liblog.so
libgnustl_shared.so
libdl.so
libc.so
And when I checked the libgnustl_shared.so it has more than 5 MBs size. So I have to place this huge library next to my tiny library in every project.
Another option is to link it statically. I previously asked the question about how is it possible to link a dependency statically and I figured out that it is possible by adding the QMAKE_LFLAGS += -static in my .pro file:
This flags works perfect and removes the dependency of stdc++ on Windows compilation. But in android I get the following errors:
error: cannot find -lgnustl_shared
error: cannot find -llog
error: cannot find -lz
error: cannot find -ldl
I searched my android-ndk folder and I realized that there is no liblog.a, libz.a and libdl.a files located in it but there is a libgnustl_static.a file.
I tried to add it using LIBS += -Lpath/to/libdir -lgnustl_static but the result was the same.
There is a solution in CMake that was mentioned in the previous question as a comment that there is the option to set APP_STL := gnustl_static in the makefile. But there seems to be no equivalent in QMake.
And a complicated issue is that when I use CONFIG += static, it compiles successfully but my library is not shared anymore. it becomes a static library.
How can I link gnustl statically so that my library works with no other dependencies?
Edit
I read the compile output and found the following line:
/home/matin/Applications/android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++
--sysroot=/home/matin/Applications/android-ndk-r13b/platforms/android-9/arch-arm/
-static -Wl,--no-undefined -Wl,-z,noexecstack -shared -Wl,-soname,libMatinChess.so -o libMatinChess.so matinchessdll.o bishop.o piece.o board.o king.o memorymanager.o pawn.o queen.o
blackpawn.o knight.o rook.o whitepawn.o squarelist.o game.o
boardhistory.o
-L/home/matin/Applications/android-ndk-r13b/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a
-L/home/matin/Applications/android-ndk-r13b/platforms/android-9/arch-arm//usr/lib
-lgnustl_shared -llog -lz -lm -ldl -lc -lgcc
And I was not able to remove gnustl_shared using LIB -= -lgnustl_shared
By reading the compile output, I executed the following command manually and created my library with 1 MB size. And it works correctly.
/home/matin/Applications/android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++
--sysroot=/home/matin/Applications/android-ndk-r13b/platforms/android-9/arch-arm/
-Wl,--no-undefined -Wl,-z,noexecstack -shared -Wl,-soname,libMatinChess.so -o libMatinChess.so matinchessdll.o bishop.o piece.o board.o king.o memorymanager.o pawn.o queen.o
blackpawn.o knight.o rook.o whitepawn.o squarelist.o game.o
boardhistory.o
-L/home/matin/Applications/android-ndk-r13b/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a
-L/home/matin/Applications/android-ndk-r13b/platforms/android-9/arch-arm//usr/lib
-lgnustl_static
But still I don't know how to automate this command in QMake
Below script android.pri did help for me which I include in my projects to get rid of some QtCreator related bugs:
## this file changes many values set by "Qt/mkspec/android-g++/qmake.conf"
## since the failed to work with newest Android SDK and/or NDK
##
## use below lines before including this file
##
#ANDROID_API = 21
#ANDROID_ARCH = armeabi-v7a
#do you after using this have still problems?
#there is an bug inside "QtCreator" (not inside "Gradle")
# that leads to .o and .so files get not found
#solved: not required to change "QMAKE_LIBDIR" just open "AndroidManifest.xml"
# and change "Minimum required SDK" to "not set" save then restore to last
# value and save agian to force "QtCreator" update
isEmpty(ANDROID_API): ANDROID_API = 21
isEmpty(ANDROID_ARCH): ANDROID_ARCH = armeabi-v7a #ANDROID_TARGET_ARCH=armeabi-v7a
#remove old values
CONFIG -= $$ANDROID_PLATFORM
QMAKE_CFLAGS -= --sysroot=$$ANDROID_PLATFORM_ROOT_PATH
QMAKE_CXXFLAGS -= --sysroot=$$ANDROID_PLATFORM_ROOT_PATH
QMAKE_LFLAGS -= --sysroot=$$ANDROID_PLATFORM_ROOT_PATH
#NDK Root directory
ANDROID_NDK_ROOT = $$(ANDROID_NDK_ROOT) #first try Environment variable
isEmpty(ANDROID_NDK_ROOT) | !exists($$ANDROID_NDK_ROOT) {
ANDROID_NDK_ROOT = D:/android/sdk/ndk-bundle }
NDK_ROOT = $$ANDROID_NDK_ROOT
#API Level
ANDROID_NDK_PLATFORM = android-$$ANDROID_API
ANDROID_PLATFORM = $$ANDROID_NDK_PLATFORM
CONFIG += $$ANDROID_PLATFORM
DEFINES += __ANDROID_API__=$$ANDROID_API
#Architecture
ANDROID_TARGET_ARCH = $$ANDROID_ARCH
equals(ANDROID_TARGET_ARCH, x86): ANDROID_ARCHITECTURE = x86
else: equals(ANDROID_TARGET_ARCH, x86_64): ANDROID_ARCHITECTURE = x86_64
else: equals(ANDROID_TARGET_ARCH, mips): ANDROID_ARCHITECTURE = mips
else: equals(ANDROID_TARGET_ARCH, mips64): ANDROID_ARCHITECTURE = mips64
else: equals(ANDROID_TARGET_ARCH, arm64-v8a): ANDROID_ARCHITECTURE = arm64
else: ANDROID_ARCHITECTURE = arm
#API Path
ANDROID_PLATFORM_ROOT_PATH = $$NDK_ROOT/platforms/$$ANDROID_PLATFORM/arch-$$ANDROID_ARCHITECTURE/
ANDROID_PLATFORM_PATH = $$ANDROID_PLATFORM_ROOT_PATH/usr
QMAKE_CFLAGS += --sysroot=$$ANDROID_PLATFORM_ROOT_PATH
QMAKE_CXXFLAGS += --sysroot=$$ANDROID_PLATFORM_ROOT_PATH
QMAKE_LFLAGS += --sysroot=$$ANDROID_PLATFORM_ROOT_PATH
# used to compile platform plugins for android-4 and android-5
QMAKE_ANDROID_PLATFORM_INCDIR = $$NDK_ROOT/sysroot/usr/include #headers bundled
QMAKE_ANDROID_PLATFORM_LIBDIR = $$ANDROID_PLATFORM_PATH/lib #same as before
ANDROID_SOURCES_CXX_STL_LIBDIR = $$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/$$NDK_TOOLCHAIN_VERSION/libs/$$ANDROID_TARGET_ARCH
ANDROID_SOURCES_CXX_STL_INCDIR = $$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/$$NDK_TOOLCHAIN_VERSION/include $$ANDROID_SOURCES_CXX_STL_LIBDIR/include
equals(ANDROID_TARGET_ARCH, x86_64)|equals(ANDROID_TARGET_ARCH, mips64): \
QMAKE_ANDROID_PLATFORM_LIBDIR = $${QMAKE_ANDROID_PLATFORM_LIBDIR}64
#additionl fix
QMAKE_CFLAGS += -Wno-attributes #ignore Android Macros
QMAKE_CFLAGS += \
-Wno-unused-parameter \
-Wno-unused-variable \
-Wno-unused-but-set-variable \
-Wno-unused-value \
-Wno-unused-function
INCLUDEPATH += $$ANDROID_NDK_ROOT/sysroot/usr/include
INCLUDEPATH += $$ANDROID_NDK_ROOT/sysroot/usr/include/arm-linux-androideabi
## you most times need set "ANDROID_PACKAGE_SOURCE_DIR" manualys
isEmpty(ANDROID_PACKAGE_SOURCE_DIR) {
DISTFILES += \
$$PWD/res/android/AndroidManifest.xml \
$$PWD/res/android/gradle/wrapper/gradle-wrapper.jar \
$$PWD/res/android/gradlew \
$$PWD/res/android/res/values/libs.xml \
$$PWD/res/android/build.gradle \
$$PWD/res/android/gradle/wrapper/gradle-wrapper.properties \
$$PWD/res/android/gradlew.bat
ANDROID_PACKAGE_SOURCE_DIR = $$PWD/res/android
!build_pass:warning(no ANDROID_PACKAGE_SOURCE_DIR defaulted to $$ANDROID_PACKAGE_SOURCE_DIR)
}
#ANDROID_EXTRA_LIBS = $$PWD/libTest.so ## you may need
#ANDROID_DEPLOYMENT_SETTINGS_FILE = $$PWD/android-settings.json ## no need ever
#QML_IMPORT_PATH
#QML_DESIGNER_IMPORT_PATH
#QMAKE_PROJECT_NAME
#isEmpty(ANDROID_PACKAGE_SOURCE_DIR) {
# # note that $$PWD/android/assets directory is "QFile::ReadOnly" on android
# android: varDirInstall.path =/assets
# else: varDirInstall.path =$$OUT_PWD
#
# varDirInstall.files = $$files($$PWD/android/assets)
# win32: varDirInstall.files ~= s|\\\\|/|g
# varDirInstall.depends += FORCE
# INSTALLS += varDirInstall
#}
usage sample:
ANDROID_API = 21
ANDROID_ARCH = armeabi-v7a
include($$PWD/android.pri)
but I do get below error some times any way for TEMPLATE = app:
No Android arch set by the .pro file. Error while building/deploying
project vpnAndroid (kit: Qt5_android_armeabi-v7a) When executing step
"Deploy to Android device"
This special QtCreator related bug costs time to fix:
Close IDE and delete all the QtCreator setting files (in windows "C:\Users\Admin\UserName\Roaming\QtProject")
Start IDE again and reconfigure Android Paths, Compilers, Debugers and Kits in this order
Close IDE to save changes
Backup the folder mentioned before to save time see below:
the QtCreator plugins related to Android do store there failure in settings and so you need to backup since the plugins may do that again
I trying to rebuild OpenCV static libraries for the Android ABIs of: armeabi, armeabi-7a, mips, x86.
I'm using Android Studio (because Eclipse is deprecated) latest and greatest version on Mac OS X.
I using NDK 11 that recommends Clang3.8, GCC is deprecated. Use the APP_STL (Application.mk) of libc++ instead of libstdc++.
Threading Building Blocks (libtbb.a) support: clearly indicate or configure in CMake generator for Android on Mac OS X whether tbb is supported for one more of the Android ABIs.
My current problem is the following link error building a shared library for an Android APK where a static library created by the OpenCV Android armeabi-7a CMake build does not link with libtbb.a, that was created by the same build process:
.../libopencv_imgproc.a(histogram.cpp.o):
histogram.cpp:
function tbb::interface7::internal::auto_partition_type::auto_partition_type(tbb::auto_partitioner const&):
error: undefined reference to 'tbb::internal::get_initial_auto_partitioner_divisor()'
[Many more errors like this one...]
Here is my cmake command:
cmake \
-DBUILD_PERF_TESTS=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_WITH_DEBUG_INFO=OFF \
-DBUILD_SHARED_LIBS=OFF \
-DWITH_TBB=ON \
-DENABLE_PRECOMPILED_HEADERS=OFF \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
-DCMAKE_C_FLAGS_RELEASE="-Os -DNDEBUG -fvisibility=hidden -ffunction-sections -fstack-protector-all" \
-DCMAKE_CXX_FLAGS_RELEASE="-Os -DNDEBUG -fvisibility=hidden -ffunction-sections -fstack-protector-all -fvisibility-inlines-hidden" \
-DANDROID_ABI=armeabi-v7a \
-DANDROID_TOOLCHAIN_NAME=arm-linux-androideabi-4.9 \
-DANDROID_STL=c++_static \
-DANDROID_NATIVE_API_LEVEL=android-15 \
-DCMAKE_TOOLCHAIN_FILE=../android/android.toolchain.cmake \
../..
The link error resulted from me fat fingering a change of an assign operator, ":=", to a plus-equal operator, "+=", in the OpenCV produced OpenCV.mk file for Android cross-compilation. But the overall problem with CMake generated build environments remain. That is the initial CMake configuration definition settings are only known to the CMake project author. This requires the CMake project author to correctly and clearly document the CMake definitions that will build the specific target, whether it is a cross-compiling build environment or a platform specific build environment. Without proper documentation, it's is very hard to reverse engineer the CMake script/package, whatever you call it, to figure how platform specific build configuration settings map to CMake configuration definitions.
I am trying to port libCurl to android with SSL support,
step one would be to port the curl without ssl support I guess so I started doing that. but I run into a problem.
as I read on the dev website and in the Android.mk file, the hard part is configuring the make first. so what I did is :
Download Android Source code (and compile it! since some of the intermediate libs are needed)
Download cURL
unpack curl under : {android_src}/external/curl
make the configure script for curl by creating a sh file in the external/curl folder with this content.
`
export A=/home/user/Development/AOSP/2.3.3
export CC=$A/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/arm-eabi-gcc
export NDK=/home/user/Development/Tools/sdk/android/ndk
export NDKLIBS=$NDK/platforms/android-4/arch-arm/usr/include
export SYSROOT=$A/ndk/build/platforms/android-4/arch-arm
export CPPFLAGS="-I $A/system/core/include"
export LDFLAGS="-L$A/out/target/product/generic/obj/lib/ -L$A/out/target/product/generic/system/lib/-L$SYSROOT/usr/lib -Wl,--gc-sections -nostdlib -lc -lm -ldl -llog -lgcc -Wl,--no-undefined,-z,nocopyreloc -Wl,-dynamic-linker,/system/bin/linker -L$NDK/out/target/product/generic/obj/lib/"
export CFLAGS="-fno-exceptions -Wno-multichar -mthumb -mthumb-interwork -nostdlib -lc -ldl -lm -march=armv5te -mtune=xscale -msoft-float -mandroid -fPIC -mthumb-interwork -mthumb -mlong-calls -ffunction-sections -fstack-protector -fno-short-enums -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__ -DANDROID -DOS_ANDROID -D__NEW__ -D__SGI_STL_INTERNAL_PAIR_H -I$SYSROOT/usr/include -I $A/system/core/include -I $NDKLIBS"
./configure --host=arm-eabi --with-ssl=$A/external/openssl
`
And the output summary is this one :
configure: Configured to build curl/libcurl:
curl version: 7.26.0
Host setup: arm-unknown-eabi
Install prefix: /usr/local
Compiler: /home/tanco/Development/AOSP/2.3.3/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/arm-eabi-gcc
SSL support: no (--with-{ssl,gnutls,nss,polarssl,cyassl,axtls} )
SSH support: no (--with-libssh2)
zlib support: enabled
krb4 support: no (--with-krb4*)
GSSAPI support: no (--with-gssapi)
SPNEGO support: no (--with-spnego)
TLS-SRP support: no (--enable-tls-srp)
resolver: default (--enable-ares / --enable-threaded-resolver)
ipv6 support: no (--enable-ipv6)
IDN support: no (--with-libidn)
Build libcurl: Shared=no, Static=yes
Built-in manual: enabled
--libcurl option: enabled (--disable-libcurl-option)
Verbose errors: enabled (--disable-verbose)
SSPI support: no (--enable-sspi)
ca cert bundle: /etc/ssl/certs/ca-certificates.crt
ca cert path: no
LDAP support: no (--enable-ldap / --with-ldap-lib / --with-lber-lib)
LDAPS support: no (--enable-ldaps)
RTSP support: enabled
RTMP support: no (--with-librtmp)
Protocols: DICT FILE FTP GOPHER HTTP IMAP POP3 RTSP SMTP TELNET TFTP
SONAME bump: yes - WARNING: this library will be built with the SONAME
number bumped due to (a detected) ABI breakage.
See lib/README.curl_off_t for details on this.
First strange thing that comes to mind is why is SSL not included in the config since the linker shows to the intermediate libs and ssl support flag is called, but after when I use the same curl_config.h file in the jni project which I created for the build (since it has a standalone Android.mk file it can be compiled simply by unzipping in the jni folder of a android project, copying the config file created in the AOSP source and calling ndk-build)
so I compile and I get :
$ ndk-build
Compile thumb : curl <= url.c
In file included from /Projects/temp/testNDK/jni/lib/url.c:32:0:
/Tools/sdk/android/ndk/platforms/android-14/arch-arm/usr/include/unistd.h: In function 'getpagesize':
/Tools/sdk/android/ndk/platforms/android-14/arch-arm/usr/include/unistd.h:171:3: warning: nested extern declaration of '__page_size' [-Wnested-externs]
/Tools/sdk/android/ndk/platforms/android-14/arch-arm/usr/include/unistd.h: In function '__getpageshift':
/Tools/sdk/android/ndk/platforms/android-14/arch-arm/usr/include/unistd.h:175:3: warning: nested extern declaration of '__page_shift' [-Wnested-externs]
/Projects/temp/testNDK/jni/lib/url.c: At top level:
/Projects/temp/testNDK/jni/lib/url.c:57:2: error: #error "We can't compile without socket() support!"
make: *** [/Projects/temp/testNDK/obj/local/armeabi/objs/curl/lib/url.o] Error 1
Here is the solution, updated to NDK8c
step zero: download and fix the Android NDK
I don't know how but the ndk has a very interesting flaw, which (in my oppinion) doesn't allow you to compile lot's of stuff, so to be able to compile OpenSSL you need to make a small fix, extract the ndk8c whereever you keep your tools, and then edit the file :
android-ndk-r8c/build/gmsl/__gmsl
line 512 :
change line
int_encode = $(__gmsl_tr1)$(wordlist 1,$1,$(__gmsl_input_int))
with line
int_encode = $(__gmsl_tr1)$(wordlist 1,$(words $1),$(__gmsl_input_int))
And you're good to go!
step one : Download OpenSSL and compile for Android :
either compile a ported version found here
or Download the official 1.0.0c version of OpenSSL and then compile it for android using the manual provided in the github I linked for the Android compatible version
So the next step is to get the libssl.so and libcrypto.so
and put the them in the NDK folder for easy access, so copy them from
openssl-folder/libs/armeabi/
to
android-ndk-r8c/platforms/android-8/arch-arm/usr/lib
this way when compiling you can include the libs using a simple linker switch -lssl -lcrypto
Step two : get Curl's latest source for here
Open the file in Docs/INSTALL and follow the steps needed to make the standalone toolchain and put in your desired folder, and then the tricky part, I needed to have android's source code for the config to continue, even though I have a standalone compiled openssl you can include the header files from there too, in anycase this is the more complicated version so you choose what you do, I did not choose to evade them so you can go to Google AOSP site and go trough the steps to build and initialize the environment.
so it would be something like :
1.download,
go to root of the source code and run :
~: build/envsetup.sh; lunch 1; make;
So finally we need to compile curl with SSL support, so,
Step three
extract curl to the desired folder
(I have a specific desire of disabling everything except http/s to keep the library as small as possible meaning about ~300k, if you want more protocols in your lib, remove the --disable-protocol for the desired protocol)
run the following :
make clean
export PATH=/opt/arm-linux-androideabi-4.4.3/bin:$PATH
export LDFLAGS="\
-lssl \
-lcrypto \
-L/home/user/Development/Tools/sdk/android/ndk/platforms/android-8/arch-arm/usr/lib"
export CFLAGS="\
-I/home/user/Development/AOSP/2.3.7/system/core/include \
-I/home/user/Development/Tools/sdk/android/ndk/platforms/android-8/arch-arm/usr/include"
./configure --host=arm-linux-androideabi \
--with-ssl=/home/user/Development/Projects/portingLibs/openssl-android-master \
--disable-ftp \
--disable-gopher \
--disable-file \
--disable-imap \
--disable-ldap \
--disable-ldaps \
--disable-pop3 \
--disable-proxy \
--disable-rtsp \
--disable-smtp \
--disable-telnet \
--disable-tftp \
--without-gnutls \
--without-libidn \
--without-librtmp \
--disable-dict
make
Note that in the block above, if you don't want to use the AOSP source, you could switch
-I/home/user/Development/AOSP/2.3.7/system/core/include \
with the include folder for your ssl distribution.
So finally you have :
static :
curl-7.28.1/lib/.libs/libcurl.a
and shared :
curl-7.28.1/lib/.libs/libcurl.so.5.3
So that's it.. take the file, and compile away :)
I like the way to use Docker for building such static libs. There is full script here - you just run it and grab the result. Sample: https://gist.github.com/VictorLaskin/1c45245d4cdeab033956
Such script will:
setup compilation tools / utils download sdk/ndk
create custom cross-compilation toolchain download source code for libs (zlib, openssl, curl)
setup environment settings for cross compilation
configure and make libs
gather output at one folder and create the way to get
compiled libs
This specific version is using latest NDK 10e and clang toolchain.
There is post with more details here: http://vitiy.info/dockerfile-example-to-compile-libcurl-for-android-inside-docker-container/
I hope this will help someone to not waste precious time.
Steps for compiling libcurl.so with ssl support for android jni could be found here :
http://ieroot.com/2015/03/29/1728.html
I struggled for a week for figuring out it.
Ask if any questions. I'll reply as soon as possible.
I've been trying for a couple days to compile a native ARM Android binary that will execute on my phone using a terminal application. I want to generate the same type of binary as the standard Posix binaries installed on the phone like ls, mkdir etc. I've downloaded the Android NDK under Mac OS X and have been able to compile simple ELF binaries without errors. However, when I transfer them to the phone, they always segfault. That is, they segfault when compiled with -static in GCC. If I don't use -static, they complain about not being linked, etc. Put simply, they don't work.
My hypothesis is that they are not linking to the Android standard C library properly. Even though I am linking my binaries with the libc provided by the NDK, they still don't work. I read that Android uses the Bionic C library, and tried to download source for it but I'm not sure how to build a library from it (it's all ARM assembly, it seems).
Is it true that the Android C library on the phone is different from the one provided with the Android NDK? Will the one included with the NDK not allow me to compile native binaries I can execute through a terminal? Any guidance here is greatly appreciated!
Update:
I finally got this to work using GCC 4.7.0 on Mac OS X. I downloaded the Bionic headers and then compiled a dynamically linked binary using the C library that comes with the Android NDK. I was able to get a test app to work on the phone using the phone's C lib (the binary was 33K). I also tried to statically link against the NDK's C library, and that also worked.
In order to get this all working I had to pass -nostdlib to GCC and then manually add crtbegin_dynamic.o and crtend_android.o to GCC's command line. It works something like this:
$CC \
$NDK_PATH/usr/lib/crtbegin_dynamic.o \
hello.c -o hello \
$CFLAGS \
$NDK_PATH/usr/lib/crtend_android.o
For static binaries, use "crtbegin_static.o." This is explained in the crtbegin_dynamic.S/crtbegin_static.S source.
For this experiment, I only used plain 'ol GCC 4.7.0 and Binutils 2.22. I also compiled GCC with newlib, but I am not actually linking my ARM binaries with newlib at all. I am forcing GCC/ld to link directly to the libc provided with the Android NDK, or in the case of dynamic binaries, to the libc on the phone.
Just use the android-ndk. And build a Android.mk like so.
include $(BUILD_EXECUTABLE) is what tells it build a executable instead of a JNI .lib
Android.mk
ifneq ($(TARGET_SIMULATOR),true)
LOCAL_PATH:= $(call my-dir)
include $(CLEAR_VARS)
LOCAL_CFLAGS += -Wall
LOCAL_LDLIBS := -L$(LOCAL_PATH)/lib -llog -g
LOCAL_C_INCLUDES := bionic
LOCAL_C_INCLUDES += $(LOCAL_PATH)/include
LOCAL_SRC_FILES:= main.cpp
LOCAL_MODULE := mycmd
include $(BUILD_EXECUTABLE)
endif # TARGET_SIMULATOR != true
First, make sure you have the NDK:
http://developer.android.com/tools/sdk/ndk/index.html
Here is the easiest way to compile a C binary for your phone:
http://developer.android.com/tools/sdk/ndk/index.html
http://www.kandroid.org/ndk/docs/STANDALONE-TOOLCHAIN.html
Usually $NDK(may be different) =
Linux:
/home/<user>/android-ndk
Mac OS X:
/Users/<user>/android-ndk
In Terminal:
# create tool-chain - one line
# New method in ndk 12.
$NDK/build/tools/make_standalone_toolchain.py --arch arm --install-dir=/tmp/my-android-toolchain
# Old method.
#$NDK/build/tools/make-standalone-toolchain.sh --platform=android-3 --install-dir=/tmp/my-android-toolchain
# add to terminal PATH variable
export PATH=/tmp/my-android-toolchain/bin:$PATH
# make alias CC be the new gcc binary
export CC=arm-linux-androideabi-gcc
# compile your C code(I tried hello world)
$CC -o foo.o -c foo.c
# push binary to phone
adb push foo.o /data/local/tmp
# execute binary
adb /data/local/tmp/foo.o
Using CMake with the Android NDK is a nice way to compile Android console applications.
Download CMake and android-cmake (set it up like this). If your program is called main.c, then write the following in file CMakeLists.txt:
project(test)
cmake_minimum_required(VERSION 2.8)
add_executable(test ./main.c)
and run cmake -DCMAKE_TOOLCHAIN_FILE=$ANDTOOLCHAIN .
You will then have a Makefile for your program, you can run make to have your test executable.
In CMake, you can cross build using toolchain files.
From google developers:
cmake \
-DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake \
-DANDROID_ABI=$ABI \
-DANDROID_PLATFORM=android-$MINSDKVERSION \
$OTHER_ARGS
CMake has its own built-in NDK support. Before CMake 3.21, this workflow is not supported by Android and is often broken with new NDK releases. Starting from CMake 3.21, the implementations are merged.
Starting from cmake 3.21 you can:
mkdir build
cd build
cmake ..
-DCMAKE_SYSTEM_NAME=Android
-DCMAKE_SYSTEM_VERSION=23 # API level. optional, recommanded
-DCMAKE_ANDROID_NDK=path/to/ndk
-DCMAKE_ANDROID_ARCH=arm # optional, recommanded
-DCMAKE_ANDROID_ARCH_ABI=armeabi # optional, recommanded
Note: in the command above, line endings (<line feed>) are not escaped, please don't copy-paste this command directly in your shell
See Cross Compiling for Android with the NDK for more information about variables, possible values, and determenation algorithms.
Try if if the agcc wrapper can help you as referenced in the Android-tricks blog. According to the blog post you want to use the bionic library, but the one already installed on the phone, not some separately compiled version.
I've read in the gcc documentation that it supports building android binaries with the -mandroid and -mbionic switches. First I tried it with the native gcc built from svn. Result:
dancsi#dancsi-VirtualBox:~$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/i686-pc-linux-gnu/4.7.0/lto-wrapper
Target: i686-pc-linux-gnu
Configured with: ../source/configure --enable-threads --disable-nls
Thread model: posix
gcc version 4.7.0 20110611 (experimental) (GCC)
dancsi#dancsi-VirtualBox:~$ g++ test.cpp -mandroid -o test.out
dancsi#dancsi-VirtualBox:~$ file test.out
test.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, not stripped
Basically, it failed. Next, I tried to compile the same gcc sources with target=arm-linux-androideabi, (without defining sysroot,...), and it passed the configuration, but failed to build saying pthread.h is not found:
In file included from ../.././gcc/gthr-default.h:1:0,
from ../../../combined/libgcc/../gcc/gthr.h:160,
from ../../../combined/libgcc/../gcc/unwind-dw2.c:37:
../../../combined/libgcc/../gcc/gthr-posix.h:41:21: fatal error: pthread.h: No such file or directory
compilation terminated.
So, does anybody have the instructions for how to use gcc to build android binaries (I don't want to use code sourcery or android-ndk)?
EDIT:
my configure options
configure --target=arm-linux-android --host=i686-pc-linux-gnu --build=i686-pc-linux-gnu --disable-libssp --disable-libgomp --disable-nls --enable-languages=c,c++,java
I had exactly the same problem as described by dancsi while trying to compile my own cross compilation tool chains for Android (I wanted to build a 32 bit variant of them so I don't need to buy a 64 bit PC or do my android stuff in a VM).
I am using the tool chain sources from the official AOSP tool chain repo:
repo init -u https://android.googlesource.com/toolchain/manifest
The solution is to use the following 2 extra configure options: --with-headers and --with-libs. I've got this solution by reading the following file: [android-toolchain-repo]/build/README:
The other way is to specify the headers and libraries with --with-headers and
--with-libs separately. If you configure your tree with a prefix. The headers
and libraries will be copied to the install directory specified by the prefix.
For the values of those 2 extra parameters I used [android-src-repo]/prebuilts/ndk/8/platforms/android-14/arch-arm/usr/lib and [android-src-repo]/prebuilts/ndk/8/platforms/android-14/arch-arm/usr/include. [android-src-repo] is referring to my repo checkout of the base AOSP sources (in my case I am on the android-4.2.2_r1 branch of it).
The end-result of the two configure statements I need are the following:
./configure --target=arm-eabi --host=i686-linux-gnu --build=i686-linux-gnu --disable-gold \
--with-gcc-version=4.6 --with-binutils-version=2.21 --with-gmp-version=4.2.4 --with-mpfr-version=2.4.1 \
--with-gdb-version=7.3.x --with-gold-version=2.21 --prefix=/tmp/toolchain --disable-multilib \
--with-libs=/home/boeboe/android-src/prebuilts/ndk/8/platforms/android-14/arch-arm/usr/lib \
--with-headers=/home/boeboe/android-src/prebuilts/ndk/8/platforms/android-14/arch-arm/usr/include \
--program-transform-name='s&^&arm-eabi-&' --with-arch=armv5te --disable-option-checking
./configure --target=arm-linux-androideabi --host=i686-linux-gnu --build=i686-linux-gnu --enable-gold \
--with-gcc-version=4.6 --with-binutils-version=2.21 --with-gmp-version=4.2.4 --with-mpfr-version=2.4.1 \
--with-gdb-version=7.3.x --with-gold-version=2.21 --prefix=/tmp/toolchain \
--with-libs=/home/boeboe/android-src/prebuilts/ndk/8/platforms/android-14/arch-arm/usr/lib \
--with-headers=/home/boeboe/android-src/prebuilts/ndk/8/platforms/android-14/arch-arm/usr/include \
--program-transform-name='s&^&arm-linux-androideabi-&' --disable-option-checking
Notice the extra --disable-option-checking argument which was necessary for configure to accept all arguments!
This long list of arguments was constructed by checking the original configure arguments passed for the default 64 bit version of the 4.6 gcc toolchain. This can be checked in the SOURCES files available in [android-src-repo]/prebuilts/gcc/linux-x86/arm/arm-eabi-4.6 and [android-src-repo]/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.6.
In the same files, git tags can be found, so I also checkout those specific version in my [android-toolchain-repo]. For the extra modules needed by gcc, like gmp/mpfr/gdb..., I extracted the correct versions of the tar-balls available in [android-toolchain-repo]/gmp|mpfr|gdb. This was the easiest way I could find out, for my original configure statement to be able to pick up those modules.
The same technique as described above also allowed me to compile a 32 bit variant of the latest gcc 4.7 tool chain (currently being used on the MAIN branch of AOSP). The configure options were of course different in this case.
In file included from ../.././gcc/gthr-default.h:1:0,
from /mnt/network/source/gcc/libgcc/../gcc/gthr.h:160,
from /mnt/network/source/gcc/libgcc/../gcc/unwind-dw2.c:37:
/mnt/network/source/gcc/libgcc/../gcc/gthr-posix.h:41:21: fatal error: pthread.h: No such file or directory
compilation terminated.
make[2]: *** [unwind-dw2.o] Error 1
make[1]: *** [all-target-libgcc] Error 2
make: *** [all] Error 2
I'm getting the same error as you, but I'm not compiling for Android. I'm trying to build w64-mingw32 compiler.
AR=x86_64-w64-mingw32-ar RANLIB=x86_64-w64-mingw32-ranlib
../gcc-4.5.1/configure --prefix=/tools \
--with-local-prefix=/tools --enable-clocale=gnu \
--enable-shared --enable-threads=posix \
--enable-__cxa_atexit --enable-languages=c,c++ \
--disable-libstdcxx-pch --disable-multilib \
--disable-bootstrap --disable-libgomp \
--without-ppl --without-cloog --build=$MACHTYPE --host=$MACHTYPE --target=x86_64-w64-mingw32
My response is so you know its not just Android, the trouble is more general then just building for the ARM architecture.
I think its a c++ issue, cause I had/have no trouble with building a c compiler alone.