Develop Reference

OpenCV for Android via Visual Studio to Unity

I tried to compile a .so library using Visual Studio 2019 along with OpenCV Android in order to use this library in Unity.
There are some answers on how to configure Visual Studio to use OpenCV Android (here or here) but none of these work for me. Below you can see my configurations.
Visual Studio 2019 (running on Windows 10)
android-ndk-r21e // also tried with android-ndk-r15c android-ndk-r16b and android-ndk-r17c
OpenCV Android 4.5.2 // also tried with OpenCV Android 4.0.0, 4.0.1 and 4.1.0
My settings in Visual Studio 2019 look as follows:
Configuration Properties
- General
Platform Toolset Clang 5.0 (also tried Clang 3.8 or GCC 4.9)
Configuration Type Dynamic Library (.so)
Target API Level Nougat 7.0 (android-24) (also tried different versions)
Use STL LLVM libc++ static library (c++_static) (also tried "GNU STL static library (gnustl_static)")
C/C++
- General
Additional Include Directories "Path to OpenCV_4_5_2_Android\sdk\native\jni\include"
Code Generation Enable C++ Exceptions "Yes(-fexceptions)"
Language C++17(-std=c++1z)
Precompiled Headers Not using Precompiled Headers
Linker
- General
Additional Library Directories Path to OpenCV_4_5_2_Android\sdk\native\libs\armeabi-v7a
- Input
Additional Dependencies Path to OpenCV_4_5_2_Android\sdk\native\libs\armeabi-v7a\libopencv_java4.so
My Source.cpp I try to compile is just a single function for testing purposes
#include <opencv2/core.hpp>
extern "C" float test(float a, float b)
{float c = a * b; return c;}
Which gives me the following errors:
E0035 #error directive: This constructor has not been ported to this platform
E0020 identifier "__fp16" is undefined
use of undeclared identifier 'ANDROID_LOG_INFO'
The ANDROID_LOG_INFO error can be fixed when I add #include "android/log.h" at the top of the file that throws this error. But the other two errors still remain.

I had the exact same issue as you (though I used c++ 11) with the exact same setup, and struggled for days. I believe the errors you're seeing (like me) are from arm_neon.h. Very oddly, I was able to just build (not run) the .so successfully, even with those errors (I say "errors" because if you look at arm_neon.h, others pop up), so try it. Maybe it's some kind of IntelliJ/Intellisense mistake more than anything else where it's catching false negatives from some other toolchain setup.
At the same time, I'm not 100% sure I was always able to build with that issue, so try these steps as well if you can't:
use OpenCV 4.0.1 with Android NDK 16rb. The NDK matters when it comes to OpenCV builds, and this is the only supposed match that I know of.
follow this tutorial from scratch: https://amin-ahmadi.com/2019/06/03/how-to-use-opencv-in-unity-for-android/
if the downloaded OpenCV android SDK is still giving trouble, build OpenCV from the source using his other tutorial here: https://amin-ahmadi.com/2019/02/03/how-to-build-opencv-4-x-for-native-android-development/
and then repeat step 2.
MAJOR EDIT:
OpenCV 4.5.2 needs to be treated differently because it no longer uses toolchains with gnu c++.
-When you build OpenCV from CMake, build with Android NDK 21e, and do not use the toolchain in OpenCV 4.5.2. Use the one inside the Android NDK's build folder (android-ndk-r21e\build\cmake).
-When you build your .so from Visual Studio 2019, do not use the GNU STL, use the LLVM. GNU c++ is no longer part of Android NDKs, and you need to cut it out of the process entirely.
-In the Linker Input, put the names of your library files (or file, if it's just the world one) in the Library Dependencies field, not the Additional Dependencies field.
-Everything else is the same as in those common tutorials.

How can I integrate OpenCV 4.0 into a pure C++ Android NDK project?

What are the steps necessary for me to integrate,
the latest version of OpenCV
into a pure C++(No Java Code) Android NDK project, such as Android NDK Google tutorial:
Endless-Tunnel
?
official documentation for android integration refers to much older version :OpenCV-2.4 and folder structures are no longer the same.
I am using Android Studio on Linux.
All help is highly Appreciated.

Download opencv Android package (e.g. opencv-4.0.1-android-sdk) and unpack to, say, ~/android.
To the bottom of CMakeLists.txt, add
set( OpenCV_DIR "~/android/OpenCV-android-sdk/sdk/native/jni" )
find_package( OpenCV REQUIRED )
target_link_libraries(game opencv_java)
The package will define the following variables:
OpenCV_LIBS : The list of all imported targets for OpenCV modules.
OpenCV_INCLUDE_DIRS : The list of OpenCV include directories. With CMake >= 2.8.11 you don't even need to write
include_directories(${OpenCV_INCLUDE_DIRS})
This version of prebuilt OpenCV SDK defines also
OpenCV_VERSION : The version of this OpenCV build: "4.0.1"
OpenCV_ANDROID_NATIVE_API_LEVEL : Minimum required level of Android API: "16".
This means that your app manifest needs minSdkVersion 16 or higher (the original sample needs a fix here).
Instead of the shared library that contains all OpenCV functionality, you can use static libraries (opencv_imgcodecs, opencv_stitching, et al). These static libraries assume the default ANDROID_STL=c++_static.
For best results, use NDK r.18 or r.19.
UPDATE: NDK r.21 works well for opencv 4.3.0.

If you would like to create shared objects(.so) and compile without using Android studio, here is my blog. This is often useful when you would like to create a native third-party library.

Building capicxx-core-runtime for Android

We are trying to build capicxx-core-runtime for an ARM based platform running Android Pie. It's an open source IPC framework which is part of the GENIVI initiative by major automotive OEM's. Our AIM is to bring the IPC framework into our Android system.
The git repo is placed here https://github.com/GENIVI/capicxx-core-runtime.git
I am using the Android NDK version r17b and building using the following command to configure the cmake based build.
cmake -DCMAKE_TOOLCHAIN_FILE=/home/hp/downloads/android-ndk-r17b/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_NATIVE_API_LEVEL=27 ../
When I compile using 'make', I see the individual cpp files are compiled, but I am getting following errors during linking as shown in the link below.What is the configuration that I may be missing.
Linker error log

the capicxx-core-runtime is building a shared library, so you need to resolve all of it's internal symbols, including the c++ runtime.
To quickly test this: edit the capicxx-core-runtime CMakeLists.txt to link the c++_shared runtime, line 130:
target_link_libraries(CommonAPI PRIVATE ${DL_LIBRARY} ${DLT_LIBRARIES})
becomes
target_link_libraries(CommonAPI PRIVATE ${DL_LIBRARY} ${DLT_LIBRARIES} c++_shared)
and rerun your cmake command and make command.

It seems that capicxx-core-runtime can be buildable now for Android (both NDK and AOSP) out the box, please see corresponding pull requests: https://github.com/GENIVI/capicxx-core-runtime/pulls?q=author%3Ankh-lab+
Also here is simple example for AndroidStudio how it could be used with vSOME/IP transport: https://github.com/nkh-lab/ndk-capi-hello-world

Problems with using tensorflow lite C++ API in Android Studio Project

I am currently working on a project regarding neural networks.
For this, I want to build an Android Application which should use tensorflow [lite] to solve some object detection / recognition problems.
As I want the code to be as portable as possible, I want to write most of the code in C++, thus using the C++ API of tensorflow lite over the Java API / wrapper.
So, I modified tensorflow/contrib/lite/BUILD and added the following to be able to create a shared tensorflow library.
cc_binary(
name = "libtensorflowLite.so",
linkopts=["-shared", "-Wl"],
linkshared=1,
copts = tflite_copts(),
deps = [
":framework",
"//tensorflow/contrib/lite/kernels:builtin_ops",
],
)
(Which is based on the answer to this issue: https://github.com/tensorflow/tensorflow/issues/17826)
Then I used
bazel build //tensorflow/contrib/lite:libtensorflowLite.so --crosstool_top=//external:android/crosstool --cpu=arm64-v8a --host_crosstool_top=#bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11"
to finally build it.
Afterwards I headed over to Android Studio and set up a basic project.
For adding the shared library to the project, I refered to this example:
https://github.com/googlesamples/android-ndk/tree/840858984e1bb8a7fab37c1b7c571efbe7d6eb75/hello-libs
I also added the needed dependencies for flatbuffers.
The build / compilation process succeeds without any linker errors (well, at least after trying around for some hours..).
The APK is then successfully installed on an Android device, but immediately crashes after it starts. Logcat gives the following output:
04-14 20:09:59.084 9623-9623/com.example.hellolibs E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.hellolibs, PID: 9623
java.lang.UnsatisfiedLinkError: dlopen failed: library "/home/User/tensorflowtest/app/src/main/cpp/../../../../distribution/tensorflow/lib/x86/libtensorflowLite.so" not found
at java.lang.Runtime.loadLibrary0(Runtime.java:1016)
at java.lang.System.loadLibrary(System.java:1657)
at com.example.hellolibs.MainActivity.<clinit>(MainActivity.java:36)
at java.lang.Class.newInstance(Native Method)
at android.app.Instrumentation.newActivity(Instrumentation.java:1174)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2669)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2856)
at android.app.ActivityThread.-wrap11(Unknown Source:0)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1589)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6494)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
I tried this on an android x86 emulator and on a real arm64-v8a android smartphone.
So for me this looks like on startup the application tries to load the tensorflowLite shared library, but is unable to find it.
Opening the apk with a zip archive manager I can verify that the platform (arm, x86) dependent .so files are packed into the APK as expected (by adding the following to build.gradle:
sourceSets {
main {
// let gradle pack the shared library into apk
jniLibs.srcDirs = ['../distribution/tensorflow/lib']
}
})
What I do not understand is why it looks for the library in the path where I placed it on my Ubuntu 17.10 PC.
So, I thought I had done a mistake trying to adapt the example about adding external libraries to an Android Studio Project I mentioned earlier.
That's why I downloaded the whole project and opened it in Android Studio and verified that the example works as expected. Afterwards I replaced the example libgperf.so by the libtensorflowLite.so and left everything else, especially the CMakeLists.txt, untouched.
But I get the exact same error again, therefore I suspect this to be a problem with the libtensorflowLite library itself and not the android project (although that's just my guess).
I am working on android studio 3.1.1, NDK Version 14 and API Level 24 (Android 7.0).
If anyone has an idea what could be wrong, any help would be highly appreciated.
I am also open for any other methods which allow me to use tensorflow lite with C++ for an android application.
Thanks a lot,
Martin

I just remembered I asked this question a few weeks ago.
Meanwhile, I found a solution to the problem and TensorflowLite is now nicely embedded into my Android Project, where I do all the programming using the C++ API!
The problem was that the Tensorflow shared library I built did not contain a soname. So, during build process, the library was stripped and as no name was found, the path was used as the "name". I noticed that while I further investigated my native-lib.so (the NDK C++ library which is then loaded by the App) using linux "strings" tool. Here I found out that indeed the path to load the library from "/home/User/tensorflowtest/app/src/main/cpp/../../../../distribution/tensorflow/lib/x86/libtensorflowLite.so" was set.
Adding a "-Wl,-soname=libtensorflowLite.so" to the build options in the BUILD file fixed this issue! You can find the whole rule I used below.
As it was a pain to get everything set up due to the lack of explanations (it seems TensorflowLite is mostly used via Java API on Android ?), I want to give a short guidance on how use the C++ API of TensorflowLite in Android Studio (from within an Android NDK project).
1. Build the library for your architecture
To use the C++ API, you first need to build the TensorflowLite library. For this, add the following rule to the BUILD file in tensorflow/contrib/lite:
cc_binary(
name = "libtensorflowLite.so",
linkopts=[
"-shared",
"-Wl,-soname=libtensorflowLite.so",
],
linkshared = 1,
copts = tflite_copts(),
deps = [
":framework",
"//tensorflow/contrib/lite/kernels:builtin_ops",
],
)
Note: With this, a shared library can be built! A static one might also work.
Now you can build the library using
bazel build //tensorflow/contrib/lite:libtensorflowLite.so --crosstool_top=//external:android/crosstool --cpu=arm64-v8a --host_crosstool_top=#bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11"
If you want to support multiple architectures, you will have to build the library several times and change the --cpu flag correspondingly.
NOTE: This works fine at least for arm64-v8a and the armeabi-v7a (haven't tested it with MIPS so this might work aswell). However on an x86 device, I get the "atomic_store_8" error already adressed in this topic: https://github.com/tensorflow/tensorflow/issues/16589
2. Add the library and the needed headers to be included in your Android Studio project
Having built the library, you now need to make sure it also is linked into your Application (more specifically: Into your Android NDK library, which in my case is named "native-lib"). I will give a short overview on how to do this, however if you need a more detailed explanation you may refer to the github link I provided in my initial question: https://github.com/googlesamples/android-ndk/tree/840858984e1bb8a7fab37c1b7c571efbe7d6eb75/hello-libs
2.1. In your Android Studio Project, open the CMakeLists.txt
2.2. Add the following:
# This will create a new "variable" holding the path to a directory
# where we will put our library and header files.
# Change this to your needs
set(distribution_DIR ${CMAKE_SOURCE_DIR}/distribution)
# This states that there exists a shared library called libtensorflowLite
# which will be imported (means it is not built with the rest of the project!)
add_library(libtensorflowLite SHARED IMPORTED)
# This indicates where the libtensorflowLite.so for each architecture is found relative to our distribution directory
set_target_properties(libtensorflowLite PROPERTIES IMPORTED_LOCATION
${distribution_DIR}/lib/${ANDROID_ABI}/libtensorflowLite.so)
# This indicates where the header files are found relative to our distribution dir
target_include_directories(native-lib PRIVATE
${distribution_DIR}/include)
# Finally, we make sure our libtensorflowLite.so is linked to our native-lib and loaded during runtime
target_link_libraries( # Specifies the target library.
native-lib
libtensorflowLite
# Links the target library to the log library
# included in the NDK.
${log-lib} )
2.3. Open the build.gradle for your Module: App (not the project one!)
2.4. Make sure our library will be packed into your APK
Add this inside the Android section:
sourceSets {
main {
// let gradle pack the shared library into apk
jni.srcDirs = []
jniLibs.srcDirs = ['distribution/lib']
}
}
You may have to edit the path accoding to your needs: The files here will be packed in to your .apk inside the lib directory.
3. Include flatbuffers
TensorflowLite uses the flatbuffers serialization library. I guess this will be added automatically if you build your project using bazel. But this is not the case when using Android Studio.
Of course, you could also add a static or shared library too.
However, for me it was easiest to just let flatbuffers compile each time with the rest of my app (it is not that big).
I copied all of the flatbuffers *.cpp source files to my project and added them to the CMakeLists.
4. Copy the needed headers for TensorflowLite and flatbuffers
In 3. I just copied the cpp files to my project.
However, the header files need to be located in the directory we set in target_include_directories in step 2.2.
So go ahead and copy all of the flatbuffers (from the flatbuffers repository) *.h files to this directory.
Next, from the TensorflowLite repository, you need all header files inside the tensorflow/contrib/lite directory. However you should keep the folder structure
For me it looks like this:
distribution
lib
arm64-v8a
libtensorflowLite
armeabi-v7a
libtensorflowLite
include
flatbuffers
tensorflow
contrib
lite
kernels
nnapi
schema
tools
So, if I haven't forgotten anything everything should be set up correctly by now!
Hopefully this helped and it worked for you as it did for me ;)
Best regards,
Martin

Use ARMCC to compile Android native library

I've been working on an Android project which has several native C++ libraries. Compiling and debugging using Eclipse with ADT plugin works well. Obviously Android NDK uses arm-linux-gnueabi-gcc of some version to compile the native libraries.
Since I've been using NEON intrinsics heavily, I would like to try to compile the native libraries with ARM's official compiler armcc. I read everywhere that armcc is supposed to give better optimized code when using intrinsics. So I downloaded the trial version of DS-5 from ARM website, just to try and see whether there's really any speed difference.
The DS-5 seems to be just a modified version of Eclipse that uses the ARMCC toolchain, so I installed the ADT plugin. But when I compile using DS-5, it seems that the code is still generated using gcc rather than armcc.
Do you have any idea how to force DS-5 or Eclipse to build the Android native code using armcc? Or is it possible (and how) to build the static NDK libraries from command line and then replace the libraries in my project, so they get deployed to the testing phone?

ARM DS-5 Community Edition doesn't include ARM compiler (armcc).
If you could get hold of armcc best would be to separate your processing heavy algorithms to individual compilation units (separate C files), build them with armcc as you would do for any compilation unit. When you get the object files, convert them into an archive then use that in Android.mk as LOCAL_STATIC_LIBRARIES += <your_archive>.
You can't use armcc plainly to build Android compatible libraries mostly because of Bionic dependencies, I think.

You can use armcc to build Android compatible static libraries even though Android has a different C library (bionic). The key is the --library_interface flag for armcc. According to the documentation:
Use an option of the form --library_interface=aeabi_* when linking with an ABI-compliant C library. Options of the form --library_interface=aeabi_* ensure that the compiler does not generate calls to any optimized functions provided by the ARM C library.
In addition, there are a few more flags to ensure compatibility with the Android EABI resulting in the following command for an Android armeabi-v7a target:
armcc --library_interface=aeabi_clib --wchar32 --enum_is_int --no_hide_all --cpu=7-A --fpu=softvfp+vfpv3 -c -o libfunc.o libfunc.c
You can then use armar --create libfunc.a libfunc.o to create a static library that can be linked with the Android NDK as LOCAL_STATIC_LIBRARIES.
I have successfully tested this with Android NDK r10d on Android KitKat 4.4.2.