Develop Reference

Tensorflow android cmake build can't find python

I'm trying to build tflite in my android project using cmake. I've managed to work this approach on the linux (debian) version of the same project.When compiling the arm64 version I followed the official docs When I open android studio and press build/run on top right corner, I get this error:
In file included from /home/user/Desktop/official_stuff/tensorflow_src/tensorflow/lite/python/interpreter_wrapper/numpy.cc:17:
In file included from /home/user/Desktop/official_stuff/tensorflow_src/tensorflow/lite/python/interpreter_wrapper/numpy.h:49:
In file included from /usr/include/python3.9/Python.h:8:
/usr/include/python3.9/pyconfig.h:9:12: fatal error: 'aarch64-linux-gnu/python3.9/pyconfig.h' file not found
When I open the file that throws the error I see the this line indicating that it's searching for it in the system:
#include <aarch64-linux-gnu/python3.9/pyconfig.h>
I ran sudo find / -name "aarch64-linux-gnu" to see whether I have that file or not and indeed I have this file installed:
user#debian:~$ sudo find / -name "aarch64-linux-gnu"
...
/home/user/toolchains/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu/aarch64-linux-gnu/include/c++/8.3.0/aarch64-linux-gnu
find: ‘/run/user/1000/doc’: Permission denied
find: ‘/run/user/1000/gvfs’: Permission denied
/usr/lib/mesa-diverted/aarch64-linux-gnu
/usr/lib/aarch64-linux-gnu
/usr/share/gdb/auto-load/lib/aarch64-linux-gnu
/usr/share/gdb/auto-load/usr/lib/aarch64-linux-gnu
/usr/include/finclude/aarch64-linux-gnu
/usr/include/aarch64-linux-gnu
I located inside /usr/include/aarch64-linux-gnu and indeed saw /python3.9/pyconfig.h.
The way I did everything is this:
sudo git clone https://github.com/tensorflow/tensorflow.git /home/user/Desktop/official_stuff/tensorflow_src
curl -LO https://storage.googleapis.com/mirror.tensorflow.org/developer.arm.com/media/Files/downloads/gnu-a/8.3-2019.03/binrel/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz
mkdir -p ${HOME}/toolchains
tar xvf gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu.tar.xz -C ${HOME}/toolchains
ARMCC_PREFIX=${HOME}/toolchains/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu/bin/aarch64-linux-gnu-
ARMCC_FLAGS="-funsafe-math-optimizations"
cmake -DCMAKE_C_COMPILER=${ARMCC_PREFIX}gcc \
-DCMAKE_CXX_COMPILER=${ARMCC_PREFIX}g++ \
-DCMAKE_C_FLAGS="${ARMCC_FLAGS}" \
-DCMAKE_CXX_FLAGS="${ARMCC_FLAGS}" \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON \
-DCMAKE_SYSTEM_NAME=Linux \
-DCMAKE_SYSTEM_PROCESSOR=aarch64 \
../tensorflow/lite/
And then I needed to ran dpkg --add-architecture arm64, apt-get update, sudo apt install libpython3-dev:arm64
I simply clicked build ran after connecting my android device. It compiled for a while and then throw the error.
Here is the cmake snippet I ran that contains my tflite inclusion:
set(TENSORFLOW_SOURCE_DIR "" CACHE PATH
"Directory that contains the TensorFlow project" )
if(NOT TENSORFLOW_SOURCE_DIR)
get_filename_component(TENSORFLOW_SOURCE_DIR
"/home/user/Desktop/official_stuff/tensorflow_src" ABSOLUTE)
endif()
add_library(tensorflowlite SHARED IMPORTED)
add_subdirectory(
"/home/user/Desktop/official_stuff/tensorflow_src/tensorflow/lite"
"${CMAKE_CURRENT_BINARY_DIR}/tensorflow-lite" EXCLUDE_FROM_ALL
)
...
target_link_libraries(
my_proj
tensorflow-lite
)
How can I fix this issue?
Note: I got to this point following a series of questions which all resolved:
An undefined error.
Question regarding android build.
More details about how I built everything can be found here. I'm on debian 11.

Building Openvino for Android using Cmake

I tried to build openvino for Android devices (arm64) using Cmake, to do that I followed the following steps:
Operating system: Ubuntu 20
1 - I installed OpenJDK 8 using:
sudo apt-get install -y openjdk-8-jdk
2 - I downloaded the android NDK for linux using:
wget https://dl.google.com/android/repository/android-ndk-r20-linux-x86_64.zip
3 - I cloned the OpenVINO and OpenVINO Contrib repositories using:
git clone --recurse-submodules --shallow-submodules --depth 1 --branch=2021.4.1 https://github.com/openvinotoolkit/openvino.git
git clone --recurse-submodules --shallow-submodules --depth 1 --branch=2021.4 https://github.com/openvinotoolkit/openvino_contrib.git
4 - I built openvino using cmake:
cmake -DCMAKE_BUILD_TYPE=Release -DTHREADING=SEQ -DIE_EXTRA_MODULES=../../openvino_contrib/modules -DBUILD_java_api=ON -DCMAKE_TOOLCHAIN_FILE=../../android-ndk-r20/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=24 -DANDROID_STL=c++_shared -DENABLE_SAMPLES=OFF -DENABLE_OPENCV=OFF -DENABLE_CLDNN=OFF -DENABLE_TESTS=OFF -DENABLE_GAPI_TESTS=OFF -DENABLE_BEH_TESTS=OFF -DENABLE_PYTHON=OFF .. && make --jobs=$(nproc --all)
Expectations:
As many tutorials showed it I was expecting a file called inference_engine_java_api.jar to be generated in this path: openvino/bin/aarch64/Release/lib/
Results:
I didn't get any file which has _java_api in its name.
I just got some .so and .a files like: inference_engine_c_api.so and inference_engine_legacy.so.

You just need to install: scons in ubuntu.
sudo apt-get update
sudo apt-get install scons

How to compile Boost 1.61 for Android NDK 11

I want to install compile Boost 1.61 with clang 3.6 for android with the NDK 11 but this software : https://github.com/moritz-wundke/Boost-for-Android isn't updated and doesn't support this versions.
I want to know if anyone has managed to this !
Thank you !

Build boost_1_62_0 for Android-21 under Windows64.
Assuming NDK installed to C:\Programs\Android\sdk\ndk-bundle and boost in c:\boost_1_62_0.
Install mingw: using msys2-x86_64 from MSYS2
Install build tools from mingw prompt (something like this):
$ pacman -S gcc binutils
Create android.clang.jam file in C:\boost_1_62_0\ with such text content:
import os ;
local AndroidNDKRoot = C:/Programs/Android/sdk/ndk-bundle ;
using clang : android
:
C:/Programs/Android/toolchain21/bin/clang++
:
<compileflags>-fexceptions
<compileflags>-frtti
<compileflags>-fpic
<compileflags>-ffunction-sections
<compileflags>-funwind-tables
<compileflags>-Wno-psabi
<compileflags>-march=armv7-a
<compileflags>-mfloat-abi=softfp
<compileflags>-mfpu=vfpv3-d16
<compileflags>-fomit-frame-pointer
<compileflags>-fno-strict-aliasing
<compileflags>-finline-limit=64
<compileflags>-I$(AndroidNDKRoot)/platforms/android-21/arch-arm/usr/include
<compileflags>-Wa,--noexecstack
<compileflags>-DANDROID
<compileflags>-D__ANDROID__
<compileflags>-DNDEBUG
<compileflags>-O2
#<compileflags>-g
<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.9/include
<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi/include
<architecture>arm
<compileflags>-fvisibility=hidden
<compileflags>-fvisibility-inlines-hidden
<compileflags>-fdata-sections
<cxxflags>-D__arm__
<cxxflags>-D_REENTRANT
<cxxflags>-D_GLIBCXX__PTHREADS
;
Setup boost from mingw prompt:
$ export NDK=/c/Programs/Android/sdk/ndk-bundle
$ echo ensure msi-installed Python is on path (not msys version):
$ export PATH=/c/Python27:$PATH
$ $NDK/build/tools/make_standalone_toolchain.py --arch arm --api 21 --install-dir /c/Programs/Android/toolchain21
$ ./bootstrap.sh --with-toolset=gcc
$ ./b2 --user-config=android.clang.jam threading=multi link=static \
runtime-link=static toolset=clang-android target-os=linux \
threadapi=pthread --stagedir=android --with-chrono \
--with-program_options --with-system --with-thread --with-random \
--with-regex

Yes, the repository you mentioned is apparently not maintained anymore. The author also seems not to answer any mails on the subject. If you look you'll see that each new boost version supported there required a lot of work (many special flags in the config files). That's presumably why he doesn't have time to maintain it any longer.
I also tried to update to 1.64 using a fork but gave up after countless error messages and instead used a different method based on a crystax script. Its simple and should work for pretty much any version. You can find the details and the script (simple and painless to execute) here: http://silverglint.com/boost-for-android/
Works with clang and gcc.
Also included is a sample app that shows you how to use the boost binaries thus built.

Building webrtc library

I am trying to build webrtc android demo application. I was following README applied by the project. First thing was mentioned about gclient but when i tried to run it "no external or internal command". Then i got this link according to the this link i cloned the depot_tool repository but when at the step when i try to run gclient i got this
So after hours of trying i am not able to run it. if any one has experience with this WebRTC library please provide some help. Any guidance? Any Hlep will be appreciated

I heard few things were changed from svn to GIT. We tried building webRTC to windows7 using visual studio. See if you can use any parts on this.
Get depot_tools software and extract to a location (set this path in environment variable).
cd to depot_tools directory and hit 'gclient' (this should take sometime)
Install python 2.7 and put it path.
create a folder of your choice for webRTC and open command prompt there.
Now hit, gclient config http://webrtc.googlecode.com/svn/trunk
then, gclient sync.
Then set GYP_GENERATORS=msvs environment variable (use ninja if you want that).
Also set msvs_version=2012 (depending on your visual studio version)
now run, gclient runhooks --force
This should generate solutions files for your folder. fireup the libjingle/webrtc.sln files and start building it. I'm not sure if it builds as is. But we had hicupps with that.
See this link on WebRTC native development if you haven't already. From my personal experience building this tricky job initially.
Also note that links might be deprecated as they moved to git, don't know if they are still maintaining it.

How to compile WEBRTC On ubuntu
download script and run:
https://cs.chromium.org/chromium/src/build/install-build-deps.sh
./build/install-build-deps.sh --no-chromeos-fonts
Now download depot tools
$ git clone --depth 1 https://chromium.googlesource.com/chromium/tools/depot_tools.git
Put it in Path
$ export PATH=`pwd`/depot_tools:"$PATH"
Define your settings:
export GYP_DEFINES="target_arch=x64 host_arch=x64 build_with_chromium=0 use_openssl=0 use_gtk=0 use_x11=0 include_examples=0 include_tests=1 fastbuild=1 remove_webcore_debug_symbols=1 include_pulse_audio=0 include_internal_video_render=0 clang=1 "
make file to download the source
mkdir webrtc-checkout
cd webrtc-checkout
I chose branch 50 you can change it
gclient config https://chromium.googlesource.com/external/webrtc.git#branch-heads/50 --name=src
gclient sync --force --with_branch_heads
gclient sync --force --with_branch_heads --nohooks
cd src
git checkout branch-heads/50
gclient runhooks
ninja -C ./out/Release
Collect your LIB & Include file to use it in your project
mkdir ../lib/
find ./ -name "*.o" -and -not -name do_not_use -and -not -name protoc -and -not -name genperf -exec ar crs ../lib/libwebrtc.a {} +
mkdir ../include
find ./ -name *.h -exec cp --parents '{}' ../include ';'

Building OpenSSL for Android (ARMv7) on Win32

How can I build OpenSSL for Android ARM v7 (using Android NDK) on Win32?

Until the OpenSSL's wiki and setenv-android.sh are updated accordingly, I'll publish the recipe here. The required fixes to the process are:
Update setenv-android.sh to support Windows.
Update PATH to use Android NDK's (mingw) GNU make (rather than Cygwin's).
Invoke make with a Windows-style path to Cygwin's perl.
This recipe will be a strange hybrid of Cygwin and mingw (since Android NDK gcc toolchains for win32 are mingw). I'm assuming a Windows x86_64 build of the Android NDK unpacked into c:\android-ndk-r9d, and that you wish to use a gcc 4.8 toolchain.
Install Android NDK (duh!).
Install Cygwin -- make sure to include perl
Start Cygwin shell as an administrator to make sure native symlinks will work.
Within the console, run the following script to set the variables:
export \
CYGWIN=winsymlinks:native \
ANDROID_API=android-14 \
ANDROID_DEV=c:/android-ndk-r9d/platforms/android-14/arch-arm/usr \
PATH=/cygdrive/c/android-ndk-r9d/prebuilt/windows-x86_64/bin:/cygdrive/c/android-ndk-r9d/toolchains/arm-linux-androideabi-4.8/prebuilt/windows-x86_64/bin:$PATH \
MACHINE=armv7 \
SYSTEM=android \
ARCH=arm \
CROSS_COMPILE=arm-linux-androideabi-
Now, unpack openssl:
tar xzfv openssl-1.0.1i.tar.gz (or whatever your tarball is)
cd openssl-1.0.1i (or whatever your version is)
Make sure you have actual native Win32 (!) symlinks in include/openssl:
cmd /c "dir include\openssl"
You should see something like:
13-Aug-14 05:59 PM <SYMLINK> aes.h [..\..\crypto\aes\aes.h]
13-Aug-14 05:59 PM <SYMLINK> asn1.h [..\..\crypto\asn1\asn1.h]
(etc.)
Now it's time to configure:
./config shared -no-ssl2 -no-ssl3 -no-comp -no-hw -no-engine --openssldir=/foo/bar
Ignore the failure to build (due to failure to find perl). We'll rectify this right away. Do this:
make PERL=$(cygpath -w $(which perl))
Now wait for a few minutes until it builds, and presto, you have your libcrypto.so etc.

Just a couple of comments on my experience with this:
Executing this statement:
PERL=$(cygpath -w $(which perl))
in the cygwin shell allows the shell to interpret the backslashes as escape characters and the build process chokes. To solve this I did the following:
$ echo $(cygpath -w $(which perl))
which produced the windows formatted path to the perl executable:
C:\cygwin64\bin\perl
Then I added this line to the export shown above:
PERL=c:/cygwin64/bin/perl \
There are other ways of doing this, but it worked and headed off the problem with the ./config statement documented above (not finding perl).
Second issue was the -no- statements. After running the configure, the script reports that you'll have to run make depend. I wanted to exclude MD5 (i.e. -no-md5) and when I did the make depend, it errored out with a report that MD5 was disabled. Uhhh, yes, that was kind of the idea, but I just won't use MD5 hashes. I did use the -no-ssl2 and didn't get any complaints after the make depend.
Third issue and this is a mystery. The build broke on compiling crypto because it could not find a symbol that is supposed to be defined in /crypto/objects/obj_xref.h. When I looked at the file, it was empty. Something in the perl script I suppose, but no time to debug right now, since I'm at proof of concept phase. I placed a copy from a patch that I picked up at https://github.com/devpack/openssl-android
After that, my build ran to completion. I've done no testing with this and it is not a trustworthy solution, but it did compile and produce the static libraries that I need for proof of concept for my client.
Just as an update, my shared library built with these libraries loaded fine on my target.