I am currently working on a project regarding neural networks.
For this, I want to build an Android Application which should use tensorflow [lite] to solve some object detection / recognition problems.
As I want the code to be as portable as possible, I want to write most of the code in C++, thus using the C++ API of tensorflow lite over the Java API / wrapper.
So, I modified tensorflow/contrib/lite/BUILD and added the following to be able to create a shared tensorflow library.
cc_binary(
name = "libtensorflowLite.so",
linkopts=["-shared", "-Wl"],
linkshared=1,
copts = tflite_copts(),
deps = [
":framework",
"//tensorflow/contrib/lite/kernels:builtin_ops",
],
)
(Which is based on the answer to this issue: https://github.com/tensorflow/tensorflow/issues/17826)
Then I used
bazel build //tensorflow/contrib/lite:libtensorflowLite.so --crosstool_top=//external:android/crosstool --cpu=arm64-v8a --host_crosstool_top=#bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11"
to finally build it.
Afterwards I headed over to Android Studio and set up a basic project.
For adding the shared library to the project, I refered to this example:
https://github.com/googlesamples/android-ndk/tree/840858984e1bb8a7fab37c1b7c571efbe7d6eb75/hello-libs
I also added the needed dependencies for flatbuffers.
The build / compilation process succeeds without any linker errors (well, at least after trying around for some hours..).
The APK is then successfully installed on an Android device, but immediately crashes after it starts. Logcat gives the following output:
04-14 20:09:59.084 9623-9623/com.example.hellolibs E/AndroidRuntime: FATAL EXCEPTION: main
Process: com.example.hellolibs, PID: 9623
java.lang.UnsatisfiedLinkError: dlopen failed: library "/home/User/tensorflowtest/app/src/main/cpp/../../../../distribution/tensorflow/lib/x86/libtensorflowLite.so" not found
at java.lang.Runtime.loadLibrary0(Runtime.java:1016)
at java.lang.System.loadLibrary(System.java:1657)
at com.example.hellolibs.MainActivity.<clinit>(MainActivity.java:36)
at java.lang.Class.newInstance(Native Method)
at android.app.Instrumentation.newActivity(Instrumentation.java:1174)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2669)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2856)
at android.app.ActivityThread.-wrap11(Unknown Source:0)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1589)
at android.os.Handler.dispatchMessage(Handler.java:106)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6494)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
I tried this on an android x86 emulator and on a real arm64-v8a android smartphone.
So for me this looks like on startup the application tries to load the tensorflowLite shared library, but is unable to find it.
Opening the apk with a zip archive manager I can verify that the platform (arm, x86) dependent .so files are packed into the APK as expected (by adding the following to build.gradle:
sourceSets {
main {
// let gradle pack the shared library into apk
jniLibs.srcDirs = ['../distribution/tensorflow/lib']
}
})
What I do not understand is why it looks for the library in the path where I placed it on my Ubuntu 17.10 PC.
So, I thought I had done a mistake trying to adapt the example about adding external libraries to an Android Studio Project I mentioned earlier.
That's why I downloaded the whole project and opened it in Android Studio and verified that the example works as expected. Afterwards I replaced the example libgperf.so by the libtensorflowLite.so and left everything else, especially the CMakeLists.txt, untouched.
But I get the exact same error again, therefore I suspect this to be a problem with the libtensorflowLite library itself and not the android project (although that's just my guess).
I am working on android studio 3.1.1, NDK Version 14 and API Level 24 (Android 7.0).
If anyone has an idea what could be wrong, any help would be highly appreciated.
I am also open for any other methods which allow me to use tensorflow lite with C++ for an android application.
Thanks a lot,
Martin
I just remembered I asked this question a few weeks ago.
Meanwhile, I found a solution to the problem and TensorflowLite is now nicely embedded into my Android Project, where I do all the programming using the C++ API!
The problem was that the Tensorflow shared library I built did not contain a soname. So, during build process, the library was stripped and as no name was found, the path was used as the "name". I noticed that while I further investigated my native-lib.so (the NDK C++ library which is then loaded by the App) using linux "strings" tool. Here I found out that indeed the path to load the library from "/home/User/tensorflowtest/app/src/main/cpp/../../../../distribution/tensorflow/lib/x86/libtensorflowLite.so" was set.
Adding a "-Wl,-soname=libtensorflowLite.so" to the build options in the BUILD file fixed this issue! You can find the whole rule I used below.
As it was a pain to get everything set up due to the lack of explanations (it seems TensorflowLite is mostly used via Java API on Android ?), I want to give a short guidance on how use the C++ API of TensorflowLite in Android Studio (from within an Android NDK project).
1. Build the library for your architecture
To use the C++ API, you first need to build the TensorflowLite library. For this, add the following rule to the BUILD file in tensorflow/contrib/lite:
cc_binary(
name = "libtensorflowLite.so",
linkopts=[
"-shared",
"-Wl,-soname=libtensorflowLite.so",
],
linkshared = 1,
copts = tflite_copts(),
deps = [
":framework",
"//tensorflow/contrib/lite/kernels:builtin_ops",
],
)
Note: With this, a shared library can be built! A static one might also work.
Now you can build the library using
bazel build //tensorflow/contrib/lite:libtensorflowLite.so --crosstool_top=//external:android/crosstool --cpu=arm64-v8a --host_crosstool_top=#bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11"
If you want to support multiple architectures, you will have to build the library several times and change the --cpu flag correspondingly.
NOTE: This works fine at least for arm64-v8a and the armeabi-v7a (haven't tested it with MIPS so this might work aswell). However on an x86 device, I get the "atomic_store_8" error already adressed in this topic: https://github.com/tensorflow/tensorflow/issues/16589
2. Add the library and the needed headers to be included in your Android Studio project
Having built the library, you now need to make sure it also is linked into your Application (more specifically: Into your Android NDK library, which in my case is named "native-lib"). I will give a short overview on how to do this, however if you need a more detailed explanation you may refer to the github link I provided in my initial question: https://github.com/googlesamples/android-ndk/tree/840858984e1bb8a7fab37c1b7c571efbe7d6eb75/hello-libs
2.1. In your Android Studio Project, open the CMakeLists.txt
2.2. Add the following:
# This will create a new "variable" holding the path to a directory
# where we will put our library and header files.
# Change this to your needs
set(distribution_DIR ${CMAKE_SOURCE_DIR}/distribution)
# This states that there exists a shared library called libtensorflowLite
# which will be imported (means it is not built with the rest of the project!)
add_library(libtensorflowLite SHARED IMPORTED)
# This indicates where the libtensorflowLite.so for each architecture is found relative to our distribution directory
set_target_properties(libtensorflowLite PROPERTIES IMPORTED_LOCATION
${distribution_DIR}/lib/${ANDROID_ABI}/libtensorflowLite.so)
# This indicates where the header files are found relative to our distribution dir
target_include_directories(native-lib PRIVATE
${distribution_DIR}/include)
# Finally, we make sure our libtensorflowLite.so is linked to our native-lib and loaded during runtime
target_link_libraries( # Specifies the target library.
native-lib
libtensorflowLite
# Links the target library to the log library
# included in the NDK.
${log-lib} )
2.3. Open the build.gradle for your Module: App (not the project one!)
2.4. Make sure our library will be packed into your APK
Add this inside the Android section:
sourceSets {
main {
// let gradle pack the shared library into apk
jni.srcDirs = []
jniLibs.srcDirs = ['distribution/lib']
}
}
You may have to edit the path accoding to your needs: The files here will be packed in to your .apk inside the lib directory.
3. Include flatbuffers
TensorflowLite uses the flatbuffers serialization library. I guess this will be added automatically if you build your project using bazel. But this is not the case when using Android Studio.
Of course, you could also add a static or shared library too.
However, for me it was easiest to just let flatbuffers compile each time with the rest of my app (it is not that big).
I copied all of the flatbuffers *.cpp source files to my project and added them to the CMakeLists.
4. Copy the needed headers for TensorflowLite and flatbuffers
In 3. I just copied the cpp files to my project.
However, the header files need to be located in the directory we set in target_include_directories in step 2.2.
So go ahead and copy all of the flatbuffers (from the flatbuffers repository) *.h files to this directory.
Next, from the TensorflowLite repository, you need all header files inside the tensorflow/contrib/lite directory. However you should keep the folder structure
For me it looks like this:
distribution
lib
arm64-v8a
libtensorflowLite
armeabi-v7a
libtensorflowLite
include
flatbuffers
tensorflow
contrib
lite
kernels
nnapi
schema
tools
So, if I haven't forgotten anything everything should be set up correctly by now!
Hopefully this helped and it worked for you as it did for me ;)
Best regards,
Martin
Related
To make this a little more complicated I'm loading a native library I've compiled with ndk and cmake into Unity which will be deployed on Android. This was all working fine until I wanted to link another shared library to the library I was building. Now my library fails to load and I believe it is because it can't find the new shared library. I can say that the new shared library works on its own because I'm using it in another NDK project.
I see that Unity copies the new .so library over to the apps install directory and puts it along with my original library in a /data/appname/lib/arm64 directory. I have also tried installing the library in /system/lib64 but no luck. I have compiled my library without the new shared library and it works, and then only modified cmake to add it back in and my library will no longer load. I have also objdump'd it to make sure it is x64 as well as exported the symbols in case it was corrupted. So I'm wondering, is there a way to find out where my library file is looking?
This is how I include the new shared library in my CmakeList.txt
add_library(libusb SHARED IMPORTED)
set_target_properties(libusb PROPERTIES IMPORTED_LOCATION D:/projects/arm64-v8a/libusb1.0.so)
where D:\projects\arm64-v8a is the location of the library file on my build machine. I wonder if this is screwing things up. But I use the same syntax in my Android studio Cmake project and I can access the library but I'm sure that's different somehow. Any help is always appreciated.
Thank you
The imported library may have its own dependencies, which must also be copied to your APK (or AAR). To check this, run objdump -p libusb1.so | findstr "NEEDED".
It will look in the lib/[platform-abi] directory of your app. But there is a list of steps for using prebuilt libraries:
https://developer.android.com/ndk/guides/prebuilts
Have you checked those?
I am trying to build Qt application for android using Qt Creator. I use CrystaX NDK for android instead of goolge's one because I need to use boost libraries in my project, and, as CrystaX's official site says, it is comes with it.
I am using following versions of tools:
Qt Creator 3.4.2
Qt 5.5.0
CrystaX NDK 10.2.1
At first, I had to manually add libraries and headers paths in my .pro file, because it hasn't been found automatically. There was compiler error: can't locate libcrystax, there was some errors in source files about including boost headers. I've added following lines in my project file:
android {
INCLUDEPATH += $$NDK_ROOT/sources/crystax/include \
$$NDK_ROOT/sources/boost/1.58.0/include \
$$PWD/ssl
LIBS += -L"$$PWD/ssl" -lssl -lcrypto
LIBS += -L"$$NDK_ROOT/sources/crystax/libs/$$ANDROID_TARGET_ARCH"
ANDROID_EXTRA_LIBS = $$NDK_ROOT/sources/crystax/libs/$$ANDROID_TARGET_ARCH/libcrystax.so
ANDROID_PACKAGE_SOURCE_DIR = $$PWD/android
}
After rebuilding again, I've got an error (runtime error) that says libgnustl_shared requires libcrystax, but libcrystax is not loaded or something similar.
After searching the internet, I've found that it's happening because one library that requires another, is loading before it, and that second library, is not being searched for, at application directory, only in system paths.
I've found a workaround - to load required library manually. I copied default QtActivity.java into my project directory (android/src/.../QtActivity.java) to replace default one and added following code:
static {
System.loadLibrary( "crystax" );
}
After that I am not getting that error, but now I'm stuck with another one:
java.lang.UnsatisfiedLinkError: Cannot load library: reloc_library[1285]: 37 cannot locate '__aeabi_ldiv0'...
Is it possible to use Qt + CrystaX NDK for build android app? Am I doing it wrong way? Please, explain how to do it properly, if I've mistaken or if I misunderstood whole concept. Any help is appreciated.
This happens because you haven't linked with libgcc.a. I don't know how exactly your build system works (well, Qt's one), but generally, adding libgcc.a to the list of additional libraries should help:
ANDROID_EXTRA_LIBS = $$NDK_ROOT/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/lib/gcc/arm-linux-androideabi/4.9/libgcc.a
This line specify arm variant of libgcc.a; obviously, you should use proper one depending on ANDROID_TARGET_ARCH value.
Does anyone have instructions for building the Point Cloud Library (PCL) for Android? I found a few superbuilds of PCL that claim to be able to build PCL and its dependencies. I tried the superbuild from http://www.hirotakaster.com/weblog/how-to-build-pcl-for-android-memo/. I'm using Ubuntu 14.10, Android 19, NDK r10d, and PCL 1.6.0, but I'm willing to use any versions. I'm also using the terminal for compiling. For Android hardware, I'm using a Project Tango.
I tried using android-cmake (http://code.google.com/p/android-cmake/), but I'm not sure how to build the proper toolchain. I continually receive the error "Could not find any working toolchain in the NDK. Probably your Android NDK is broken." I get this error with plain cmake and ccmake, too.
Does anyone have any detailed instructions for building PCL for Android (e.g., a bash script or terminal instructions)? Or, does anyone have a link to pre-built libraries?
(Caveat Emptor: This is not a long term solution)
I was able to get past the CMAKE error by editing the cmake file
pcl-superbuild/toolchains/toolchain-android.cmake
These two changes should get past the errors mentioned above:
set( ANDROID_NDK_HOST_SYSTEM_NAME "linux-x86" ) # Line 468
Should be
set( ANDROID_NDK_HOST_SYSTEM_NAME "linux-x86_64" )
This will generate another error, unless you change the following line (Line 1023)
if( ANDROID_NDK_RELEASE STRGREATER "r8" ) # r8b
Should be
if( ANDROID_NDK_RELEASE STRGREATER "r10" ) # r8b
The first change adds _64 to x86_64. The second adds compatibility for r10d.
This doesn't fix all of the errors though, because BOOST threads don't place nicely with GCC 4.6+. Implement the patch shown at this link (https://svn.boost.org/trac/boost/ticket/6165).
Again, that still doesn't fix all of the errors. (I haven't figured out why this is needed yet, math.h shouldn't need std::). In the file,
pcl-superbuild/CMakeExternals/Source/pcl/common/include/pcl/pcl_macros.h
Edit lines 99-102:
# define pcl_isnan(x) isnan(x)
# define pcl_isfinite(x) isfinite(x)
# define pcl_isinf(x) isinf(x)
They should be:
# define pcl_isnan(x) std::isnan(x)
# define pcl_isfinite(x) std::isfinite(x)
# define pcl_isinf(x) std::isinf(x)
PCL still generates many warnings, but at least it compiles (so far)
**EDIT: **
This doesn't get you all the way (unfortunately) because the boost libraries don't play well with C++11.
To fix this, download boost 1.55 from http://sourceforge.net/projects/boost/files/boost/1.55.0/, and overwrite the boost directory
pcl-superbuild/CMakeExternals/Source/boost/boost_1_45_0
(This directory gets created when you run make for the first time).
Next, modify
pcl-superbuild/CMakeExternals/Source/boost/CMakeLists.txt
and find the line:
file(GLOB lib_srcs ${boost_root}/libs/filesystem/v2/src/*.cpp)
replace it with
file(GLOB lib_srcs ${boost_root}/libs/filesystem/src/*.cpp)
That's as far as I've gotten
this link helps a lot, you can take a look. I also left some comments there..
http://www.hirotakaster.com/weblog/how-to-build-pcl-for-android-memo/
for compile pcl 1.6 with ndk r10d, you need to replace toolchain-android.cmake with the toolchain from opencv library
I built PCL and its dependencies using Ubuntu 14.10, 32-bit. I also had to use NDK r8c, 32-bit. The key to building Hirotakaster's superbuild was using a 32-bit os.
I managed to compile your super build with Ubuntu 15.10 64-bit and NDK r10e. I changed the toolchain-android.cmake with the one from OpenCV. Then in common.hpp(found in /Source/pcl/common/include/pcl/common/impl) I had to add the following lines:
# include < math.h >
# define pcl_isnan(x) std::isnan(x)
# define pcl_isfinite(x) std::isfinite(x)
# define pcl_isinf(x) std::isinf(x)
Hi I am trying to port an OpenGL desktop app to android. I have no knowledge of android development so am depending on Qt Creator to package the app. As part of the setup, I have invoked 'make-standalone-toolchain' script in android ndk with following settings
--platform=android-21
--toolchain=arm-linux-androideabi-4.9
--system=linux-x86_64
Then I used android-cmake and passed it the path of my newly created standalone-toolchain, which created libassimp.so, libassimp.so.3, and libassimp.so.3.1.1(ln) inside my assimp directory tree.
I passed the libassimp.so path to Qt creator project build menu under 'additional libraries'. However, on deploying the app on android, it crashes with error:
dlopen("/data/app/org.qtproject.example.a3dqtquick-2/lib/arm/lib3dqtquick.so", RTLD_LAZY) failed: dlopen failed: could not load library "libassimp.so.3" needed by "lib3dqtquick.so"; caused by library "libassimp.so.3" not found
I can even see the libassimp.so (not libassimp.so.3) file inside the project build directory at
../android-build/libs/armeabi-v7a.
Not sure where to go from here, manually placing libassimp.so.3 at this location does not sort out the problem. Thanks for reading. I will add further info on your feedback . please forgive any info deficiency as this is my first experiment with android.
Following is the deployment-settings.json file
"description": "This file is generated by qmake to be read by androiddeployqt and should not be modified by hand.",
"qt": "/home/ubashir/programs/Qt/5.4/android_armv7",
"sdk": "/home/ubashir/programs/android-sdk-linux",
"sdkBuildToolsRevision": "21.1.2",
"ndk": "/home/ubashir/programs/android-ndk-r10d",
"toolchain-prefix": "arm-linux-androideabi",
"tool-prefix": "arm-linux-androideabi",
"toolchain-version": "4.9",
"ndk-host": "linux-x86_64",
"target-architecture": "armeabi-v7a",
"qml-root-path": "/home/ubashir/code/3dqtquick",
"application-binary": "/home/ubashir/code/3dqtquickAndroid/lib3dqtquick.so"
UPDATE:
I have now tried this.. replace all links to assimp.so.3.1.1 with copies of the latter so now my library libassimp.so.3 is a file instead of link to libassimp.so.3.1.1. I manually added libassimp.so.3 to my project subfolder android/libs/aremabi-v71 --- no good. I confirm that my build directory shows all libassimp files as I manually added them so presumably they are being deployed but the error remains :
failed: dlopen failed: could not load library "libassimp.so.3" needed by "lib3dqtquick.so".
As outlined here http://webmail.dev411.com/p/gg/android-ndk/1386vger6e/use-assimp-c-library-in-ndk-ld-error-obj-local-armeabi-v7a-libassimp-so-incompatible-target-for-use-with-vuforia
I even edited the link.txt file after running cmake on my assimp build directory for android, altering the entry -soname,libassimp.so.3 with -soname,libassimp.so but it still creates libassimp.so.3.1.1 with its two links , i.e., libassimp.so.3 and libassimp.so. So still stuck..
I ran into the same problem with a shared library I built with CMake for and Android project. I found a way to fix it. There might be a cleaner solution if you were more familiar with CMake.
Search through the CMakeLists.txt file(s) for "SOVERSION" and "SET_TARGET_PROPERTIES()"
In the SET_TARGET_PROPERTIES() routine comment out the lines for VERSION and SOVERSION as follows
SET_TARGET_PROPERTIES(${PROJECT_NAME} PROPERTIES # create *nix style library versions + symbolic links
DEFINE_SYMBOL DSO_EXPORTS
# VERSION ${PROJECT_VERSION}
# SOVERSION ${PROJECT_SOVERSION}
CLEAN_DIRECT_OUTPUT 1 # allow creating static and shared libs without conflicts
OUTPUT_NAME "${PROJECT_NAME}${PROJECT_DLLVERSION}" # avoid conflicts between library and binary target names
)
Then rerun the configure and generate steps in CMake and rebuild the target. This should give you a .so without any version numbers.
I'll suggest you to take a look at the solution I've found to my problem (that is very similar to yours):
libgdal.so android error: could not load library "libgdal.so.1"
Hope this helps.
I wish to back port the Android RTP APIs introduced in version 3.1(Honeycomb) to earlier versions. I downloaded the source of version 4.0 and found that it these APIs had both java and native code. In order to build the native code with the NDK, certain shared libraries are required.
According the Android.mk file, these are libnativehelper, libcutils, libutils, and libmedia. Though the source of all of these are present in the source code, building them was difficult. Each required many other shared libraries. For eg, libmedia requires these shared libraries: libui, libcutils, libutils, libbinder, libsonivox, libicuuc, libexpat, libcamera_client, libstagefright_foundation, libgui and libdl.
So my question is, is there some way of obtaining the original 4 shared libs? Does it involve building the entire source?
Say I need to build a piece of native code which is going to use standard Android shared libraries such as libutils, libcutlis, libmedia. I would perform following steps:
Install AOSP repository with target version.
Add my source code to appropriate directories under ./frameworks/base. In your case it might be easier to create a separate folder and put proper Android.mk of course.
You might get compile errors if required functions from those standard shared libraries are not present in the previous version.
When you build the code as part of AOSP it will build required libraries and link them for you automatically.
P.S. To accomplish that you're better to use a Linux-based build host.
using cygwin terminal, build native part i.e. jni folder. To build using cygwin, goto jni folder using cygdrive command. Then type ndk-build. After successful completion, shared libraries i.e. .so files will be created in libs folder.
I can understand your problem, you can pull the libraries from /system/lib of device or emulator. But you need a system permission. But you can do it by installing application.
Otherwise build your source code on linux platfor. Building process is very easy, just using 2 or 3 command. First time it is needed long time to build. After that you need very short time to build, it will build only according to the timestamp of modified code.
Please have a look here