Develop Reference

How to apply metadata on tflite model?

I'm trying to launch TF Object detection Android app (https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android) with a custom model
I need to fix this issue
java.lang.AssertionError: Error occurred when initializing ObjectDetector: Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images.
I found one suggestion, that I need to apply metadata on my .tflite model, so I tried to run
python tflite_convert.py \ --input_shapes="1,300,300,3" \ --input_arrays=normalized_input_image_tensor \ --output_arrays="TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3" \ --allow_custom_ops \ --saved_model_dir=alexey/saved_model \ --inference_input_type=FLOAT \ --inference_type=FLOAT \ --output_file=detect.tflite
And it was done without any errors, but when I launch the app with this generated .tflite I get the same error as without applying metadata.
So it seems to me that metadata was not applied

I had the same error today. I solved it by running this script. It generated a tflite and json file and I put both of them in the assets/models folder.
It's a good idea to just modify the flgs in the string than use command line parameters if you are on Windows 10 (just replace where it says none with the correct path)

How to cross-compile Stockfish 13 from windows 10 to armv8?

Stockfish 13, released February 19, 2021, is the strongest chess engine at the moment I am writing these lines. You can download it for windows, linux, mac os, and also android. Yet at the moment I am writing these lines, the android binaries download link points to Stockfish 12 and not 13. (This happens often with Stockfish, as the android binaries are not the ones people need the most.) As Stockfish is open source (github here), how could we cross-compile Stockfish 13 on a windows 10 computer for an Android phone ? (In my case, a Samsung Galaxy S21 Ultra running Android 11.)

We are going to need the source code : git clone https://github.com/official-stockfish/Stockfish.git (see the right address at Stockfish's github as this address could change) somewhere on your PC. (You need the command line utilities of Tortois Git for instance.) Let's call StockFishDIr the full path to the StockFish folder (that contains src as subfolder) on the PC.
We also need a cross-compiler to ... cross-compile (!) the source-code on the PC into an executable targeting the Android OS, meaning : being executable on the ANdroid OS, i.e. on the phone. We are going to use Android-NDK, which contains toolchains and cross-compilers. We download the "Latest Stable Version" r22 (as of the moment I am writing these lines) and unzip it to a folder on the PC ; I call "AndroidNdkDir" the full path to this directory (containing build, meta, prebuilt, toolchains etc folders) on the PC.
Finally, as in Stockfish's source code there is only a Makefile, we are going to need make under windows. Yes. Just get it from GnuWin32 and note MakeDir the installation directory on you PC, the one containg the bin folder containing make.exe. Analyzing the MakeFile linearly show that the cross-compilation boils down to only one command line provided some file is downloaded, and to another "optimizing" command line.
Now we look at the MakeFile from StockFishDIr\src ; we will modify it slightly.
First we change the executable name by changine the line
EXE = stockfish
into
EXE = stockfish_13_armv8_xcomp_win10x64
to give it a meaningful name.
Second, we locate the occurence of armv8 (it is the ARCH of the Samsung Galaxy S21 Ultra ; if you have another Android phone, maybe you should look for 'armv7' and adapt consequently what follows) concerning the compiler : namely, we replace the bit
ifeq ($(arch),armv8)
CXX=aarch64-linux-android21-clang++
STRIP=aarch64-linux-android-strip
endif
with
ifeq ($(arch),armv8)
CXX=AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android30-clang++.cmd
STRIP=AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-strip.exe
endif
endif
(Note that there was the initial android21 in the CXX line that I replaced with android30. It is because the Samung Galaxy s21 Ultra runs Android 11 whose real version number is android30. If you have an android phone with another android version, adapt the lines consequently.)
Now, before compiling, we need one last piece : the net file (technical, net = networks = neural network = the weights file of the neural network Stockfish operates with). "Normally" (ah ah), under a linux setup (all what I do here works almost mutatis mutandis on Linux), we would just do a "MakeDir\bin\make.exe" profile-build ARCH=x86-64 which would download the net file where needed, and we would not even bother we it. Sadly, running here "MakeDir\bin\make.exe" profile-build ARCH=x86-64 doesn't work for some reason, and we need to download the net file ourselves manually. Hence we have to look in the Makefile to find how the net file is downloaded. The bit that matters in the Makefile is :
# evaluation network (nnue)
net:
$(eval nnuenet := $(shell grep EvalFileDefaultName evaluate.h | grep define | sed 's/.*\(nn-[a-z0-9]\{12\}.nnue\).*/\1/'))
#echo "Default net: $(nnuenet)"
$(eval nnuedownloadurl := https://tests.stockfishchess.org/api/nn/$(nnuenet))
$(eval curl_or_wget := $(shell if hash curl 2>/dev/null; then echo "curl -skL"; elif hash wget 2>/dev/null; then echo "wget -qO-"; fi))
#if test -f "$(nnuenet)"; then \
echo "Already available."; \
else \
if [ "x$(curl_or_wget)" = "x" ]; then \
echo "Automatic download failed: neither curl nor wget is installed. Install one of these tools or download the net manually"; exit 1; \
else \
echo "Downloading $(nnuedownloadurl)"; $(curl_or_wget) $(nnuedownloadurl) > $(nnuenet);\
fi; \
fi;
$(eval shasum_command := $(shell if hash shasum 2>/dev/null; then echo "shasum -a 256 "; elif hash sha256sum 2>/dev/null; then echo "sha256sum "; fi))
#if [ "x$(shasum_command)" != "x" ]; then \
if [ "$(nnuenet)" != "nn-"`$(shasum_command) $(nnuenet) | cut -c1-12`".nnue" ]; then \
echo "Failed download or $(nnuenet) corrupted, please delete!"; exit 1; \
fi \
else \
echo "shasum / sha256sum not found, skipping net validation"; \
fi
We see that the address of the file is https://tests.stockfishchess.org/api/nn/$(nnuenet)) and that the variable nnuenet is defined by greping the string EvalFileDefaultName inside the header file evaluate.h (which is inside StockFishDIr\src), the line of interest being :
#define EvalFileDefaultName "nn-62ef826d1a6d.nnue"
Hence the variable nnuenet is equal to the string "nn-62ef826d1a6d.nnue" and the address we are going to download the nets from manually is https://tests.stockfishchess.org/api/nn/nn-62ef826d1a6d.nnue.
Of course, this address may change in the future, but you see the idea. Now we place the net file inside StockFishDIr\src
Now we are ready to cross-compile, with the following command to execute inside StockFishDIr\src :
"MakeDir\bin\make.exe" stockfish_13_armv8_xcomp_win10x64 COMP=ndk COMPCC="AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android30-clang --target=aarch64-linux-android30" ARCH=armv8
(Adapt here also the aarch64-linux-android30 bits according to your Android phone.)
When the preceeding command executes properly it produces the stockfish_13_armv8_xcomp_win10x64 binary inside the StockFishDIr\src folder. This binary is already usuable on you phone (by putting it inside the DroidFish\uci folder of your DroidFish app, for instance) but is not optimal as the size is big and the execution is a little bit slow. Hence we have to "strip" the binary with the following command line, still inside the StockFishDIr\src folder :
"AndroidNdkDir\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-strip" stockfish_13_armv8_xcomp_win10x64
(There's no android30 anymore but there is still a aarch64 : adapt here also the aarch64-linux-android30 bits according to your Android phone.)
Now, your binary file stockfish_13_armv8_xcomp_win10x64 is perfectly usable. You can play chess against StockFish 13 on your phone, and suffer ! ;)

Tensor Flow Could not load custom protobuf files in

I have just created a protobuf file (.pb file) for my own custom images using a TensorFlow tutorial.
But when I replaced the same file into the assets folder in tensorflow/examples/android/assets and try to build and generate an APK, the APK gets generated, but when I run the APK in an Android device, the APK crashes.
If I run the classify_image from Python, it gives me proper results.
Appreciate any help.

Since DecodeJpeg isn't supported as part of the core, you'll need to strip it out of the graph first.
bazel build tensorflow/python/tools:strip_unused && \
bazel-bin/tensorflow/python/tools/strip_unused \
--input_graph=your_retrained_graph.pb \
--output_graph=stripped_graph.pb \
--input_node_names=Mul \
--output_node_names=final_result \
--input_binary=true
Change few parameters in this file
/tensorflow/examples/android/src/org/tensorflow/demo/TensorFlowImageListener.java
The input sizes need to be 299, not 224. You'll also need to change the mean and std values both to 128.
INPUT_NAME to "Mul:0" ,
OUTPUT_NAME to "final_result:0"
after which you will be able to compile the apk.
Good Luck

Entangled with Google test

I am having trouble to get up and running with Google test. I have read the suggested steps from Google, I also looked a previous post, and read some other examples, but it doesn't clear much things up.
To keep things simple, I am trying the suggested example from Google test that is available from the directory in the Android ndk - sample1:
// main.cpp
#include <QtGui/QGuiApplication>
#include "qtquick2applicationviewer.h"
#include "gtest/gtest.h"
int main(int argc, char *argv[])
{
QGuiApplication app(argc, argv);
testing::InitGoogleTest(&argc, argv);
QtQuick2ApplicationViewer viewer;
viewer.setMainQmlFile(QStringLiteral("qml/factorial/main.qml"));
viewer.showExpanded();
return RUN_ALL_TESTS();
}
// sample1_unittest.cpp
#include <limits.h>
#include "sample1.h"
#include "gtest/gtest.h"
// Tests factorial of 0.
TEST(FactorialTest, Zero) {
EXPECT_EQ(1, Factorial(0));
}
The files sample1.h, sample1.cpp are also in the project, which contain the factorial function. Google test was equally informed to the project file - factorial.pro:
INCLUDEPATH +=
/opt/android-studio/ndk/sources/third_party/googletest/googletest/include
When I press [Build > Build Project "factorial"] it gets the following error:
main.cpp:8: error: undefined reference to 'testing::InitGoogleTest(int*, char**)'
sample1_unittest.cpp:17: error: undefined reference to 'testing::Test::Test()'
I am working with Ubuntu, QtCreator, Android and C++. Indeed I have spent already 3 days mocking around, but getting not much anywhere so far. Thus, I am posting here in hope some guru may give any hint on this. Any help will be mostly appreciated.

It seems you haven't built Google Test from what you describe. You need to compile the project into a library and then link against it. If you have CMake installed, then you have two options:
Use CMake's GUI (it's fairly intuitive) to generate the build system files, and then use those as usual (e.g. if you generate a Visual Studio solution, open the .sln file and build the project).
Use the command line to do the same thing; essentially you just create a new directory and do cmake <path-to-google-test> inside of it. The rest is the same.
You could also build the library by yourself. The distribution contains a folder named fused-src which should contain at least two files: gtest_main.cpp and gtest-all.cpp. Compile those files and you're done. You need to generate two libraries here: gtest out of gtest-all.cpp and gtest_main out of gtest_main.cpp.
Another alternative would be to get already built libraries. I've never searched for them, but they might be out there.

Try something like this:
$ g++ -I $GTEST_HOME/include -L $GTEST_HOME/lib -lgtest -lgtest_main -lpthread test.cpp
For more details:
How to setup googleTest as a shared library on Linux
If it still doesn't work, may find interesting to consider to use Makefile:
# A sample Makefile for building Google Test and using it in user
# tests. Please tweak it to suit your environment and project. You
# may want to move it to your project's root directory.
#
# SYNOPSIS:
#
# make [all] - makes everything.
# make TARGET - makes the given target.
# make clean - removes all files generated by make.
# Please tweak the following variable definitions as needed by your
# project, except GTEST_HEADERS, which you can use in your own targets
# but shouldn't modify.
# Points to the root of Google Test, relative to where this file is.
# Remember to tweak this if you move this file.
GTEST_DIR = ..
# Where to find user code.
USER_DIR = ../samples
# Flags passed to the preprocessor.
# Set Google Test's header directory as a system directory, such that
# the compiler doesn't generate warnings in Google Test headers.
CPPFLAGS += -isystem $(GTEST_DIR)/include
# Flags passed to the C++ compiler.
CXXFLAGS += -g -Wall -Wextra -pthread
# All tests produced by this Makefile. Remember to add new tests you
# created to the list.
TESTS = sample1_unittest
# All Google Test headers. Usually you shouldn't change this
# definition.
GTEST_HEADERS = $(GTEST_DIR)/include/gtest/*.h \
$(GTEST_DIR)/include/gtest/internal/*.h
# House-keeping build targets.
all : $(TESTS)
clean :
rm -f $(TESTS) gtest.a gtest_main.a *.o
# Builds gtest.a and gtest_main.a.
# Usually you shouldn't tweak such internal variables, indicated by a
# trailing _.
GTEST_SRCS_ = $(GTEST_DIR)/src/*.cc $(GTEST_DIR)/src/*.h $(GTEST_HEADERS)
# For simplicity and to avoid depending on Google Test's
# implementation details, the dependencies specified below are
# conservative and not optimized. This is fine as Google Test
# compiles fast and for ordinary users its source rarely changes.
gtest-all.o : $(GTEST_SRCS_)
$(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
$(GTEST_DIR)/src/gtest-all.cc
gtest_main.o : $(GTEST_SRCS_)
$(CXX) $(CPPFLAGS) -I$(GTEST_DIR) $(CXXFLAGS) -c \
$(GTEST_DIR)/src/gtest_main.cc
gtest.a : gtest-all.o
$(AR) $(ARFLAGS) $# $^
gtest_main.a : gtest-all.o gtest_main.o
$(AR) $(ARFLAGS) $# $^
# Builds a sample test. A test should link with either gtest.a or
# gtest_main.a, depending on whether it defines its own main()
# function.
sample1.o : $(USER_DIR)/sample1.cc $(USER_DIR)/sample1.h $(GTEST_HEADERS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1.cc
sample1_unittest.o : $(USER_DIR)/sample1_unittest.cc \
$(USER_DIR)/sample1.h $(GTEST_HEADERS)
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -c $(USER_DIR)/sample1_unittest.cc
sample1_unittest : sample1.o sample1_unittest.o gtest_main.a
$(CXX) $(CPPFLAGS) $(CXXFLAGS) -lpthread $^ -o $#
If you have to use Makefile to get gtest working, you probably may need to adjust the given template for your case, as you intend to build it to use with Android.

What is the proper syntax for getting a Makefile to print the output directory of one of its output zip files?

I'm trying to edit an Android Makefile in the hopes of getting it to print out the directory (path) location of one the ZIP files it creates. Ideally, since the build process is long and does many things, I would like for it print out the pathway to the ZIP file to a text file in a different directory I can access later:
Pseudo-code idea:
# print the desired pathway to output file
print(getDirectoryOf(variable-name.zip)) > ~/Desktop/location_of_file.txt
The Makefile snippet where I would like to insert this new bit of code is shown below. I am interested in finding the directory of $(name).zip (that is specific file I want to locate):
# -----------------------------------------------------------------
# A zip of the directories that map to the target filesystem.
# This zip can be used to create an OTA package or filesystem image
# as a post-build step.
#
name := $(TARGET_PRODUCT)
ifeq ($(TARGET_BUILD_TYPE),debug)
name := $(name)_debug
endif
name := $(name)-target_files-$(FILE_NAME_TAG)
intermediates := $(call intermediates-dir-for,PACKAGING,target_files)
BUILT_TARGET_FILES_PACKAGE := $(intermediates)/$(name).zip
$(BUILT_TARGET_FILES_PACKAGE): intermediates := $(intermediates)
$(BUILT_TARGET_FILES_PACKAGE): \
zip_root := $(intermediates)/$(name)
# $(1): Directory to copy
# $(2): Location to copy it to
# The "ls -A" is to prevent "acp s/* d" from failing if s is empty.
define package_files-copy-root
if [ -d "$(strip $(1))" -a "$$(ls -A $(1))" ]; then \
mkdir -p $(2) && \
$(ACP) -rd $(strip $(1))/* $(2); \
fi
endef

As part of your rule's action (that is, as one of the lines indented by a tab, or as a command that follows a semicolon):
#echo $(dir $(variable-name).zip)
If output to a file is desired,
#echo $(dir $(variable-name).zip) >~/Desktop/location_of_file.txt
and you can make the > a >> if you prefer to append to the file rather than to overwrite.
UPDATE
What does it mean, "As part of your rule's action (that is, as one of the lines indented by a tab, or as a command that follows a semicolon)"?
Answer: It means that you can #echo like this:
file-to-be-built: dependencies
some-command
#echo foo
another-command
Or like this:
file-to-be-built: dependencies
#some-command; \
echo foo; \
another-command
(note the placement of the #). Or like this:
file-to-be-built: dependencies; #some-command; echo foo; another-command
But not like this:
#echo foo
file-to-be-built: dependencies
some-command
another-command
(Here is yet another parenthetical statement, which you can ignore if you like: The #, as you may know, suppresses Make's copying of the command itself to standard output. You can omit it, of course, but if you include it then you must put it before the specific command you don't want Make to copy. Formally, the versions without semicolons issue multiple, separate commands in their own, separate environments, using separate invocations of the shell; whereas the versions with semicolons only invoke the shell once, with a single environment, and leave it to the shell to separate and execute the commands. Does this make sense? Maybe not, if you just read it -- I can't make much sense of such tangled verbiage, either, even though I wrote it, myself -- but do try it with the # in the various spots indicated and it should soon start to make sense to you. The # is no big deal in any case, but it's there to use to keep Make's output clean if you understand how to use it.)