Develop Reference

How to apply metadata on tflite model?

I'm trying to launch TF Object detection Android app (https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android) with a custom model
I need to fix this issue
java.lang.AssertionError: Error occurred when initializing ObjectDetector: Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images.
I found one suggestion, that I need to apply metadata on my .tflite model, so I tried to run
python tflite_convert.py \ --input_shapes="1,300,300,3" \ --input_arrays=normalized_input_image_tensor \ --output_arrays="TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3" \ --allow_custom_ops \ --saved_model_dir=alexey/saved_model \ --inference_input_type=FLOAT \ --inference_type=FLOAT \ --output_file=detect.tflite
And it was done without any errors, but when I launch the app with this generated .tflite I get the same error as without applying metadata.
So it seems to me that metadata was not applied

I had the same error today. I solved it by running this script. It generated a tflite and json file and I put both of them in the assets/models folder.
It's a good idea to just modify the flgs in the string than use command line parameters if you are on Windows 10 (just replace where it says none with the correct path)

Tensorflow: Building graph and label files form checkpoint file

I want to build the graph and labels file from inception-resnet-v2.ckpt file. I have already downloaded the check point file form
wget http://download.tensorflow.org/models/inception_resnet_v2_2016_08_30.tar.gz.
I want to replace the inception5h model in tensorflow: android camera domo app with inception-resnet-v2. which requires a MODEL_FILE and a LABEL_FILE .
Now I don't know how I can get a .pb file and a label files from a checkpoint file.
I am learning tensorflow, still at beginner level.

Not sure, what the label file is, but to convert a checkpoint into a .pb file (which is binary protobuf), you have to freeze the graph. Here is a script I use for it:
#!/bin/bash -x
# The script combines graph definition and trained weights into
# a single binary protobuf with constant holders for the weights.
# The resulting graph is suitable for the processing with other tools.
TF_HOME=~/tensorflow/
if [ $# -lt 4 ]; then
echo "Usage: $0 graph_def snapshot output_nodes output.pb"
exit 0
fi
proto=$1
snapshot=$2
out_nodes=$3
out=$4
$TF_HOME/bazel-bin/tensorflow/python/tools/freeze_graph --input_graph=$proto \
--input_checkpoint=$snapshot \
--output_graph=$out \
--output_node_names=$out_nodes
Here, proto is a Graph definition (text protobuf), and snapshot is a checkpoint.

You will need to optimize your model after you have frozen it.
Look at this great tutorial
For the labels you can get them here (credits to Hands-On Machine Learning with Scikit-Learn and TensorFlow )
bazel build tensorflow/python/tools:optimize_for_inference
bazel-bin/tensorflow/python/tools/optimize_for_inference \
--input=/tf_files/retrained_graph.pb \
--output=/tf_files/optimized_graph.pb \
--input_names=Mul \
--output_names=final_result

How are .java files in android_stubs_current_intermediates directory generated?

The Android build process generates(?) Java stubs for each of the classes in the android.jar, and stores them in the following directory:
./out/target/common/obj/JAVA_LIBRARIES/android_stubs_current_intermediates/src/
For example, the subdirectory java/lang/ of the above directory contains .java files corresponding to java.lang.* classes, and the subdirectory `android/app/' contains .java files corresponding to android.app.* classes. These .java files dont contain actual code, but just signatures with dummy bodies.
I am assuming that those .java files are generated from the actual source code using a tool. My question is, which is this tool, and is it usable outside of the Android build process?
I want to use that tool to generate stubs for non-Android Java classes.

The "stubs" here is the framework API stub generated by running javadoc tool.
In most cases, when we talk about stub file in Android, we mean the java file generated by aidl tool. For example see How to generate stub in android? - Stack Overflow
In particular, the Android build system contains a makefile called droiddoc.mk that can be used to generate documentation, java API stubs and API xml files, which actually calls javadoc.
droiddoc.mk is under build/core. In build/core/config.mk there is a variable named BUILD_DROIDDOC to make it easier to include the droiddoc.mk.
Look at the droiddoc.mk, it calls javadoc:
javadoc \
\#$(PRIVATE_SRC_LIST_FILE) \
-J-Xmx1280m \
$(PRIVATE_PROFILING_OPTIONS) \
-quiet \
-doclet com.google.doclava.Doclava \
-docletpath $(PRIVATE_DOCLETPATH) \
-templatedir $(PRIVATE_CUSTOM_TEMPLATE_DIR) \
$(PRIVATE_DROIDDOC_HTML_DIR) \
$(addprefix -bootclasspath ,$(PRIVATE_BOOTCLASSPATH)) \
$(addprefix -classpath ,$(PRIVATE_CLASSPATH)) \
-sourcepath $(PRIVATE_SOURCE_PATH)$(addprefix :,$(PRIVATE_CLASSPATH)) \
-d $(PRIVATE_OUT_DIR) \
$(PRIVATE_CURRENT_BUILD) $(PRIVATE_CURRENT_TIME) \
$(PRIVATE_DROIDDOC_OPTIONS) \
&& touch -f $#
There is nothing about the stub right? Don't worry, notice that there is a PRIVATE_DROIDDOC_OPTIONS variable, and
PRIVATE_DROIDDOC_OPTIONS := $(LOCAL_DROIDDOC_OPTIONS)
Many Android.mk files in AOSP, for example the framework/base/Android.mk, contain the include $(BUILD_DROIDDOC) to generate docs. In framework/base/Android.mk, there is a piece of code:
LOCAL_DROIDDOC_OPTIONS:=\
$(framework_docs_LOCAL_DROIDDOC_OPTIONS) \
-stubs $(TARGET_OUT_COMMON_INTERMEDIATES)/JAVA_LIBRARIES/android_stubs_current_intermediates/src \
-api $(INTERNAL_PLATFORM_API_FILE) \
-nodocs
LOCAL_DROIDDOC_CUSTOM_TEMPLATE_DIR:=build/tools/droiddoc/templates-sdk
LOCAL_UNINSTALLABLE_MODULE := true
include $(BUILD_DROIDDOC)
The LOCAL_DROIDDOC_OPTIONS contains a -stubs option. And it will finally put into the javadoc command used by droiddoc.mk.
However, we may notice that the javadoc doesn't contain any option like -stubs. The key is that you can customize the content and format of the Javadoc tool's output by using doclets. The Javadoc tool has a default "built-in" doclet, called the standard doclet, that generates HTML-formatted API documentation. You can modify or subclass the standard doclet, or write your own doclet to generate HTML, XML, MIF, RTF or whatever output format you'd like.
We can use the -doclet option to specify our customized doclet. And the javadoc command in droiddoc.mk use the -doclet com.google.doclava.Doclava. That doclet receives the -stubs option.
Look at the Doclava implementation under external/doclava/src/com/google/doclava/Doclava.java
else if (a[0].equals("-stubs")) {
stubsDir = a[1];
} else if (a[0].equals("-stubpackages")) {
stubPackages = new HashSet<String>();
for (String pkg : a[1].split(":")) {
stubPackages.add(pkg);
}
}
It receives the -stubs option. And here is how it process the stubsDir.
// Stubs
if (stubsDir != null || apiFile != null || proguardFile != null) {
Stubs.writeStubsAndApi(stubsDir, apiFile, proguardFile, stubPackages);
}
And trace the implementation of the Stubs.writeStubsAndApi, you can see why the content in the stub files are like that.
You can even write your own java files and generate your stubs like what the test cases under build/tools/droiddoc/test.

Dynamically linking to a shared Java Library in Android

I'm trying to separate a Java Library, that is used by multiple Android "services", into a dynamic or shared library that can be loaded by those independent services without having the library included into the APK of each service.
I know there are different ways of doing this like creating an Android Service or using DexLoader and Reflection but I'm trying to avoid changing the source of the library. Instead I'm trying to build it and install it on my device (essentially extending the provided android API).
The following is a very similar question which is still unanswered:
Create Android apps in Eclipse sharing common library
I know this is something Google doesn't want to disclose so finding information online is extremely difficult.
So far I've tried placing a simple "Hello World" program under the frameworks dir and build it which successfully created a jar for my program. Then I added my package in product/core.mk and in addition added my package definition under api/10.xml after which I ran "make sdk" which resulted in the following error message:
******************************
You have tried to change the API from what has been previously released in
an SDK. Please fix the errors listed above.
******************************
make: *** [out/target/common/obj/PACKAGING/checkapi-last-timestamp] Error 38
As a workaround I added my package into "public_api.xml" file, inside the out directory, which is somehow dynamically created during the build process. With this workaround the SDK is built with no errors (although if I do clean again I'll have to modify the public_api.xml again because it will be removed due to clean). However, when I try to import and use my package anywhere it still says that my package "does not exist"
Any help will be greatly appreciated! Thank you!

Finally figured it out. The solution turns out to be very simple!
Place your library in the frameworks/base folder and make sure all your source code is inside under java directory like so:
../frameworks/base/HelloWorld/java/<source files and folders>
Edit core.mk file located under build/target/product/ to include your package in the list. This will add HelloWorld library to the framework:
PRODUCT_PACKAGES := \
bouncycastle \
:
:
DefaultContainerService \
Bugreport \
HelloWorld
Edit pathmap.mk file located under build/core/ to include your directory in the list. This will add HelloWorld library to the android.jar
FRAMEWORKS_BASE_SUBDIRS := \
$(addsuffix /java, \
core \
graphics \
location \
media \
opengl \
sax \
telephony \
wifi \
vpn \
keystore \
voip \
HelloWorld \
)
Done. rebuild android and it should not complain and add your library to framework.jar!
I hope this helps.

Compiling and using OpenSSL for Android, on windows using eclipse

I am developing an Android app on windows, using eclipse.
I would like to use OpenSSL for my Android application. It needs to be used with C++ via NDK.
I tried to download fries' OpenSSL source code from https://github.com/fries/android-external-openssl and build it.
I read the README.android file, but I didn't understand what these instructions are for.
Do I need to download the OpenSSL source code in addition to the fries source code?
I have tried putting all the files from Fries' into my jni folder of the Android project. It complained about not finding e_os.h and openssl/crypto.h and others like that.
More specifically:
In file included from D:/Projects/Fatal/Android/OpenSSL/jni/ssl/s2_meth.c:59:
D:/Projects/Fatal/Android/OpenSSL/jni/ssl/ssl_locl.h:124:18: error: e_os.h: No such file or directory
D:/Projects/Fatal/Android/OpenSSL/jni/ssl/ssl_locl.h:126:28: error: openssl/buffer.h: No such file or directory
D:/Projects/Fatal/Android/OpenSSL/jni/ssl/ssl_locl.h:127:26: error: openssl/comp.h: No such file or directory
I am not sure why it doesn't find those files, I tried to add all sort of paths into LOCAL_C_INCLUDES in all sort of Android.mk files, but nothing worked.
I would like to know how to build OpenSSL for Android. Thank you.

I know this is old, but I kept coming across this when I was searching for this problem. In the case that you are building a standalone OpenSSL to go with your project and are getting this error, I found a solution as per this thread:
In openssl-android/crypto, openssl-android/ssl, and openssl-android/apps, you'll find a variable declaration for local_c_includes that is something like this:
local_c_includes := \
$(NDK_PROJECT_PATH) \
$(NDK_PROJECT_PATH)/crypto/asn1 \
$(NDK_PROJECT_PATH)/crypto/evp \
$(NDK_PROJECT_PATH)/include \
$(NDK_PROJECT_PATH)/include/openssl
If you've placed your openssl-android directory in your Android project in the project/jni directory, then these variables no longer point to the correct location. The way I solved it was to modify these paths to be relative to LOCAL_PATH:
local_c_includes := \
$(LOCAL_PATH)/.. \
$(LOCAL_PATH)/asn1 \
$(LOCAL_PATH)/evp \
$(LOCAL_PATH)/../include \
$(LOCAL_PATH)/../include/openssl
Remember, you'll have to do this for each of the directories you're trying to build (crypto, ssl, or apps).

I realized that the fries' github is just something to patch the source of OpenSSL, it's not a stand alone OpenSSL for android.
It is required because it provide Android.mk files which are required by ndk-build of ndk. You don't want to build it using standard gcc because you want to build it for Android.