I have a model that implements sklearn's RandomForestClassifier. I have saved the model in a 'pickle' file and I want to deploy it to an android application.
I've seen that this can be accomplished using a Tensorflow Lite file, and a 'hdf5' file can be converted to the same. However I have no idea how to convert my pickle file to hdf5 and then to tflite.
Do I re-implement Random Forest using Tensorflow? What else can I do to deploy it to android?
I do not want to train my model on android, but use my custom trained model on it.
You can use m2cgen (Model 2 Code Generator), which is a lightweight library that can provide an easy way to transpile trained statistical models into a native code (Python, C, Java, Go, JavaScript, Visual Basic, C#, PowerShell, R, PHP, Dart, Haskell, Ruby, F#)
Related
This question is about finding a solution on how to run a trained model on an Android device without using the convert TF Lite and without using a external service.
I don't owned the model and cannot modify it. I just have the trained saved model files.
The device is out of network and should embed the trained model. No connection to an external server is possible.
Tensorflow Lite is not an option since TF Lite doesn't support 5D tensors: https://github.com/tensorflow/tensorflow/issues/56946
In order to do my test I will get the basic model I have provided in the above tensorflow issue to do my tests.
I have found this blog article, but didn't manage to make it work yet: https://medium.com/#vladislavsd/undocumented-tensorflow-c-api-b527c0b4ef6
Do you know any updated solution that enables to load the model inside a Java or C++ lib on Android?
No example is proposed by Tensorflow on their GitHub: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android
If TFLite doesn't work for your model due to the limited support, you can use Select TensorFlow ops feature. https://www.tensorflow.org/lite/guide/ops_select
It allows you to use TF ops in TFLite so you can overcome the limited 5D supports of TFLite but it impacts your binary size.
I have succeeded to deploy my trained model using 5D tensor on Android Emulator.
In order to do that, I have converted my model using the converter from Tensorflow to ONNX: https://github.com/onnx/tensorflow-onnx
python -m tf2onnx.convert --saved-model tensorflow-model-path --output model.onnx
Then I have created a C++ lib that loads the ONNX model from the converted file and calls it.
In order to copy the asset on the phone storage, I have followed this topic: https://stackoverflow.com/a/69941051/12851157
You can find ONNX samples here: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/c_cxx
And finally I have integrated the C++ lib in Android like this: https://github.com/android/ndk-samples/tree/master/hello-libs
If I have enough time, I will try to use the TF API.
I used the keras to create the LSTM model and I've already trained models that are stored as models.h5 files.
I want to know how to predict the model in the Android studio.
how can i solve this problem?
You can do this using deeplearning4j. You can set up deeplearning4j in Android Studio following the instructions here.
Trained Keras models can be imported directly into deeplearning4j using the Keras model import functionality described here. You can then perform predictions with the trained model in Java.
I searched to understand if there is a technique to keep a trained tensorflow model (.pb file) safe in an Android app but didn't find anything useful. I am releasing an app containing a tensorflow model which I built on a training set. When I release the app, anyone can access the model and use it for his own app. I wonder if there is a way to protect a tensorflow model that I put in the asset folder of my Android application?
This is the way that I load my model in Android:
TensorFlowInferenceInterface tf = new TensorFlowInferenceInterface();
tf.initializeTensorFlow(context.getAssets(), "file:///android_asset/model.pb");
I was thinking to embed the model encrypted in the app and decrypt it during runtime, but if someone debugs the app, it can get the password and decrypt it. Moreover, there is just one implementation of initializeTensorFlow method in the TensorFlowInferenceInterface class that just accepts (AssetManager assetManager, String model). It is possible to write one that accepts the encrypted one, but it needs some modification of Tensorflow C++ library. I wonder if there is a more reliable solution. Any suggestion, please?
As mentioned in the comments, there is no real safe way to keep your model safe when you run it locally. That being said, you can hide your model and make things a tad more difficult than having a .pb around.
Apart from name obfuscation provided by freeze_graph, a good solution is to compile to model to a binary using XLA AOT compilation using tfcompile. It generates a binary library containing your model as well as a header file to use it. Somebody who want to peek at your network would then have to go through compiled code, which is a higher bar to clear than reading a .pb file for most people.
I am trying to combine data process with android app development. I have trained a svm model in matlab using libsvm, and the android app I am working on can obtain data features from a device. What I want is to use this svm model in android studio to predict the new data coming in. Is there a way to only USE the model instead of implement the entire svm algorithm is android studio? Thank you :) p.s. The model is .mat file.
In order to make a prediction you need to save the support vectors x_i, the associated dual variables (the signed alpha's you get from libsvmrtrain) and the bias (-rho you get from libsvmrtrain). Then for making a prediction on a new sample x, you calculate:
where K is the kernel used for learning. Note that libsvm embeds the label y_i into the dual variables, so you can just use the alphas.
The easiest way is to save all the variables you need to a text file (or some binary file you can easily read on the mobile device).
I have an algorithm and some other code which is in MATLAB and I want to use it in my Android application.
How can I do this?
Can I make a jar file from MATLAB for use with Android?
I have to do something else?
If you have an additional product, MATLAB Builder JA for Java, you can produce a .jar file from your MATLAB code.
However, the .jar file requires the MATLAB Compiler Runtime (a freely redistributable component that you get with MATLAB Compiler and MATLAB Builder products) to be present. The MCR has a much larger footprint than is suitable for the typical Android device (it's like a copy of MATLAB itself, without the user interface).
You could think about either
Running your MATLAB .jar file remotely on a server, and having your Android application connect to it, or
Instead of using MATLAB Compiler and Builder products, use MATLAB Coder, which will convert a subset of the MATLAB language directly into C code. This C code doesn't require the MCR, and could be compiled to run directly on Android. Make sure your MATLAB algorithm falls within, or can be expressed in, the appropriate subset of the MATLAB language.
Edit: As of R2015a, functionality from MATLAB Builder JA for Java has been replaced by a new product, MATLAB Compiler SDK.
I am the developer of Addi. http://addi.googlecode.com Addi is quickly becoming a full port of Octave (which is an open source tool that uses Matlab syntax). Addi is going to have intents for other applications to use it as their math engines or plotting engines. So, if you can run your code on Octave, then very soon you will be able to run it on Android.
Our only option is to get C++ code from M code using MATLAB Coder toolbox, that generates standalone C and C++ code from MATLAB® code. It supports only some subset of all Matlab functions, therefore might be not suitable for your needs.
Having C code you can compile it using NDK. MATLAB Compiler is not an option here.
A new feature in Matlab 2014a:
http://www.mathworks.com/help/simulink/samsung-galaxy-android-devices.html
You can now directly install (limited set of) models to Samsung Android devices, and this should work actually on any Android device.
You can convert matlab code into python and then use the python code in the android .There are many tools to do this conversion. Python goes well with android than matlab.
You have 2 options,
Create a JAR and include in your Java Application and start using it. (I have not tested this by creating a JAR outside Eclipse)
You can code the same thing in C and use Android NDK to process it. (This will be faster and safer way)