I'm currently searching for a bug which is based on a OpenGL program being invalid. But it is difficult to find the source of the problem without knowing where it might come from.
When I create the program it is valid. Furthermore I don't use glDeleteProgram().
To determine wether my program is valid or not I use glIsProgram().
Generally in OpenGL, objects are not created until they are first bound.
glGenTextures (...) for instance, reserves and returns one or more names for texture objects but those names do not become actual textures until bound to something like glBindTexture (GL_TEXUTRE_2D, ...). In other words, the names are reserved but what they reference is not instantiated/initialized yet.
What glIs* (...) actually returns is whether the name you pass it is the name of a created object. Thus, if you never bind an object it is never created, and this function will return GL_FALSE.
glUseProgram (...) is the function that OpenGL uses to bind GLSL program objects. Even though GLSL program and shader objects work differently from all other types of OpenGL objects it is very likely that glIsProgram (...) is not going to return GL_TRUE until sometime after you have called glUseProgram (...) on it at least once.
Incidentally, to validate a program I would suggest you use glValidateProgram (...) instead.
Here is some C pseudo-code that demonstrates how to use glValidateProgram:
GLint valid = GL_FALSE;
glValidateProgram (program);
glGetProgramiv (program, GL_VALIDATE_STATUS, &valid);
If valid is GL_TRUE after this, your program is valid (e.g. it was successfully linked).
Related
TLDR: How to use Variables from frozen tensorflow graphs on Android?
1. What I want to do
I have a Tensorflow model that keeps an internal state in multiple variables, created with: state_var = tf.Variable(tf.zeros(shape, dtype=tf.float32), name='state', trainable=False).
This state is modified during inference:
tf.assign(state_var, new_value)
I now want to deploy the model on Android. I was able to make the Tensorflow example App run. There, a frozen model is loaded, which works fine.
2. Restoring variables from frozen graph does not work
However, when you freeze a graph using the freeze_graph script, all Variables are converted to constants. This is fine for weights of the network, but not for the internal state. The inference fails with the following message. I interpret this as "assign does not work on constant tensors"
java.lang.RuntimeException: Failed to load model from 'file:///android_asset/model.pb'
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.<init>(TensorFlowInferenceInterface.java:113)
...
Caused by: java.io.IOException: Not a valid TensorFlow Graph serialization: Input 0 of node layer_1/Assign was passed float from layer_1/state:0 incompatible with expected float_ref.
Luckily, you can blacklist Variables from being converted to constants. However, this also doesn't work because the frozen graph now contains uninitialized variables.
java.lang.IllegalStateException: Attempting to use uninitialized value layer_7/state
3. Restoring SavedModel does not work on Android
One last version I have tried is to use the SavedModel format which should contain both, a frozen graph and the variables. Unfortunately, calling the restore method does not work on Android.
SavedModelBundle bundle = SavedModelBundle.load(modelFilename, modelTag);
// produces error:
E/AndroidRuntime: FATAL EXCEPTION: main
Process: org.tensorflow.demo, PID: 27451
java.lang.UnsupportedOperationException: Loading a SavedModel is not supported in Android. File a bug at https://github.com/tensorflow/tensorflow/issues if this feature is important to you at org.tensorflow.SavedModelBundle.load(Native Method)
4. How can I make this work?
I don't know what else I can try. Here's what I would imagine, but I don't know how to make it work:
Figure out a way to initialize variables on Android
Figure out a different way to freeze the model, so that maybe the initializer op is also part of the frozen graph and can be run from Android
Find out if/how RNNs/LSTMs are implemented internally, because these should also have the same requirement of using variables during inference (and I assume LSTMs to be able to be deployed on Android).
???
I have solved this myself by going down a different route. To the best of my knowledge, the "variable" concept cannot be used in the same way on Android as I was used to in Python (e.g. you cannot initialize variables and then have an internal state of the network be updated during inference).
Instead, you can use placehlder and output nodes to preserve the state inside your Java code and feed it to the network on every inference call.
replace all tf.Variable occurances with tf.placeholder. The shape stays the same.
I also defined an additional node used to read the output. (Maybe you can simply read the placeholder itself, I haven't tried that.) tf.identity(inputs, name='state_output')
During inference on Android, you then feed the initial state into the network.
float[] values = {0, 0, 0, ...}; // zeros of the correct shape
inferenceInterface.feed('state', values, ...);
After inference, you read the resulting internal state of the network
float[] values = new float[output_shape];
inferenceInterface.fetch('state_output', values);
You then remember this output in Java to pass it into the 'state' placeholder for the next inference call.
I'm new to open gl in android and I need to draw some text in my GLSurfaceView. I found only one solution - to create bitmap with text and display it like texture (like this, for example; http://giantandroid.blogspot.ru/2011/03/draw-text-in-opengl-es.html). I tried to do like this, but it didn't work for me. What is textures array in this code? And is there any simplier way to display text?
openGL by itself doesn't offer any "simpler" way to render text - it even doesn't "know" anything about the way glyphs may be represented in bitmap or outline font sets.
Using some other library "knowing" how to handle different font sets and how to rasterize them to directly let them paint into an openGL texture doesn't seem so complicated that any other approach may claim to be a lot easier.
You might want to take a look at both this (see: Draw text in OpenGL ES) article here on stackoverflow as well as into the linked pages to get an overview of other methods available to choose the one that seems best to you.
Regarding your second question about the textures array in the code sample you linked the array is just used to fulfill the API requirements of the call to glGenTextures() as it expects an array as the second argument.
In fact just a single texture id is allocated in this line:
gl.glGenTextures(1, textures, 0);
Taking a look at the spec (see: http://docs.oracle.com/javame/config/cldc/opt-pkgs/api/jb/jsr239/javax/microedition/khronos/opengles/GL10.html) it turns out there is just a single texture id allocated stored at index 0 in the textures array.
Why then a (pseudo) array at all?
The seemingly complex signature of the java method
public void glGenTextures(int n, int[] textures, int offset)
is due to the fact that there are no pointers in java while the C function that gets wrappped requires a pointer as it's second argument:
void glGenTextures(GLsizei n, GLuint *textures);
As in C code allocating a single texture can be done with:
GLuint texture_id;
glGenTextures (1, &texture_id);
there is no need for a special function just returning a single texture id, so there's only a single glGenTextures() function available.
To allow for allocations that can be done with pointer arithmetic like:
GLuint textures[10];
glGenTextures (3, textures + 5);
in C the java wrapper allows for a third parameter specifying the starting index from which to assign allocated texture ids.
The corresponding method invocation in java would look like this:
int[] textures = new int[10];
glGenTextures (3, textures, 5);
Thus one more parameter in java wrapper and the requirement to use an array even if only a single id is needed.
I am using OpenGL ES 1.1 with the NDK and occasionally when I call glGenTextures it does not change the value of name holder that I pass in (doesn't even set it to 0).
glGetError returns 0, no error.
All GL code is in a JNI method called from onDrawFrame of the surface renderer so the context shouldn't be a problem. (edit: this was a wrong assumption and was the cause of the problem)
The code works in some cases and not others. If I repeat the call each frame it works after about 5 tries. (I am generating glyphs on request).
The first lot of textures get created up to at least #32 but after that it is hit and miss.
Does anyone know of a reason why glGenTextures would appear to do nothing?
Is glGenTextures called with a valid OpenGL(-ES) context being active? If you're using multiple threads: A OpenGL(-ES) context can be active in only one thread at a time. But each thread may have a different context active.
I have got some strange and unexpected results from my program in OpenGL ES for android for example in the code below:
matrix = ThisRot.get();
gl.glMultMatrixf(matrix, 0);
currentRotation.toMatrix(matrix);
temp.set(matrix);
I set the matrix value before I use it as an argument for gl.glMultMatrixf and after that I change the value of matrix and use it for another purpose, but it has effect an the way the object rotate so it should have effect on gl.glMultMatrixf(). and that's not the only one, some other places in my code I had this unexpected results. so I have thought maybe these happen due to mutual exclusion and multitreading and those kind of things.
am I right? should we worry about multithreading when we code in Opengl ES for android? How can I avoid these kind of problems.
Of course you should worry about multithreading. In particular, Android creates its own GLThread for rendering, when you attach a GLRenderer-derived class to a GLSurfaceView using the setRenderer() function.
In fact, multithreading can cause crashes (not only unexpected behavior) in your programs especially when you loop through arrays adding/removing objects and such.
Check if you are modifying the same data inside the onDrawFrame function of your GLRenderer and your own thread. If you are, try adding the following around the modified code (in both threads):
synchronize(variable) {
modify(variable);
}
This will lock the variable throughout the modify() function until it ends the block. Try not to overuse it, though, only in places where you need it. One thread will block the other one until it's finished!
I'm trying to find a way to check to see if a current EGLContext exists and is ready to use on Android. By specification, I've tried using
((EGL10)EGLContext.getEGL()).eglGetCurrentContext()
and then comparing it to EGL10.EGL_NO_CONTEXT (tried .equals() and != ). However, even though through debugging it 'seems' that it is returning an instance of 'EGL_NO_CONTEXT' (seems meaning all the internal values are uninitialized) however no matter what comparison I do I can't get it to work.
Anyone know of another/proper method to get this done? I don't want to do it by throwing a random GL call and catching the EGLError...
I ran into the problem of not being able to re-use the current EGLContext when trying to render what was on screen in a GLSurfaceView to an offscreen EGLPixelBufferSurface. From what I can tell, the problem with using the static method
EGLContext.getEgl()
is that it creates a default EGL instance - this would mean that the EGLContext associated with it is equivalent to EGL10.EGL_NO_CONTEXT.
Also, in Android the EGLContext can only be associated with one thread (Android developer Romain Guy says so here). So in order to properly use
EGL.getCurrentContext()
you would have to have a pre-existing EGL instance and call the getCurrentContext() method in the thread that created the EGLContext.
NOTE: Android now handles saving the EGLContext when the GLThread is paused/resumed in the GLSurfaceView class (take a look at the setPreserveEGLContextOnPause(boolean preserveOnPause) method).
There seems to be a bug in Android's implementation of EGL10.eglGetCurrentContext(), where the result of eglGetCurrentContxt() has to be compared using
result.equals(EGL10.EGL_NO_CONTEXT)
rather than
result == EGL10.EGL_NO_CONTEXT
For example:
if (((EGL10) EGLContext.getEGL()).eglGetCurrentContext().equals(EGL10.EGL_NO_CONTEXT)) {
// no current context.
}
You could try testing it to see if it is null, rather than equal to a given context. This is what I would do in a standard opengl program.
[EDIT] There's an example here which uses it as follows:
if ((eglGetCurrentContext () != context->egl_context) ||
(eglGetCurrentSurface ( EGL_READ ) != drawable->egl_surface))
I don't know if that's any help.