Develop Reference

GLES20 calls in onDrawFrame causes long delay

I have written a game in which when onDrawFrame() is called I update the game state first (game logic and the buffers for drawing) and then proceed to do the actual drawings. On the Moto G and Nexus 7 everything works smoothly and each onDrawFrame() call takes only between 1-5ms. However, on the Samsung Galaxy S3, 90% of the time the onDrawFrame() call takes as long as 30-50ms to update.
Further investigating the issue I found the problem wholly lies on the first render method which I attach below:-
(Edit: the block is now on glclear(); having removed the unnecessary calls to get the handles, please refer to the comments)
public void render(float[] m, FloatBuffer vertex, FloatBuffer texture, ShortBuffer indices, int TextureNumber) {
GLES20.glVertexAttribPointer(mPositionHandle, 3, GLES20.GL_FLOAT, false, 0, vertex);
GLES20.glEnableVertexAttribArray(mPositionHandle);
GLES20.glVertexAttribPointer(mTexCoordLoc, 2, GLES20.GL_FLOAT, false, 0, texture);
GLES20.glEnableVertexAttribArray(mTexCoordLoc);
GLES20.glUniformMatrix4fv(mtrxhandle, 1, false, m, 0);
int mSamplerLoc = GLES20.glGetUniformLocation(fhGraphicTools.sp_Image, "s_texture");
GLES20.glUniform1i(mSamplerLoc, TextureNumber);
GLES20.glDrawElements(GLES20.GL_TRIANGLES, indices.capacity(), GLES20.GL_UNSIGNED_SHORT, indices);
// Disable vertex arrays used
GLES20.glDisableVertexAttribArray(mPositionHandle);
GLES20.glDisableVertexAttribArray(mTexCoordLoc);
}
The above method is called 5 times in each onDrawFrame() to draw things on different texture atlas. (the first 4 are actually the game's background which is 1 long rectangle). Having logged the time it takes in each line of the code I found that the lag I am having on the S3 always reside in one of the below lines:
int mPositionHandle = GLES20.glGetAttribLocation(fhGraphicTools.sp_Image, "vPosition");
or
GLES20.glDrawElements(GLES20.GL_TRIANGLES, indices.capacity(), GLES20.GL_UNSIGNED_SHORT, indices);
The lag only occurs in the first call of render() as the subsequent calls takes about 1-2ms each. I have tried disabling the first render() call but then the second which before only take 1-2ms now become the source of the lag, under the same lines.
Does anyone have an idea of what is wrong in my code which the S3 could not handle? It seem that the S3 have problem commencing GL calls at the beginning of each onDrawFrame(), but why is there such behavior is puzzling me. What can be done to decrease the "startup" delay?
Many thanks for your patience in reading.
Edited code to take Selvin's recommendation.

I have fixed the issue when I moved all my png image files from /drawable to /drawable-nodpi the game speed goes back to normal on the S3.
I assume the auto-scaling done by Android causes the raw bitmap I supplied to opengl in "bad" sizes causing unnecessary texture filtering work on each onDrawFrame, causing the call to miss out vsync and giving a long delay for the glClear while waiting for next vsync. Please correct me if I am wrong, still a newbie to graphics.

android - Can't read pixels from GraphicBuffer at adreno GPU, by Karthik's method(Hacky alternatives of glReadPixels)

Since July, I have developed Android Application to edit video files like .avi, .flv etc. I use FFMPEG and OpenGL ES 2.0 to implement this application.
Because it is required too many calculations to execute a filter effect like "Blur" by CPU, I decide to use OpenGl ES 2.0 for applying filter effect to a frame of video by using GPU and Shader.
What I try to do is 'Using shader to apply a filter effect to a frame of video and get pixels which are stored in Frame Buffer'.
So I have to use glReadPixels only OpenGl ES 2.0 method that can be used to get pixels from FrameBuffer. But according to many GPU Development Guides, using glReadPixels was not recommended and guide books warned the potential risk when using glReadPixels. Also, the performance of glReadPixels differs depending on GPU version and vendor. I cannot concretely decide to use glReadPixels and tried to find other method for getting pixels which is result of GPU calculation.
After a few days, I found the hacky method for getting pixels data by using Android GraphicBuffer.
Here is the link.
From this link, I tried Karthik's method to my codes.
Only difference is:
//render method I made.
void renderFrame(){
/* some codes to init */
glBindFramebuffer(GL_FRAMEBUFFER, iFBO);
/* Set the viewport according to the FBO's texture. */
glViewport(0, 0, mTexWidth , mTexHeight);
/* Clear screen on FBO. */
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// Different Code compare to Karthik's.
contents->setTexture();
contents->draw(mPositionVarIndex, mTextrueCoIndex);
contents->releaseData();
/* And unbind the FrameBuffer Object so subsequent drawing calls are to the EGL window surface. */
glBindFramebuffer(GL_FRAMEBUFFER,0);
LOGI("Read Graphic Buffer");
// Just in case the buffer was not created yet
void* vaddr;
// Lock the buffer and retrieve a pointer where we are going to write the data
buffer->lock(GRALLOC_USAGE_SW_WRITE_OFTEN, &vaddr);
if (vaddr == NULL) {
LOGE("lock error");
buffer->unlock();
return;
}
/* some codes that use the pixels from GraphicBuffer...*/
}
void setTexture(){
glGenTextures(1, mTexture);
glBindTexture(GL_TEXTURE_2D, mTexture[0]);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, mWidth, mHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, mData);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glGenerateMipmap(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, 0);
}
void releaseData(){
glDeleteTextures(1, mTexture);
glDeleteBuffers(1, mVbo);
}
void draw(int positionIndex, int textureIndex){
mVbo[0] = create_vbo(lengthOfArray*sizeOfFloat*2, NULL, GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, mVbo[0]);
glBufferSubData(GL_ARRAY_BUFFER, 0, lengthOfArray*sizeOfFloat, this->vertexData);
glEnableVertexAttribArray(positionIndex);
// checkGlError("glEnableVertexAttribArray");
glVertexAttribPointer(positionIndex, 2, GL_FLOAT, GL_FALSE, 0, BUFFER_OFFSET(0));
// checkGlError("glVertexAttribPointer");
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindBuffer(GL_ARRAY_BUFFER, mVbo[0]);
glBufferSubData(GL_ARRAY_BUFFER, lengthOfArray*sizeOfFloat, lengthOfArray*sizeOfFloat, this->mImgTextureData);
glEnableVertexAttribArray(textureIndex);
glVertexAttribPointer(textureIndex, 2, GL_FLOAT, GL_FALSE, 0, BUFFER_OFFSET(lengthOfArray*sizeOfFloat));
glBindBuffer(GL_ARRAY_BUFFER, 0);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, mTexture[0]);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 6);
checkGlError("glDrawArrays");
}
I use a texture and render frame to fill the Buffer. I have 2 Test Phones, One is Samsung Galaxy S 2, which renderer is Mali-400MP. The other is LG Optimus G Pro, and renderer is Adreno(TM) 320. Galaxy S2 works well with above code and Karthik's method. But in case of LG smartphone, there are some problems.
E/libgenlock(17491): perform_lock_unlock_operation: GENLOCK_IOC_DREADLOCK failed (lockType0x1,err=Connection timed out fd=47)
E/gralloc(17491): gralloc_lock: genlock_lock_buffer (lockType=0x2) failed
W/GraphicBufferMapper(17491): lock(...) failed -22 (Invalid argument)
Accroding to this link,
On Qualcomm hardware pre-Android-4.2, a Qualcomm-specific mechanism,
named Genlock, is used.
Only I could see the error related to GenLock, so I carefully guessed at some problem between GraphicBuffer and Qualcomm GPU. After that, I searched and read the code of Gralloc.cpp, GraphicBufferMapper.cpp, GraphicBuffer.cpp and *.h for finding reasons of those errors, but failed.
My questions are:
Is it right approach to get filter effect from GPU calculation? If not, how to get a filter effect like "Blur" which requires so many calculations?
Is Karthik's method not working for Qualcomm GPU? I want to know that why those errors occured only at Qualcomm GPU, Adreno.

Make sure your GraphicBuffer allocation has GRALLOC_USAGE_SW_READ_OFTEN specified. Without it you may not be able to lock the buffer from code running on the CPU.
Unrelated but possibly suggestive of a better approach: see the CameraToMpegTest example, which does a trivial edit to live camera input using a GLES 2.0 shader.
Update: there's now an example of applying filters with the GPU in Grafika. You can see a screenrecorded demo here.

Is optimization of Android's RenderBuffer possible?

I'm using RenderBuffer and OpenGL instead of Canvas and Bitmap on Android Java. However, drawing to a texture takes time. I referred to this question and tried speed improvement (Reduces the call of glEnableVertexAttribArray, glClearColor, glBindFramebuffer), but it's ineffective. Is this limitations of GPU and Android OS?
*This is an extract of the program.
GLES20.glAttachShader(program, vertex);
GLES20.glAttachShader(program, fragment);
GLES20.glLinkProgram(program);
GLES20.glUseProgram(program);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textures[i]);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGBA, s, s,
0, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, null);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fbs[i]);
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER,
GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, textures[i], 0);
GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, renders[i]);
GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER, GLES20.GL_DEPTH_COMPONENT16, s, s);
GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER,
GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER, renders[i]);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fbs[i]);
GLES20.glClearColor(fr, fg, fb, fa);
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
GLES20.glGetIntegerv(GLES20.GL_FRAMEBUFFER_BINDING, old, 0);
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fbs[pos]);
GLES20.glViewport(0, 0, ww, hh);
:
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
:
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, old[0]);
GLES20.glViewport(0, 0, scw, sch);

If, as I understand, you use all the code you pasted each time you need to draw something, it's not a surprise that your program is running slow.
OpenGL (wheter ES or not) is based on a "state machine" model: the OpenGL context retains the changes you make to it (using the various glXXX commands), and modify the way OpenGL will react to future commands (most notably to the ones you use for drawing).
This means that when you issue the command:
GLES20.glClearColor(fr, fg, fb, fa);
what you are really doing is set the
GL_COLOR_CLEAR_VALUE
variable inside the state machine to [fr, fg, fb, fa]. From that moment on, GL_COLOR_CLEAR_VALUE is already set to [fr, fg, fb, fa], and every extra call to glClearColor with the same arguments will change nothing in the machine.
OpenGL ES 2.0 glClearColor reference page
Why am I going into this? Because OpenGL state changes are performance critical (and some are more than others): changing the state of the context has a performance hit that often goes beyond a simple variable assignment, as it may require OpenGL to "talk" to the GPU, discard caches and whatnot. It's a necessary evil when done in a setup stage, but frequent changes might have a huge performance hit when done every frame.
Therefore the first thing you should do is to minimize the sheer number of glXXX command per second in your program by creating an initialization method and a drawframe method.
The standard approach is to use the GLSurfaceView.Renderer class
Android GLSurfaceView.Renderer class reference
cram all initialization commands into
public abstract void onSurfaceCreated (GL10 gl, EGLConfig config)
and
public abstract void onSurfaceChanged (GL10 gl, int width, int height)
and put in the method:
public abstract void onDrawFrame (GL10 gl)
the least possible amount of OpenGL commands (just what you need to actually draw stuff)
Leaving the theory aside and going into the specific, here is a list of optimizations I would suggest you to consider:
if you are using a single program, leave the whole:
GLES20.glAttachShader(program, vertex);
GLES20.glAttachShader(program, fragment);
GLES20.glLinkProgram(program);
GLES20.glUseProgram(program);
block in the setup section (and even if you don't, just switch between programs with glUseProgram: you don't have to link it over and over again!). glLinkProgram is likely to have a big performance hit!
texture initialization goes in the setup (on top of the "state change" thing you are actually moving the same very data from CPU memory to GPU memory over and over again, each frame)
framebuffer initialization (with all the renderbuffer creation and binding) goes in setup as well. Besides, why do you need multiple framebuffers anyway? :)
glViewport goes in the onSurfaceChanged method
it seems to me that you are using different framebuffers to draw different objects: I'm not sure of what you are trying to accomplish, but I'd suggest you to change strategy: draw to the whole screen and on the same buffer, unless you have a very good reason to do otherwise.
Last but not least, here is a good book on the topic:
OpenGL ES 2.0 Programming Guide
it's both a good introduction to OpenGL ES 2.0 and a nice introduction to OpenGL in general (I don't get a cut on their sales ;-) )
Hope it helps

Using VBOs/IBOs in OpenGL ES 2.0 on Android

I am trying to create a simple test program on android (API 10) using OpenGL ES 2.0 to draw a simple rectangle. I can get this to work with float buffers referencing the vertices directly, but i would rather do it with VBOs/IBOs. I have looked for countless hours trying to find a simple explanation (tutorial), but have yet to come across one. My code compiles and runs just fine, but nothing is showing up on the screen other than the clear color.
Here are some code chunks to help explain how I have it set up right now.
Part of onSurfaceChanged():
int[] buffers = new int[2];
GLES20.glGenBuffers(2, buffers, 0);
rectVerts = buffers[0];
rectInds = buffers[1];
GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, rectVerts);
GLES20.glBufferData(GLES20.GL_ARRAY_BUFFER, (rectBuffer.limit()*4), rectBuffer, GLES20.GL_STATIC_DRAW);
GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, rectInds);
GLES20.glBufferData(GLES20.GL_ELEMENT_ARRAY_BUFFER, (rectIndices.limit()*4), rectIndices, GLES20.GL_STATIC_DRAW);
Part of onDrawFrame():
GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, rectVerts);
GLES20.glEnableVertexAttribArray(0);
GLES20.glVertexAttribPointer(0, 3, GLES20.GL_FLOAT, false, 0, 0);
GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, rectInds);
GLES20.glDrawElements(GLES20.GL_TRIANGLES, 6, GLES20.GL_INT, 0);

I don't see anything immediately wrong, but here's some ideas you can touch on.
1) 'Compiling and running fine' is a useless metric for an opengl program. Errors are reported through actively calling glGetError and checking compile and link status of the shaders with glGet(Shader|Program)iv. Do you check for errors anywhere?
2) You shouldn't be assuming that 0 is the correct index for vertices. It may work now but will likely break later if you change your shader. Get the correct index with glGetAttribLocation.
3) You're binding the verts buffer onDraw, but I don't see anything about the indices buffer. Is that always bound?
4) You could also try drawing your VBO with glDrawArrays to get the index buffer out of the equation, just to help debugging to see which part is wrong.
Otherwise what you have looks correct as far as I can tell in that small snippet. Maybe something else outside of it is going wrong.

OpenGL Depth Buffer issue on Android

I am developing a 3D Rendering Engine for Android. I have experienced some issues with the depth buffer. I am drawing some cubes, one big and two small ones that will fall on top of the bigger one. While rendering I have seen that obviously something with the depth buffer is wrong, as seen in this screen shot:
This screen shot was taken on an HTC Hero (running Android 2.3.4) with OpenGL ES 1.1 The whole application is (still) targeted at OpenGL ES 1.1. It does look the same on the Emulator.
These are the calls in my onSurfaceCreated method in the renderer:
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
Log.d(TAG, "onsurfacecreated method called");
int[] depthbits = new int[1];
gl.glGetIntegerv(GL_DEPTH_BITS, depthbits, 0);
Log.d(TAG, "Depth Bits: " + depthbits[0]);
gl.glDisable(GL_DITHER);
gl.glHint(GL_PERSPECTIVE_CORRECTION_HINT, GL_FASTEST);
gl.glClearColor(1, 1, 1, 1);
gl.glClearDepthf(1f);
gl.glEnable(GL_CULL_FACE);
gl.glShadeModel(GL_SMOOTH);
gl.glEnable(GL_DEPTH_TEST);
gl.glMatrixMode(GL_PROJECTION);
gl.glLoadMatrixf(
GLUtil.matrix4fToFloat16(mFrustum.getProjectionMatrix()), 0);
setLights(gl);
}
The GL Call for the depth bits returns 16 on the device and 0 on the emulator. It would've made sense if it only didn't work on the emulator since there obviously is no depth buffer present. (I've tried setting the EGLConfigChooser to true, so it would create a Config with as close to 16 bits depth buffer as possible, but that didn't work on the emulator. It wasn't necessary on the device.)
In my onDrawFrame method I make the following OpenGL Calls:
gl.glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
gl.glClearDepthf(1);
And then for each of the cubes:
gl.glEnableClientState(GL_VERTEX_ARRAY);
gl.glFrontFace(GL_CW);
gl.glVertexPointer(3, GL_FIXED, 0, mVertexBuffer);
// gl.glColorPointer(4, GL_FIXED, 0, mColorBuffer);
gl.glEnableClientState(GL_NORMAL_ARRAY);
gl.glNormalPointer(GL_FIXED, 0, mNormalBuffer);
// gl.glEnable(GL_TEXTURE_2D);
// gl.glTexCoordPointer(2, GL_FLOAT, 0, mTexCoordsBuffer);
gl.glDrawElements(GL_TRIANGLES, mIndexBuffer.capacity(),
GL_UNSIGNED_SHORT, mIndexBuffer);
gl.glDisableClientState(GL_NORMAL_ARRAY);
gl.glDisableClientState(GL_VERTEX_ARRAY);
What am I missing? If more code is needed just ask.
Thanks for any advice!

I got it to work correctly now. The problem was not OpenGL. It was (as Banthar mentioned) a problem with the projection matrix. I am managing the projection matrix myself and the calculation of the final matrix was somehow corrupted (or at least not what OpenGL expects). I can't remember where I got the algorithm for my calculation, but once I changed it to the way OpenGL calculates the projection matrix (or directly call glFrustumf(...)) it worked fine.

try enabling:
-glDepthFunc(GL_LEQUAL)
-glDepthMask( true );