Develop Reference

Gles2.0 Overlay on Android

I'm trying to build an overlay for an Android application that uses GLESv2.
I've hooked eglSwapBuffers in order to insert my rending code just before the frame finishes.
I'm able to do simple things like drawing a square with the scissor test:
glEnable(GL_SCISSOR_TEST);
glScissor(0, 0, 200, 200);
glClearColor(1, 0, 0, 1);
glClear(GL_COLOR_BUFFER_BIT);
glDisable(GL_SCISSOR_TEST);
I've also had success drawing simple shapes with the following code, but as soon as I start using vertex attrib pointers the application stops rending correctly and shows a mostly-black screen with a small section that still displays correctly. I'm sure there's some open-gl state that I'm clobbering here but I'm not sure what it is. What would I need to save/restore before/after my draw calls in order to allow the app to continue to render correctly with my overlay?
// Save application state
GLint prev_program;
glGetIntegerv(GL_CURRENT_PROGRAM, &prev_program);
// Do overlay drawing
glUseProgram(program);
glVertexAttribPointer(vPosition, 2, GL_FLOAT, GL_FALSE, 0, RectangleVertices);
glEnableVertexAttribArray(vPosition);
glDrawArrays(GL_TRIANGLE_FAN, 0, 4);
glDisableVertexAttribArray(vPosition);
// Trying to restore application state here - there are probably more things that I'm missing.
glUseProgram(prevProgram);

What would I need to save/restore before/after my draw calls in order to allow the app to continue to render correctly with my overlay?
Everything that you modified ...
Note that even full state restoration can be inadequate for some developer use cases. Normally this is a bug in the application (application making assumptions about object ID assignments), but there are use cases in development tooling such as verbatim API trace replay tools where such assumptions are made.

Android canvas multiple draws causes flickering

the keywords about this topic:
CustomSurfaceView: three custom surfaceview for three different levels.
Canvas: lock/unlockAndPost method (i'm not using custom bitmap )
Multi thread ( each surface is a separate thread )
Shapes ( shapes on canvas )
Client/Server ( architecture )
Flickering ( IS WHY I'M HERE )
We are developing a client/server application and I'm working on the client side. I'm receiving messages from the server containing general data (coordinates, color, width [...] ) about paths like, circle, rectangle, line and other shapes. The web application allows the user to send these data drawing on HTML5 Canvas, to an android device that receiving this messages and parsing it, will be able to redraw all the shapes. From my own experience on this subjects, I learned that the best way to keep in control all the things you draw on the canvas, is saving everything into a buffer, array, list or something like that, then reuse it when you want (for example, you can use the older path for show, hide, move or simply change something on the canvas). In my opinion, the android application follows the best practice of android development and OOP paradigm so I'm not assuming errors related to the bad architecture. In this case, I'm saving the messages on web client side. When the user draws on HTML5 Canvas, the messages which contain shape info are perfectly reported to the android canvas, but the problem appears when:
[example]
Consider you draw 10 objects (10 messages) and you want delete only one object on web app canvas, so the only way is clearing all the canvas, and redraw all the previous shapes without the deleted shape (so resend to the client 9 messages by loop the messages buffer ). This method works perfectly for the web app but cause flickering problem on android client. So after too many experiments I found a workaround, using a Thread.sleep(100)(Whooo! 100ms is too much), in order to parse slowly the messages and let the surfaceview thread to read correctly the data (data access through singleton pattern) and write on the double-buffer of the canvas.Well, it's slow and ugly but it works ! Actually I don't like this “horrible” workaround so please help me to see an exit strategy.
This is a piece of code where the canvas get data from shapes containers and draw if data are present. The data of each containers came from server messages.
#Override
public void run() {
Canvas canvas = null;
while (running) {
//this is the surface's canvas
try {
canvas = shapesSurfaceHolder.lockCanvas();
synchronized (shapesSurfaceHolder) {
if (shapesSurfaceHolder.getSurface().isValid()) {
if(!Parser.cmdClear){
//draw all the data present
canvas.drawPath(PencilData.getInstance().getPencilPath(),
PencilData.getInstance().getPaint());
canvas.drawPath(RectData.getInstance().getRectPath(),
RectData.getInstance().getPaint());
canvas.drawPath(CircleData.getInstance().getCirclePath(),
CircleData.getInstance().getPaint());
canvas.drawPath(LineData.getInstance().getLinePath(),
LineData.getInstance().getPaint());
canvas.drawText(TextData.getInstance().getText(),
TextData.getInstance().getX(),
TextData.getInstance().getY(),
TextData.getInstance().getPaint());
} else {
//remove all canvas content and clear data.
canvas.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR);
for (int i = 0; i < AbstractFactory.SHAPE_NUM; i++) {
abstracFactory.getShape(i).clearData();
}
}
}
}
} finally {
if (canvas != null) {
shapesSurfaceHolder.unlockCanvasAndPost(canvas);
}
}
}
}//end_run()
I can summarize that, apparently, my problem is to draw too quickly
Note:
Similar concept: Android thread controlling multiple texture views causes strange flickering
Hardware acceleration is enabled.
minSdkVersion 17
Tested on
Tablet Samsung SM-T113
Google Nexus 5

The TextureView issue was due to a bug specific to TextureView. You're using SurfaceView, so that does not apply here.
When drawing on a SurfaceView's Surface, you must update every pixel inside the dirty rect (i.e. the optional arg passed to lockCanvas()) every time. If you don't provide a dirty rect, that means the entire screen must be updated. This is because the Surface is double- or triple-buffered, and swapped when you call unlockCanvasAndPost(). If you lock / clear / unlock, then the next time you lock / draw / unlock, you will not be drawing into the buffer you previously cleared.
If you want to do incremental rendering, you should point your Canvas at an off-screen Bitmap and do all your rendering there. Then just blit the entire bitmap between lock and unlock. The alternative is to store up the drawing commands, starting with the initial clear, and play them all back between lock/unlock.
The phrase "three custom surfaceview" is somewhat concerning if they're all on screen at once. If you have them all at different Z depths (default, media overlay, top) then they will behave correctly, but the system is generally more efficient if you can put everything on one.

How to detect if bitmap is too large for texture

So I have done a lot of looking around and the answer to this seems to be to use:
int[] maxSize = new int[1];
gl.glGetIntegerv(GL10.GL_MAX_TEXTURE_SIZE, maxSize, 0);
to detect the size of the texture, now my issue is how do I create or get access to the gl var that holds the function I need? Is it already there somewhere? I would like to support android 2.2 and above, so the 4.0+ new trick wont work. If this is a repeat question just point me in the right direction in the comments and I will take t down. Couldn't seem to find a good explanation of how to set this up properly anywhere, just those two lines of code.

If you take a look on how OpenGL Apps are made you will notice there are the main app thread (main activity) and a renderer class(http://developer.android.com/guide/topics/graphics/opengl.html). The heart of the renderer class if the method public void onDrawFrame(GL10 gl) , this is called by the android infrastructure when the frame needs to be redraw.
So basically, a context object (GL10 gl var) is passed to the renderer (yours) when required and there you can check your max texture size.

Fast dynamic vertices in OpenGL ES 2.0 on Android

I am trying to batch draw a bunch of lines on Android using OpenGL ES 2.0 and I need to know the best way to do this.
Right now I made a class called LineEngine which builds up a FloatBuffer of all the vertices to draw and then draws all the lines at once. The problem is that apparently FloatBuffer.put() is very slow and is gobbling up CPU time like crazy.
Here is my class
public class LineEngine {
private static final float[] IDENTIY = new float[16];
private FloatBuffer mLinePoints;
private FloatBuffer mLineColors;
private int mCount;
public LineEngine(int maxLines) {
Matrix.setIdentityM(IDENTIY, 0);
ByteBuffer byteBuf = ByteBuffer.allocateDirect(maxLines * 2 * 4 * 4);
byteBuf.order(ByteOrder.nativeOrder());
mLinePoints = byteBuf.asFloatBuffer();
byteBuf = ByteBuffer.allocateDirect(maxLines * 2 * 4 * 4);
byteBuf.order(ByteOrder.nativeOrder());
mLineColors = byteBuf.asFloatBuffer();
reset();
}
public void addLine(float[] position, float[] color){
mLinePoints.put(position, 0, 8); //These lines
mLineColors.put(color, 0, 4); // are taking
mLineColors.put(color, 0, 4); // the longest!
mCount++;
}
public void reset(){
mLinePoints.position(0);
mLineColors.position(0);
mCount = 0;
}
public void draw(){
mLinePoints.position(0);
mLineColors.position(0);
GraphicsEngine.setMMatrix(IDENTIY);
GraphicsEngine.setColors(mLineColors);
GraphicsEngine.setVertices4d(mLinePoints);
GraphicsEngine.disableTexture();
GLES20.glDrawArrays(GLES20.GL_LINES, 0, mCount * 2);
GraphicsEngine.disableColors();
reset();
}
}
Is there a better way to batch all these lines together?

What you are trying to do is called SpriteBatching. If you want your program to be robust in openGL es you have to check the following( a list of what makes your program slow ) :
-Changing to many opengl ES states each frame.
-Enable//Dissable (textures etc) again and again. Once you enable something you dont have to do it again it will be applied automaticly its frame.
-Using to many assets instead of spriteSheets. Yes thats make a huge performance impact. For example if you have 10 images you have to load for each image a different texture and that is SLOW. A better way to do this is to create a spriteSheet.(You can check google for free spriteSheet creators).
-Your images are high quality. Yes! check your resolution and file extension. Prefer .png files.
-Care for the api 8 float buffer bug(you have to use int buffer instead to fix the bug).
-Using java containers. This is the biggest pitfall, if you use string concatenation(that's not a container but it returns a new string each time) or a List or any other container that RETURNS A NEW CLASS INSTANCE chances are your program will be slow due to garbage collection. For input handling i would suggest you to search a teqnique called the Pool Class. Its use is to recycle objects instead of creating new ones. Garbage collector is the big enemy, try to make him happy and avoid any programming technique that might call him.
-Never load things on the fly, instead make a loader class and load all the necessary assets in the begining of the app.
If you do implement this list then your chances are your program will be robust.
One last adition. What is a sprite batcher? A spriteBatcher is a class that uses a single texture to render multiple objects. It will create automaticly vertices, indices, colors, u - v coords and it will render them as a batch. This pattern will save GPU power as well as cpu power. From your posted code i can't be sure what causes the cpu to slow down but from my experience is due to one(or more) things of the list i previously mention. Check the list, follow it, search google for spriteBatching and i am sure your program will run fast. Hope i helped!
Edit: I think i found what causes your program to slow down, you dont flip the buffer dude! You just reset the position. You just add add add more objects and you cause buffer overload. In the reset method just flip the buffer. mLineColors.flip mLinePaints.flip will do the job. Make sure you call them each frame if you send new verices each frame.

FloatBuffer.put(float[]) with a relatively large float array should be considerably faster. The single put(float) calls have plenty of overhead.

Just go for a very simple native function which will be called from your class. You can put float[] to OpenGL directly, no need to kill CPU time with a silly buffer interface.

Preventing "flickering" when calling Drawable.draw()

I have a little experimentation app (essentially a very cut-down version of the LunarLander demo in the Android SDK), with a single SurfaceView. I have a Drawable "sprite" which I periodically draw into the SurfaceView's Canvas object in different locations, without attempting to erase the previous image. Thus:
private class MyThread extends Thread {
SurfaceHolder holder; // Initialised in ctor (acquired via getHolder())
Drawable sprite; // Initialised in ctor
Rect bounds; // Initialised in ctor
...
#Override
public void run() {
while (true) {
Canvas c = holder.lockCanvas();
synchronized (bounds) {
sprite.setBounds(bounds);
}
sprite.draw(c);
holder.unlockCanvasAndPost(c);
}
}
/**
* Periodically called from activity thread
*/
public void updatePos(int dx, int dy) {
synchronized (bounds) {
bounds.offset(dx, dy);
}
}
}
Running in the emulator, what I'm seeing is that after a few updates have occurred, several old "copies" of the image begin to flicker, i.e. appearing and disappearing. I initially assumed that perhaps I was misunderstanding the semantics of a Canvas, and that it somehow maintains "layers", and that I was thrashing it to death. However, I then discovered that I only get this effect if I try to update faster than roughly every 200 ms. So my next best theory is that this is perhaps an artifact of the emulator not being able to keep up, and tearing the display. (I don't have a physical device to test on, yet.)
Is either of these theories correct?
Note: I don't actually want to do this in practice (i.e. draw hundreds of overlaid copies of the same thing). However, I would like to understand why this is happening.
Environment:
Eclipse 3.6.1 (Helios) on Windows 7
JDK 6
Android SDK Tools r9
App is targetting Android 2.3.1
Tangential question:
My run() method is essentially a stripped-down version of how the LunarLander example works (with all the excess logic removed). I don't quite understand why this isn't going to saturate the CPU, as there seems to be nothing to prevent it running at full pelt. Can anyone clarify this?

Ok, I've butchered Lunar Lander in a similar way to you, and having seen the flickering I can tell you that what you are seeing is a simple artefact of the double-buffering mechanism that every Surface has.
When you draw anything on a Canvas attached to a Surface, you are drawing to the 'back' buffer (the invisible one). And when you unlockCanvasAndPost() you are swapping the buffers over... what you drew suddenly becomes visible as the "back" buffer becomes the "front", and vice versa. And so your next frame of drawing is done to the old "front" buffer...
The point is that you always draw to seperate buffers on alternate frames. I guess there's an implicit assumption in graphics architecture that you're always going to be writing every pixel.
Having understood this, I think the real question is why doesn't it flicker on hardware? Having worked on graphics drivers in years gone by, I can guess at the reasons but hesitate to speculate too far. Hopefully the above will be sufficient to satisfy your curiousity about this rendering artefact. :-)

You need to clear the previous position of the sprite, as well as the new position. This is what the View system does automatically. However, if you use a Surface directly and do not redraw every pixel (either with an opaque color or using a SRC blending mode) you must clear the content of the buffer yourself. Note that you can pass a dirty rectangle to lockCanvas() and it will do the union for you of the previous dirty rectangle and the one you are passing (this is the mechanism used by the UI toolkit.) It will also set the clip rect of the Canvas to be the union of these two rectangles.
As for your second question, unlockAndPost() will do a vsync, so you will never draw at more than ~60fps (most devices that I've seen have a display refresh rate set around 55Hz.)