Develop Reference

Hiding fragments in OpenGL

I am working with OpenGL-ES 3.0 in Android.
To simplify my issue, I am going to describe a scenario very similar to mine. Let's say that I have a sphere centered at the origin. Let's also say that I have another sphere centered at the origin, with a larger radius. Finally, let's say that I have a cylinder, and the center of the top face of the cylinder is lying on the origin. The cylinder intersects the two spheres. A picture of the scenario is below:
This is my initial setup:
I would like to only draw the section in between the two spheres, as shown below:
However, in my application, the smaller of the two spheres isn't visible (though it exists). It is completely transparent. Thus, the final end product I would like would look something like this:
Now, one more piece of information: as I mentioned earlier, this is a simplification of my current scenario. Instead of spheres, I have far more complex objects (not simple primitive shapes). Thus, approaching this from a mathematical perspective (such as only drawing the portion of the cylinder that is greater than the smaller sphere's radius and less than the larger sphere's radius) is not going to work. I need to approach this somehow from a programming perspective (but given my limited knowledge of OpenGL, I can only think of Depth Testing and Blending as viable options)

You can probably do this using a stencil buffer.
I haven't compiled this code and it will need modifying, but this is the general idea:
glDisable( GL_STENCIL_TEST );
<Render rest of scene (everything other than the spheres and cylinder)>
// Render the larger sphere into the stencil buffer, setting stencil bits to 1
glEnable( GL_STENCIL_TEST );
glClear( GL_STENCIL_BUFFER_BIT );
glColorMask( GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE ); // Don't render into the color buffer
glDepthMask( GL_FALSE ); // Don't render into the depth buffer
glStencilMask( 0xff ); // Enable writing to stencil buffer
glStencilFunc( GL_ALWAYS, 1, 0xff ); // Write 1s into stencil buffer
glStencilOp( GL_KEEP, GL_KEEP, GL_REPLACE ); // Overwrite for every fragment that passes depth test (and stencil <- GL_ALWAYS)
<Render big sphere>
// Render the smaller sphere into the stencil buffer, setting stencil bits to 0 (it carves out the big sphere)
glStencilFunc( GL_ALWAYS, 0, 0xff ); // Write 0s into stencil buffer
<Render small sphere>
// Render the cylinder into the color buffer, only where the stencil bits are 1
glStencilMask( 0 ); // Don't need to write to stencil buffer
glColorMask( GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE ); // Render into the color buffer
glStencilFunc( GL_EQUAL, 1, 0xff ); // Only render where there are 1s in the stencil buffer
<Render cylinder>
glDisable( GL_STENCIL_TEST );
// Now render the translucent big sphere using alpha blending
<Render big sphere>

What you are describing is Constructive Solid Geometry, but with the added complexity of using meshes as one of the primitive types.
Event with only mathematically simple primitives, it is very hard to implement CSG purely in the OpenGL pipeline because you would need to find a way to represent the scene graph in a way that the shaders can understand and efficiently parse. Once you add in meshes, it's basically impossible because the vertex and fragment shaders won't have easy access to the mesh geometry.
You might be able approximate it by executing a draw call for every item in the CGS graph and with clever manipulation of stencil and depth buffers, but you would probably still end up with lots of edge cases that didn't render properly.

Using stencil test to clip items being drawn in OpenGL-ES

I'm currently working on an OpenGL project in Android. Right now, I'm trying to create a "clip" for items to be confined to when drawing, using the stencil buffer. My problem is that I would like to be able to create a clip shape, and then "remove" portions of that clipped shape later on (much like in an Android Canvas calling clipPath() with Region.Op.UNION and then another clipPath() call with Region.Op.DIFFERENCE)
Basically, my intention is to only draw to portions of the screen with a stencil buffer bit of value 1. Therefore, as I understand it, I should be able to draw to the stencil buffer with different glStencilOp functions that will allow me to draw with 1's in some areas, and then perhaps replace them later on with the default 0 in portions of those areas.
As an example test, I tried to clip a "doughnut" on the screen, using something like this (Note: I am using the ShapeRenderer from the LibGDX open source project to draw shapes):
// Beginning LibGDX shapeRenderer.
shapeRenderer.begin(ShapeType.FILLED);
gl20.glClear(GL20.GL_STENCIL_BUFFER_BIT); // Clear stencil buffer.
gl20.glEnable(GL20.GL_STENCIL_TEST); // Enable stencil test.
gl20.glColorMask(false, false, false, false); // Disable color drawing.
gl20.glStencilMask(0xFF); // Set 0xFF as stencil mask.
// Have stencil test always fail, replace drawn stencil bits with 1.
gl20.glStencilFunc(GL20.GL_NEVER, 1, 0xFF);
gl20.glStencilOp(GL20.GL_REPLACE, GL20.GL_KEEP,GL20.GL_KEEP);
/* Draw outer circle, libGDX call. 600px radius. */
shapeRenderer.circle(screenRect.width() / 2, screenRect.height() / 2, 600);
// Now, replace newly drawn stencil bits with 0.
gl20.glStencilOp(GL20.GL_ZERO, GL20.GL_KEEP, GL20.GL_KEEP);
/* Draw inner circle, libGDX call. 200px radius. */
shapeRenderer.circle(screenRect.width() / 2, screenRect.height() / 2, 200);
// Re-enable color drawing.
gl20.glColorMask(true, true, true, true);
// Now, items drawn will be confined to areas with '1' stencil bit.
gl20.glStencilFunc(GL20.GL_EQUAL, 1, 0xFF);
gl20.glStencilOp(GL20.GL_KEEP, GL20.GL_KEEP, GL20.GL_KEEP);
gl20.glStencilMask(0x00);
/* Draw a filled, colored rectangle to the entire screen. */
shapeRenderer.setColor(Color.RED);
shapeRenderer.drawRect(screenRect);
shapeRenderer.end();
Now, as I understand it, this should result in a colored 'doughnut' drawn to the screen; However, as the code is written above, nothing is drawn to the screen.
However, if I only clip the outer circle (and not try to get rid of the inner circle with the "gl20.glStencilOp(GL20.GL_ZERO, GL20.GL_KEEP, GL20.GL_KEEP)" call and subsequent inner circle drawing, a circle the size of the intended outer circle draws on the screen.
I've looked through the documentation that I could find, and everything that I've read so far indicates that this should work. Is there something that I'm missing here? Any help is greatly appreciated.

I've found the problem; I was drawing the two circles in between one begin() and end() call in the ShapeRenderer. Therefore, both circles were being drawn at the time of the end() call. This means that the shapes weren't even drawing to the stencil buffer, seeing as I was calling these functions before the end() call:
// Now, items drawn will be confined to areas with '1' stencil bit.
gl20.glStencilFunc(GL20.GL_EQUAL, 1, 0xFF);
gl20.glStencilOp(GL20.GL_KEEP, GL20.GL_KEEP, GL20.GL_KEEP);
gl20.glStencilMask(0x00);
If I make sure that I use a begin() call before and end() call after each shape being drawn, it will draw at the expected time, therefore making sure that it is drawing the expected values to the stencil buffer.

The logic looks OK - what geometry are you drawing?
At a guess, based on the fact that you see the outer circle rendered on screen when you don't render the inner one, it looks like you are using the outer circle geometry in both cases (so you set all the stencil values, then clear all of them, so nothing has a stencil value of 1 when you come to blit the rectangle).

I implemented your solution in WebGL and it works fine (try it out):
http://ezekiel.vancouver.wsu.edu/~cs442/stackoverflow/stencil.html
Here is how I am mimicking your code:
gl.enable(gl.STENCIL_TEST);
gl.colorMask(false, false, false, false);
gl.stencilMask(0xFF);
gl.stencilFunc(gl.NEVER, 1, 0xFF);
gl.stencilOp(gl.REPLACE, gl.KEEP, gl.KEEP);
fillCircle(1); // radius of 1 maps to canvas width
gl.stencilOp(gl.ZERO, gl.KEEP, gl.KEEP);
fillCircle(0.333);
gl.colorMask(true, true, true, true);
gl.stencilFunc(gl.EQUAL, 1, 0xFF);
gl.stencilOp(gl.KEEP, gl.KEEP, gl.KEEP);
gl.stencilMask(0x00);
gl.uniform3fv(program.color, [1, 0, 0]);
fillCircle(3); // fill whole canvas width red
You'll see you get exactly what you wanted.

OpenGL ES 2.0 render to texture with transparent background

I would like to create an interactive 2D effect that I can put on anything. So I want to take a mostly transparent effect, render it to a texture, and put it wherever I want simply by putting it on a square.
The problem I encountered is that I can't get rid of the background color. When I put the effect over an object, the background color of the effect blocks out the object that I want to put the effect over.
Here is my code. Can anybody tell me what I'm missing?
Drawing:
GLES20.glClearColor(0.6f, 0.34f, 0.14f, 0.0f);
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
GLES20.glDisable(GLES20.GL_DEPTH_TEST);
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
ball.draw(); //This is the object a simple square draw with 2 triangles
particleSystem.renderToTexture(); //effect rendering to texture and drawing it
The render to texture code code:
public void renderToTexture(){
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fb[0]);
GLES20.glClear( GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, renderTex[0], 0);
GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER, depthRb[0]);
int status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER);
drawRender(); //draws the effect on a fbo and saving the texture to renderTex[0]
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, renderTex[0]);
drawEffect(); //draws a square with the effect texture on it
The problem should be here somewhere, but I have no idea what to do. I tried everything I could think of. I even triend to dispose of the background color in the shader.

The posted code looks perfectly fine. You will need to make sure that the texture you use for the color render target (renderTex[0]) has an alpha component.
Note that the number of texture formats in ES 2.0 that are guaranteed to be color-renderable is very limited. The only two with an alpha component (see table 4.5. in the spec) are GL_RGBA4 and GL_RGB5_A1. Most notably, this does not include GL_RGBA with 8 bits per component.
So for defining a color-renderable texture with alpha component that is guaranteed to work across all ES 2.0 implementations, you will have to use GL_RGBA for the internal format, and GL_UNSIGNED_SHORT_4_4_4_4 or GL_UNSIGNED_SHORT_5_5_5_1 for the type argument of glTexImage2D().
Most common devices (at least all the ones I have seen) do support the OES_rgb8_rgba8 extension, which adds support for rendering to 8 bit component textures. But if you want to be completely portable, you should check for the presence of this extension before using render targets with those formats.

I know this question is old, but i also encountered it right now and the solution for me was to do something with the GLSurfaceView. Just add these lines after setting the context
glSurfaceView.setEGLConfigChooser(8, 8, 8, 8, 0, 0);
glSurfaceView.getHolder().setFormat(PixelFormat.TRANSLUCENT);
I was already using GL_RGBA format and was really annoyed that nothing else was working, but this worked like magic.

Hide second-row transparent faces?

I'm trying to display a geographically complex, semi-transparent (e.g. alpha = 0.5) object (terrain). When I render this object, the hidden front-faces of this object will also be drawn (like a hill that actually lies behind another one).
I would like to see other objects behind my "terrain" object, but don't want to see the hidden faces of my terrain (the second hill). So actually set the transparency for the "whole" object, not for single faces.
Q: How could I achieve to hide the "hidden" front-faces of a semi-transparent object?
I'm setting the transparency in the vertex shader by multiplying the color vector with the desired transparency:
fColor = vec4(vColor, 1.0);
fColor *= 0.5;
// fColor goes to fragment shader
GL_DEPTH_TEST is activated with GL_LEQUAL as depth function.
GL_BLEND is activated with GL_ONE, GL_ONE_MINUS_SRC_ALPHA as blending functions.
I tried to deactivate the depth buffer by GLES20.glDepthMask(false); before drawing, but this doesn't make any difference.
Probably I don't get the idea for the right depth buffer settings or the blending functions.

Well, I think I got it now:
Actually I can resign on blending at all. With the depth test switched on only the foreground fragments are visible (the front hill of my terrain). With the multiplication in the vertex shader the fragment shader will draw these visible fragments with desired transparency (the terrain as a whole becomes semi-transparent).
So, depth test on, blending off, color multiplication in vertex shader.

Render To Texture & Trails Not Disappearing

I am trying to implement fireworks with point sprites and then drawing trails by rendering the fireworks to two different textures, and then drawing them back to each other every second time with alpha set to < 1, making each previous draw fade away more and more.
It looks nice and depending on what alpha I use when drawing the previously rendered texture the trails gets longer or shorter, as expected. However, on most devices (not all) the trails never disappear completely. They leave a very transparent but still noticable trail after they should have faded away.
This is what I do currently:
depending on a boolean called second I set the framebuffer to render to one of two textures called buff1 and buff2.
((GL11ExtensionPack)gl).glBindFramebufferOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES, myFBO.get(0));
if (second){
((GL11ExtensionPack)gl).glFramebufferTexture2DOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES, GL11ExtensionPack.GL_COLOR_ATTACHMENT0_OES, GL10.GL_TEXTURE_2D, buff2.texture.textureId, 0);
} else {
((GL11ExtensionPack)gl).glFramebufferTexture2DOES(GL11ExtensionPack.GL_FRAMEBUFFER_OES, GL11ExtensionPack.GL_COLOR_ATTACHMENT0_OES, GL10.GL_TEXTURE_2D, buff1.texture.textureId, 0);
}
gl.glViewport(0, 0, bufferSize, bufferSize);
gl.glClearColor(0, 0, 0, 0);
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
gl.glBlendFunc(GL10.GL_ONE, GL10.GL_ONE_MINUS_SRC_ALPHA);
Then I render the previously drawn texture on the current one, using a alpha value < 1. The buff object is just a helper drawing a quad with a texture. The draw function premultiplies the alpha to the other color channels.
// a = alpha, higher value = longer trails = more distinct leftovers
if (second){
buff1.setColor(1, 1, 1, a);
buff1.draw(gl);
second = false;
else {
buff2.setColor(1, 1, 1, a);
buff2.draw(gl);
second = true;
}
After that I do the drawing of the point sprite fireworks. Also using premultiplied alpha and gl.glBlendFunc(GL10.GL_ONE, GL10.GL_ONE_MINUS_SRC_ALPHA);
Finally I reset the frame buffer and have my new texture containing the old trails fading and the current fireworks on top drawn at 100% alpha. I then use this texture to draw to my scene together with all the other objects behind and in front of the fireworks texture.
As I said, almost everything is working, except for those almost transparent trails staying after the trails should have faded. The higher the alpha value drawing the previous buffer the more noticable the trail "leftovers" that stay forever.
I have been trying with different blending modes, pre-multiplied and not pre-multiplied alpha etc. But since this is an android opengl es 1.1 I think the blending should be GL_ONE and GL_ONE_MINUS_SRC_ALPHA since all bitmaps are premultiplied by default.
What am I doing wrong? Is there a better way to do this?
Thanks for any help,
Anders

What am I doing wrong?
The way you sum downscaled (in their values) image is effectively a Power Series that exponetiate your alpha value. Such a power series will asympote to zero, but never reach it.
Is there a better way to do this?
Actually this is the right way to do it. You just forgot one last step: You want to have some threshold added, so that values below a certain alpha values get clamped to zero. Most easily this is obtained adding a GL_GREATER alpha test, with the threshold slightly above the alpha value of your trails you do want to remove.