Develop Reference

Strange texture result when mix two camera textures in glsl

I'm making a simple image filter app in android and I implemented lowpass filter using same method in GPUImage(https://github.com/BradLarson/GPUImage)
it buffers the previous and current camera frames mixture and render it.
So i created a buffer FBO and render the current camera texture, re-use it as a texture for mixture in lowpass filter shader with next camera texture.
I tested my code with some smartphones(Galaxy S10, Nexus 6P, etc..) and it worked well. However in Galaxy S8(Mali-G71) the result is strange and I don't know what was wrong.
These are the wrong results
Here are my code:
Fragment shader:
varying vec2 vTextureCoord;
uniform sampler2D sTexture1;
uniform float filterStrength;
void main() {
vec4 texColor0 = texture2D(sTexture, vTextureCoord);
vec4 texColor1 = texture2D(sTexture1, vTextureCoord);
gl_FragColor = mix(texColor0, texColor1, filterStrength);
}
What can cause this results?
Thanks in advance.

The artifacts look tile aligned for Mali, so if I had to guess you are reading the currently bound framebuffer color attachment as an input texture at the same time as writing in to it.
This is "implementation defined" behavior in the specification, and concurrent reads and writes will definitely do bad things on a tile-based renderer like Mali.

Point sprite alpha blending issue (Android / OpenGL ES 2.0)

I've recently started looking into OpenGL ES for Android and am working on a drawing app. I've implemented some basics such as point sprites, path smoothing and FBO for double buffering. At the moment I am playing around with the glBlendFunc, more specifically when I put two textures close to each other with the same color/alpha values, the alpha gets added so it appears darker at the intersection of the sprites. This is a problem because the stroke opacity is not preserved if a lot of points are close together, as the color tends to more opaque rather than staying with the same opacity. Is there a way to make the textures have the same color on the intersection, i.e. have the same alpha value for the intersecting pixels, but keep the alpha values for the rest of the pixels?
Here's how I've done the relevant parts of the app:
for drawing the list of point sprites I use blending like this:
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE_MINUS_SRC_ALPHA);
the app uses an FBO with a texture, where it renders each brush stroke first and then this texture is rendered to the main screen. The blending func there is:
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
OpenGL ES 2.0 doesn't support alpha masking;
there is no DEPTH_TEST function used anywhere in the app;
the textures for the point sprites are PNGs with transparent backgrounds;
the app supports texture masking which means one texture is used for the shape and one texture is used for the content;
my fragment shader looks like this:
precision mediump float;
uniform sampler2D uShapeTexture;
uniform sampler2D uFillTexture;
uniform float vFillScale;
varying vec4 vColor;
varying float vShapeRotation;
varying float vFillRotation;
varying vec4 vFillPosition;
vec2 calculateRotation(float rotationValue) {
float mid = 0.5;
return vec2(cos(rotationValue) * (gl_PointCoord.x - mid) + sin(rotationValue) * (gl_PointCoord.y - mid) + mid,
cos(rotationValue) * (gl_PointCoord.y - mid) - sin(rotationValue) * (gl_PointCoord.x - mid) + mid);
}
void main() {
// Calculations.
vec2 rotatedShape = calculateRotation(vShapeRotation);
vec2 rotatedFill = calculateRotation(vFillRotation);
vec2 scaleVector = vec2(vFillScale, vFillScale);
vec2 positionVector = vec2(vFillPosition[0], vFillPosition[1]);
// Obtain colors.
vec4 colorShape = texture2D(uShapeTexture, rotatedShape);
vec4 colorFill = texture2D(uFillTexture, (rotatedFill * scaleVector) + positionVector);
gl_FragColor = colorShape * colorFill * vColor;
}
my vertex shader is this:
attribute vec4 aPosition;
attribute vec4 aColor;
attribute vec4 aJitter;
attribute float aShapeRotation;
attribute float aFillRotation;
attribute vec4 aFillPosition;
attribute float aPointSize;
varying vec4 vColor;
varying float vShapeRotation;
varying float vFillRotation;
varying vec4 vFillPosition;
uniform mat4 uMVPMatrix;
void main() {
// Sey position and size.
gl_Position = uMVPMatrix * (aPosition + aJitter);
gl_PointSize = aPointSize;
// Pass values to fragment shader.
vColor = aColor;
vShapeRotation = aShapeRotation;
vFillRotation = aFillRotation;
vFillPosition = aFillPosition;
}
I've tried playing around with the glBlendFunc parameters but I can't find the right combination to draw what I want. I've attached some images showing what I would like to achieve and what I have at the moment. Any suggestions?
The Solution
Finally managed to get this working properly with a few lines thanks to # Rabbid76. First of all I had to configure my depth test function before I draw to the FBO:
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
GLES20.glDepthFunc(GLES20.GL_LESS);
// Drawing code for FBO.
GLES20.glDisable(GLES20.GL_DEPTH_TEST);
Then in my fragment shader I had to make sure that any pixels with alpha < 1 in the mask are discarded like this:
...
vec4 colorMask = texture2D(uMaskTexture, gl_PointCoord);
if (colorMask.a < 1.0)
discard;
else
gl_FragColor = calculatedColor;
And the result is (flickering is due to Android emulator and gif capture tool):

If you set the glBlendFunc
with the functions (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) and you use
glBlendEquation with the equation GL_FUNC_ADD then the destination color is
calculated as follows:
C_dest = C_src * A_src + C_dest * (1-A_src)
If you blend for example C_dest = 1 with C_src = 0.5 and A_src = 0.5 then:
C_dest = 0.75 = 1 * 0.5 + 0.5 * 0.5
If you repeat blending the same color C_src = 0.5 and A_src = 0.5 then the destination color becomes darker:
C_dest = 0.625 = 0.75 * 0.5 + 0.5 * 0.5
Since the new target color is always a function of the original target color and the source color, the color can not remain equel when blending 2 times, because the target color has already changed after the 1st time blending (except GL_ZERO).
You have to avoid that any fragment is blended twice. If all fragments are drawn to the same depth (2D) then you can use the depth test for this:
glEnable( GL_DEPTH_TEST );
glDepthFunc( GL_LESS );
// do the drawing with the color
glDisable( GL_DEPTH_TEST );
Or the stencil test can be used. For example, the stencil test can be set to pass only when the stencil buffer is equal to 0.
Every time a fragment is to be written the stencil buffer is incremented:
glClear( GL_STENCIL_BUFFER_BIT );
glEnable( GL_STENCIL_TEST );
glStencilOp( GL_KEEP, GL_KEEP, GL_INCR );
glStencilFunc( GL_EQUAL, 0, 255 );
// do the drawing with the color
glDisable( GL_STENCIL_TEST );
Extension to the answer
Note that you can discard fragments which should not be drawn.
If the fragment in your sprite texture has an alpha channel of 0 you should discard it.
Note, if you discard a fragment neither the color nor the depth and stencil buffer will be written.
Fragment shaders also have access to the discard command. When executed, this command causes the fragment's output values to be discarded. Thus, the fragment does not proceed on to the next pipeline stages, and any fragment shader outputs are lost.
Fragment shader
if ( color.a < 1.0/255.0 )
discard;

It's not possible to do this using the fixed-function blending in OpenGL ES 2.0, because what you want isn't actually alpha blending. What you want is a logical operation (e.g. max(src, dst)) which is rather different to how OpenGL ES blending works.
If you want to do path / stroke / fill rendering with pixel-exact edges you might get somewhere with using stencil masks and stencil tests, but you can't do transparency in this case - just boolean operators.

GLSL - fragment shaders for image manipulation/blending

I'm trying to build a shader that allows you to combine two images by applying a gradient opacity mask to the one on top, like you do in photoshop. I've gotten to the point where I can overlay the masked image over the other but as a newbie am confused about a few things.
It seems that the images inside the shader are sometimes skewed to fit the canvas size, and they always start at position 0,0. I have played around with a few snippets I have found to try and scale the textures, but always end up with unsatisfactory results.
I am curious if there is a standard way to size, skew, and translate textures within a view, or if images in GLSL are necessarily limited in some way that will stop me from accomplishing my goal.
I'm also unsure of how I am applying the gradient/mask and if it is the right way to do it, because I do not have a lot of control over the shape of the gradient at the moment.
Here's what I have so far:
precision highp float;
varying vec2 uv;
uniform sampler2D originalImage;
uniform sampler2D image;
uniform vec2 resolution;
void main(){
float mask;
vec4 result;
vec2 position = gl_FragCoord.xy / ((resolution.x + resolution.y) * 2.0 );
mask = vec4(1.0,1.0,1.0,1.0 - position.x).a;
vec4 B = texture2D(image,uv);
vec4 A = texture2D(originalImage,uv) * (mask);
result = mix(B, A, A.a);
gl_FragColor = result;
}
Which produces an image like this:
What I would like to be able to do is change the positions of the images independently and also make sure that they conform to their proper dimensions.
I have tried naively shifting positions like this:
vec2 pos = uv;
pos.y = pos.y + 0.25;
texture2D(image, pos)
Which does shift the texture, but leads to a bunch of strange lines dragging:
I tried to get rid of them like this:
gl_FragColor = uv.y < 0.25 ? vec4(0.0,0.0,0.0,0.0) : result;
but it does nothing

You really need to decide what you want to happen when images are not the same size. What you probably want is for it to appear there's no image so check your UV coordinates and use 0,0,0,0 when outside of the image
//vec4 B = texture2D(image,uv);
vec4 getImage(sampler2D img, vec2 uv) {
if (uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0) {
return vec4(0);
}
return texture2D(img, uv);
}
vec4 B = getImage(image, uv);
As for a standard way to size/skew/translate images use a matrix
uniform mat4 u_imageMatrix;
...
vec2 newUv = u_imageMatrix * vec4(uv, 0, 1).xy;
An example of implementing canvas 2d's drawImage using a texture matrix.
In general though I don't think most image manipulations programs/library would try to do everything in the shader. Rather they'd build up the image with very very simple primitives. My best guess would be they use a shader that's just A * MASK then draw B followed by A * MASK with blending on.
To put it another way, if you have 30 layers in photoshop they wouldn't generate a single shader that computes the final image in one giant shader taking in all 30 layers at once. Instead each layer would be applied on its own with simpler shaders.
I also would expect them to create an texture for the mask instead of using math in the shader. That way the mask can be arbitrarily complex, not just a 2 stop ramp.
Note I'm not saying you're doing it wrong. You're free to do whatever you want. I'm only saying I suspect that if you want to build a generic image manipulation library you'll have more success with smaller building blocks you combine rather than trying to do more complex things in shaders.
ps: I think getImage can be simplified to
vec4 getImage(sampler2D img, vec2 uv) {
return texture2D(img, uv) * step(0.0, uv) * step(-1.0, -uv);
}

Repeated frames on Android Mali-400

On Android devices with Mali-400 GPU (Samsung Galaxy S II, Samsung Galaxy S3 Mini, Samsung Galaxy Note II), at random times the screen will start showing repeated frames.
Example from 0:51 until 1:01 on the following video https://www.youtube.com/watch?v=5-p6Oy0BZmg
It seems as if new frames aren't being rendered, and what was in the old buffer is being shown again. The game continues to advance behind the repeated frames.
This doesn't happen on other GPUs.
I read about using glFlush or glFinish, but GLSurfaceView takes care of this when doing eglSwapBuffers after onDrawFrame.
I've read about quirks of Mali-400, like is better to use varying for texture coords, or to use lowp, but it doesn't help. Here are the shaders for reference:
Vertex shader:
// Vertex Shader
attribute vec4 position;
attribute vec4 colorModelIn;
attribute vec4 colorVertexIn;
varying lowp vec4 colorOut;
uniform mat4 modelViewProjectionMatrix;
uniform mat4 modelMatrix;
attribute vec2 TexCoordIn;
varying lowp vec2 TexCoordOut;
uniform bool bUseVertexColor;
void main()
{
if( bUseVertexColor ){
colorOut = colorVertexIn * colorModelIn;
} else {
colorOut = colorModelIn;
}
TexCoordOut = TexCoordIn;
gl_Position = modelViewProjectionMatrix * modelMatrix * position;
}
Fragment shader:
// Fragment shader
varying lowp vec4 colorOut;
varying lowp vec2 TexCoordOut;
uniform sampler2D Texture;
uniform bool bUseTexture;
void main()
{
if( bUseTexture ){
gl_FragColor = colorOut * texture2D(Texture, TexCoordOut);
} else {
gl_FragColor = colorOut;
}
}
I'm aware that these shaders aren't optimal, and that I'm going down the path of reproducing the fixed pipeline.
The rendering goes back to normal after some time, or after touching the screen. The only reason I can think for going back to normal when touching is that I use color-coding to detect the touched object. I render an image to the back buffer and glReadPixels from it. Then, overwrite the back buffer with the normal game image.
I'm out of ideas on how to attack this problem.
EDIT
After following Muzza's advice I started to log GL errors. glGetInteger and glBindBuffer report out-of-memory.
Above I said that the problem solves itself after a while. When that happens these appear in the logs:
01-23 21:57:52.956: D/WebView(9860): onSizeChanged - w:480 h:75
01-23 21:57:53.126: D/TilesManager(9860): new EGLContext from framework: 40e00bd0
01-23 21:57:53.126: D/GLWebViewState(9860): Reinit shader
01-23 21:57:53.171: D/GLWebViewState(9860): Reinit transferQueue

This can happen if the OpenGL state becomes invalid in some way. The graphics drivers can just skip frames entirely. Check Logcat to see if there is any output from the drivers, and add glGetError() calls throughout your code to see if any error comes up there.

Android and Planet Rendering

I am learning how to do 3D development on android. I started with a simple rotating planet with some clouds. I have spent past 2 days trying to get atmospheric glow added to the planet. I looked online and tried working with shaders but was unable to get far.
My question is, what is the best way to achieve this? I nudge in the right direction may be all I need.
I attached a screenshot of the planet I have so far, as well as the end goal I am shooting for.
Thank you for any help.
Current progress:
End Goal:
http://25.media.tumblr.com/tumblr_m06q7l7BpO1qk01v6o1_500.png

What you need is usually done in post process render pass.So for example :
Pass -1 : Render the planet model into FBO texture attachment.
Pass- 2 :Set screen quad , and attach the texture from the previous pass as sampler uniform.
Then in the fragment shader apply a glow effect .
For the glow, there are many ways of doing it.For example you can draw the silhouette of the planet in the first pass ,give it color fill of your glow and then bluring it using smoothstep().Then in post -process pass you put it under the main planet texture.
In fact here you can see a lot of code samples on how to do a glow for circular objects.
One more thing.Adding blur based glow can impact the performance greatly on mobile device.There is a technique called "Distance Field".Valve used it to anti-alias fonts.But it also can be used to do glow.You create a distance field texture texture copy of your planet silhouette once ,then in the post - process pass use it to do smooth glow effect.Fortunately for you there are functions to generate it .Here you can find the papers and the code how to get it done.

Looks like you want Rim Lighting. This does not require any additional passes.
c.f. http://oneclick-code.blogspot.co.uk/2012/01/ios-opengl-es-20-lighting-models-for.html
Note: there's a lot of extra stuff in the example, but the key point is: you compare the normal at each vertex to the direction of your camera's view vector. When they are at right angles, (dot product == 0) apply "full" light. When they are parellel (dot product == length), apply zero light.
varying mediump vec3 OUT_View;
varying mediump vec3 OUT_Light;
varying mediump vec3 OUT_Normal;
varying mediump vec2 OUT_TexCoord;
uniform sampler2D EXT_TEXTURE_01;
uniform sampler2D EXT_TEXTURE_02;
uniform sampler2D EXT_TEXTURE_03;
uniform sampler2D EXT_TEXTURE_04;
void main(void)
{
highp vec4 vDiffuseColor = vec4( 0.0, 0.0, 0.5, 1.0 );
highp float fDiffuseFactor = 0.2 + max ( dot ( OUT_Normal, OUT_Light ), 0.0 );
highp vec4 clr;
if ( fDiffuseFactor < 0.3 )
vDiffuseColor = vDiffuseColor * 0.1;
else
if ( fDiffuseFactor < 0.7 )
vDiffuseColor = vDiffuseColor * 0.5;
else
if ( fDiffuseFactor < 0.8 )
vDiffuseColor = vDiffuseColor;
else
vDiffuseColor = vDiffuseColor * 1.3;
gl_FragColor = vDiffuseColor;
}