Apply mask to video in OpenGL - android

I am a freshman with OpenGL. I try to find the solution to my problem. I have recorded a video with alpha mask. The top part of video RGB and bottom is alpha. As I understand need to change my shaders, but I don`t understand how to do this.
Vertex shader
private static final String vertexShaderCode =
"attribute vec4 vPosition;" +
"attribute vec4 vTexCoordinate;" +
"uniform mat4 textureTransform;" +
"varying vec2 v_TexCoordinate;" +
"void main() {" +
" v_TexCoordinate = (textureTransform * vTexCoordinate).xy;" +
" gl_Position = vPosition;" +
"}";
Fragment shader
private static final String fragmentShaderCode =
"#extension GL_OES_EGL_image_external : require\n" +
"precision mediump float;" +
"uniform samplerExternalOES texture;" +
"varying vec2 v_TexCoordinate;" +
"void main () {" +
" vec4 color = texture2D(texture, v_TexCoordinate);" +
" gl_FragColor = color;" +
"}";

To skip fragments you can use the discard keyword in the fragment shader.
You have to do 2 look ups to the texture. The first one in the upper half, to get the RGB color channels and the 2nd one in the lower half to get the mask from the red color channel:
vec2 c_uv = vec2(texture.x, texture.y*0.5);
vec4 color = texture2D(c_uv, v_TexCoordinate);
vec2 m_uv = vec2(texture.x, 0.5 + texture.y*0.5);
float mask = texture2D(m_uv, v_TexCoordinate).r;
The fragment shader may look like this:
#extension GL_OES_EGL_image_external : require
precision mediump float;
uniform samplerExternalOES texture;
varying vec2 v_TexCoordinate;
void main () {
vec2 m_uv = vec2(v_TexCoordinate.x, 0.5 + v_TexCoordinate.y*0.5);
float mask = texture2D(texture, m_uv).r;
if ( mask < 0.5 )
discard;
vec2 c_uv = vec2(v_TexCoordinate.x, v_TexCoordinate.y*0.5);
vec4 color = texture2D(texture, c_uv);
gl_FragColor = color;
}

Related

Use vec4 color with vec2 texture coordinate

I have a Fragment Shader like this:
final String fragmentShader =
"precision mediump float;" +
"uniform sampler2D u_texture;" +
"varying vec2 v_texCoord;" +
"void main() {" +
" lowp vec4 textureColor = texture2D(u_texture, v_texCoord);" +
" gl_FragColor = textureColor;" +
"}";
When drawing in GLSurfaceView, I will bind a texture into u_texture, and draw with texture coordinate v_texCoord.
But now I want to change to replace the texture with solid RGBA color. I use uniform vec4 vColor; to pass 4 floats directly as color.
I know there are a way to do it using:
"gl_FragColor = (vColor * texture2D(u_texture, v_texCoord));"
But as I said above, I don't want to bind sampler2D u_texture any more, I just want to fill it directly with solid color AND FOLLOW the texture coordinate.

android openGL grid artifact

I have an android app in which I use openGL to make some neat image effects.
The base setup is that I continually draw the buffer to one of two textures, then draw on top of that texture the next frame (essentially a simulation), to give the effect that the same image keeps getting manipulated.
I'm trying to add a fade effect to this, in which I darken the background, then draw on top of that, but I'm getting a very strange artifact from this. Here is my fragment shader:
private final String mCompositeFragmentShader =
"#extension GL_OES_EGL_image_external : require\n" +
"precision mediump float;\n" +
"varying vec2 bTextureCoord;\n" +
"varying vec2 vTextureCoord;\n" +
"uniform samplerExternalOES sTexture;\n" +
"uniform sampler2D bufferTexture;\n" +
"uniform float gamma; \n" +
"void main() {\n" +
" vec4 camera = texture2D(sTexture, vTextureCoord);\n" +
" camera.r = pow(camera.r, 1.0/gamma);\n" +
" camera.g = pow(camera.g, 1.0/gamma);\n" +
" camera.b = pow(camera.b, 1.0/gamma);\n" +
" vec4 bg = texture2D(bufferTexture, bTextureCoord);\n" +
" vec3 color = max(camera.rgb, bg.rgb -vec3(.02, .02, .02));\n"+
" gl_FragColor = vec4(color,1.0);\n" +
"}\n";
with the relevant bit of code being:
vec3 color = max(camera.rgb, bg.rgb -vec3(.02, .02, .02));
So I would expect light areas to eventually fade to black over time. However, instead of black it's fading to a magenta grid:
You can see that the gridding is different for each triangle being drawn on the screen (I'm only drawing two triangles).
Here is the vertex shader, just in case:
"uniform mat4 uMVPMatrix;\n" +
"uniform mat4 uSTMatrix;\n" +
"attribute vec4 aPosition;\n" +
"attribute vec4 aTextureCoord;\n" +
"varying vec4 tempCoords;" +
"varying vec2 vTextureCoord;\n" +
"varying vec2 bTextureCoord;\n" +
"void main() {\n" +
" gl_Position = aPosition;\n" +
" bTextureCoord = aTextureCoord.xy;\n" +
" tempCoords = aTextureCoord;\n" +
" tempCoords.y = aTextureCoord.y * ("+previewSize.height+".0 /"+previewSize.width+".0);\n" +
" vTextureCoord = (uSTMatrix * tempCoords).xy;\n" +
"}\n";
Anyone have any idea what's causing this, and how to fix it?

How do I draw a triangle using a solid color without interpolation in the fragment shader?

This should really be a simple task, but I fail miserably. First a confession: I know very little about OpenGL and the few things I've learned come from various tutorials (some of which are probably outdated and deprecated)
I'm trying to draw a quad using a single color defined at runtime. The quad is drawn in the correct size and position but with the wrong color. This is my vertex and fragment shader:
public static final String VS_SOLIDCOLOR =
"uniform mat4 uMVPMatrix;" +
"attribute vec4 aPosition;" +
"attribute vec4 aColor;" +
"varying vec4 vColor;" +
"void main() {" +
" gl_Position = uMVPMatrix * aPosition;" +
" vColor = aColor;" +
"}";
public static final String FS_SOLIDCOLOR =
"precision mediump float;" +
"varying vec4 vColor;" +
"void main() {" +
" gl_FragColor = vColor;" +
"}";
The color is set like this (I have removed gl error check and check for colorHandle == -1):
float r = 1.0f;
float g = 0.0f;
float b = 0.0f;
float a = 1.0f;
final int colorHandle = GLES20.glGetAttribLocation(shaderProgramSolidColor, "aColor");
GLES20.glVertexAttrib4f(colorHandle, r, g, b, a);
I would have expected the above code to result in solid red triangles, but they end up as solid yellow. From what I've read the fragment shader will interpolate the vColor vector, but I don't want that (and I'm not sure why it ends up as yellow). How do I set a color at runtime and get that drawn unchanged on my triangles?
PS If I were to do this in the fragment shader I would get a solid red color for my entire quad:
gl_FragColor = vec4(1,0,0,1);
Let me know if you need me to post more code.
I found what I did wrong. I was under the impression that I could only set variables in the vertex shader at runtime, but it's possible to manipulate the fragment shader as well. My shader should look like this:
public static final String VS_SOLIDCOLOR =
"uniform mat4 uMVPMatrix;" +
"attribute vec4 aPosition;" +
"void main() {" +
" gl_Position = uMVPMatrix * aPosition;" +
"}";
public static final String FS_SOLIDCOLOR =
"precision mediump float;" +
"uniform vec4 uColor;" +
"void main() {" +
" gl_FragColor = uColor;" +
"}";
And set like this:
final int colorHandle = GLES20.glGetUniformLocation(shaderProgramSolidColor, "uColor");
GLES20.glUniform4f(colorHandle, r, g, b, a);

Opengl es 2.0 trying to pass a float value to fragment shader android

How can I pass a float value to fragment shader ?
This is my code on android:
int aUseTexture = GLES20.glGetAttribLocation(program, "uUseTexture");
GLES20.glUniform1f(aUseTexture, 1.0f);
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
Here is my shader:
String verticesShader =
"uniform mat4 uScreen;\n" +
"attribute vec2 aPosition;\n" +
"attribute vec3 aColor;\n" +
"attribute vec2 aTexPos; \n" +
"varying vec2 vTexPos; \n" +
"varying vec3 vColor;\n" +
"void main() {\n" +
" vTexPos = aTexPos; \n" +
" gl_Position = uScreen * vec4(aPosition.xy, 0.0, 1.0);\n" +
" vColor = aColor;\n" +
"}";
// Our fragment shader. Just return vColor.
// If you look at this source and just said 'WTF?', remember
// that all the attributes are defined in the VERTEX shader and
// all the 'varying' vars are considered OUTPUT of vertex shader
// and INPUT of the fragment shader. Here we just use the color
// we received and add a alpha value of 1.
String fragmentShader =
"uniform float uUseTexture; \n" +
"uniform sampler2D uTexture;\n" +
"precision mediump float;\n"+
"varying vec2 vTexPos; \n" +
"varying vec3 vColor;\n" +
"void main(void)\n" +
"{\n" +
" if ( uUseTexture != 1.0 ) \n" +
" gl_FragColor = vec4(vColor.xyz, 1); \n" +
" else \n" +
" gl_FragColor = texture2D(uTexture, vTexPos); \n" +
//" gl_FragColor = vec4(vColor.xyz, 1);\n" +
"}";
You can see the if statement in the fragment shader , that is the one i tried to check if i pass in 1.0 it should do texture else use color.
You are probably using the wrong function call for "uniform variable". Try glGetUniformLocation() as follow:
int aUseTexture = GLES20.glGetUniformLocation(program, "uUseTexture");
Also, the floating point testing (uUseTexture != 1.0) may not be always reliable all the time. You may want to use an integer type.
As far as I know you have to pass the value through the vertex shader before it can get to the fragment shader. e.g. add "uniform float uUseTexture_in; \n" and "varying float uUseTexture; \n" at the top of the vertex shader, in the main function add "uUseTexture = uUseTexture_in;". And your shader should work

Strange OpenGL ES 2.0 bug on Galaxy Note 2

This vertex shader code works on every device except for the Galaxy Note 2.
gl_Position = uMVPMatrix * vPosition;
where if I reverse the matrix multiplication to:
gl_Position = vPosition * uMVPMatrix; I can actually get things to appear.
Unfortunately, the reverse would require me to completely rewrite my transformations library.
Does anyone have any insight on what could be causing this, is this an opengl driver error with the device?
Shader code
private final String vertexShaderCode =
// This matrix member variable provides a hook to manipulate
// the coordinates of the objects that use this vertex shader
"uniform mat4 uMVPMatrix;" +
"attribute vec4 vPosition;" +
"attribute vec2 a_TexCoordinate;" +
"varying vec2 v_TexCoordinate;" +
"void main() {" +
// the matrix must be included as a modifier of gl_Position
"v_TexCoordinate = a_TexCoordinate;" +
"gl_Position = uMVPMatrix * vPosition;" +
"}";
private final String fragmentShaderCode =
"precision mediump float;" +
"uniform sampler2D u_Texture;" +
"varying vec2 v_TexCoordinate;" +
"void main() {" +
" gl_FragColor = texture2D(u_Texture, v_TexCoordinate);" +
//" gl_FragColor = vec4(v_TexCoordinate, 0, 1);" +
"}";
This is not about only a galaxy note 2 platform.
This is a mathematical question. Because both glsl/hlsl uses column major order,
It is a right way to multiply MATRIX x VECTOR
or
you can transpose the matrix using a option in
glUniformMatrix4fv( h_Uniforms[UNIFORMS_PROJECTION], 1, GL_FALSE or GL_TRUE, g_proxtrans.s);
Apparently, It can be a problem not to call this function with the option ( GL_TRUE is to use transposing ) each every frame just try .

Categories

Resources