In OpenGL ES 2.0 for Android, I am drawing a sphere. The sphere appears on the screen as a circle, so I need to add lighting. When I added lighting, instead of it being smooth, like it would be in real life, it is a fine line between light and dark, as shown here:
However, I want it to look like this, where the shading is much smoother and blended:
Here is my vertex shader code:
uniform mat4 u_Matrix;
uniform vec3 u_VectorToLight;
attribute vec4 a_Position;
attribute vec3 a_Color;
attribute vec3 a_Normal;
varying vec3 v_Color;
void main() {
v_Color = a_Color;
vec3 scaledNormal = a_Normal;
scaledNormal = normalize(scaledNormal);
float diffuse = max(dot(scaledNormal, u_VectorToLight), 0.0);
v_Color *= diffuse;
float ambient = 0.1;
v_Color += ambient;
gl_Position = u_Matrix * a_Position;
}
And my fragment shader:
precision mediump float;
varying vec3 v_Color;
void main() {
gl_FragColor = vec4(v_Color, 1.0);
}
The normal is calculated by getting the vector from the center of the sphere to the point on the sphere, then normalizing it (giving it a length of 1)
Here is how I set the colors:
vertices[offset++] = Color.red(color);
vertices[offset++] = Color.green(color);
vertices[offset++] = Color.blue(color);
Where color is 0xffea00.
The problem is with the range of color values you use. OpenGL operates with color component values in the range [0.0, 1.0]. But you are specifying colors in the range [0, 255] in your Java code.
You have two options to fix this:
Divide the color values you get from the Color class by 255.0f.
Specify the colors with type GL_UNSIGNED_BYTE. To do this, store the values in an array/buffer with element type byte, store those values in a VBO, and then set up the vertex attribute with:
glVertexAttribPointer(attrLoc, 3, GL_UNSIGNED_BYTE, GL_TRUE, stride, 0);
Note the value for the 4th argument. While it does not matter for GL_FLOAT attributes, it is critical that you use GL_TRUE in this case, because the byte values need to be normalized.
Related
I am currently programming an application for image processing. To achieve the needed performance, I have to use the GPU to compute the camera input, more specifically use OpenGL ES 2.0.
With the help of this project (https://github.com/yulu/ShaderCam) I achieved to pass the image to the pipeline and do simple operations with the fragment shader (like inverting colors etc).
My knowledge of GLSL, fragment shaders and vertex shaders is fairly limited but I am aware of pipeline constraints and what the two shaders do in the pipeline.
So - formulating the problem - I would like to calculate the average color of a row in my received image and return it (per row) to my application.
I read here https://stackoverflow.com/a/13866636/8038866 that this is generally possible, however I can't seem to find out the following things:
1 (edit: SOLVED by simply passing the w and h of my texture to the vertex and fragment shader):
Knowing where the row ends (and having that information in the fragment shader). For this I assume that I would have to pass the width of the picture to the vertex shader and from there to the fragment shader, right?
2.: How to calculate the average the color values of each row in the fragment shader and then pass them to the application. If I understand it correctly - the fragment shader only excutes the code per pixel, so I am not sure how to achieve this.
Here are the two very basic shaders
vertex shader:
uniform mat4 uTransformM;
uniform mat4 uOrientationM;
uniform vec2 ratios;
attribute vec2 aPosition;
varying vec2 vTextureCoord;
void main(){
gl_Position = vec4(aPosition, 0.0, 1.0);
vTextureCoord = (uTransformM * ((uOrientationM * gl_Position + 1.0)*0.5)).xy;
gl_Position.xy *= ratios;
}
fragment shader:
#extension GL_OES_EGL_image_external : require
precision mediump float;
uniform samplerExternalOES sTexture;
varying vec2 vTextureCoord;
void main(){
gl_FragColor = texture2D(sTexture, vTextureCoord);
//calc mean per row and pass it back
}
I am very thankful for every advice or help you can provide.
I found a way that does the trick for me. The idea is to calculate the mean for only one row of pixels and then later in the application to get this line with
glReadPixels( GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, GLenum type, GLvoid * data);
Here is my fragment shader (notice that the width of the surface is required as well):
#extension GL_OES_EGL_image_external : require
precision mediump float;
uniform samplerExternalOES sTexture;
varying float width;
varying vec2 vTextureCoord;
void main(){
vec4 accumulatedRGB = texture2D(sTexture, vec2(0,vTextureCoord.y));
if(vTextureCoord.x < 0.50 && vTextureCoord.x > 0.499){ //small enough to only cover one line
for(float i=1.0;i<=width;++i)
{
float xPosOnTexture = i/width;
vec4 current = texture2D(sTexture, vec2(xPosOnTexture,vTextureCoord.y));
accumulatedRGB += current;
}
vec4 mean = accumulatedRGB/width;
gl_FragColor = vec4(mean.rgb , mean.a);//avg color for one line
}
else{
gl_FragColor = vec4(0.0,0.0,0.0,0.0);//rest of the screen
}
}
I use Vuforia SDK to render the video stream of my phone's camera on the screen.
So the texture is generated by the Vuforia library, not me.
The shaders used to render this background are:
// Vertex Shader
attribute vec4 a_position;
attribute vec2 a_textureCoords;
varying vec2 v_textureCoords;
uniform mat4 u_projectionMatrix;
void main()
{
gl_Position = u_projectionMatrix * a_position;
v_textureCoords = a_textureCoords;
}
// Fragment Shader
varying highp vec2 v_textureCoords;
uniform sampler2D u_currentTexture;
void main()
{
vec4 currentColor = texture2D(u_currentTexture, v_textureCoords);
gl_FragColor = currentColor;
}
Now, I want an overlay in the upper-left corner of the screen:
I don't want this overlay to display only a pink texture, but rather a multiply blend of the pink texture and the background texture. Note that the textures do not have the same coordinates.
But for now, let's forget about the blending and let's just render the background texture in the shader program of the pink texture. So in the end, yes, one should see no difference between the background-only version and the bacground with overlay version.
As you can see (look at the painting and the top of the chair), there is a small offset...
The shaders used to render the overlay are:
// Vertex Shader
attribute vec4 a_position;
attribute vec2 a_currentTextureCoords;
varying vec2 v_currentTextureCoords;
void main()
{
gl_Position = a_position;
v_currentTextureCoords = a_currentTextureCoords;
}
// Fragment Shader
varying highp vec2 v_currentTextureCoords;
uniform sampler2D u_currentTexture;
uniform sampler2D u_backgroundTexture;
void main()
{
vec2 screenSize = vec2(1080.0, 1920.0);
vec2 cameraResolution = vec2(720.0, 1280.0);
vec2 texelSize = vec2(1.0 / screenSize.x, 1.0 / screenSize.y);
vec2 scaleFactor = vec2(cameraResolution.x / screenSize.x, cameraResolution.y / screenSize.y);
vec2 uv = gl_FragCoord.xy * texelSize * scaleFactor;
uv = vec2(scaleFactor.y - uv.y, scaleFactor.x - uv.x);
vec4 backgroundColor = texture2D(u_backgroundTexture, uv);
gl_FragColor = backgroundColor;
}
Are my calculations wrong?
Why do you need this line?
uv = vec2(scaleFactor.y - uv.y, scaleFactor.x - uv.x);
Not sure what arithmetic relationship the absolute texture coordinates have with the scale factor which needs an addition or a subtraction ...
P.S. it's not related to your question, but your shaders will be shorter and easier to read if you just use the vector operations in the language. For example, replace:
vec2 scaleFactor = vec2(cameraResolution.x / screenSize.x, cameraResolution.y / screenSize.y);
... with ...
vec2 scaleFactor = cameraResolution / screenSize;
As long as the vector types are the same length, it will do exactly what you expect with a lot less typing ...
I am trying to write a shader for a opengles 2.0 view in Android.
My shader is :
Vertex shader:
uniform mat4 u_MVPMatrix; // A constant representing the combined model/view/projection matrix.
uniform mat4 u_MVMatrix; // A constant representing the combined model/view matrix.
attribute vec4 a_Position; // Per-vertex position information we will pass in.
attribute vec3 a_Normal; // Per-vertex normal information we will pass in.
attribute vec2 a_TexCoordinate; // Per-vertex texture coordinate information we will pass in.
varying vec3 v_Position; // This will be passed into the fragment shader.
varying vec3 v_Normal; // This will be passed into the fragment shader.
varying vec2 v_TexCoordinate; // This will be passed into the fragment shader.
// The entry point for our vertex shader.
void main()
{
// Transform the vertex into eye space.
v_Position = vec3(u_MVMatrix * a_Position);
// Pass through the texture coordinate.
v_TexCoordinate = a_TexCoordinate;
// Transform the normal's orientation into eye space.
v_Normal = normalize(vec3(u_MVMatrix * vec4(a_Normal, 0.0)));
// gl_Position is a special variable used to store the final position.
// Multiply the vertex by the matrix to get the final point in normalized screen coordinates.
gl_Position = (u_MVPMatrix * a_Position);
}
Fragment Shader:
precision highp float; // Set the default precision to medium. We don't need as high of a
// precision in the fragment shader.
uniform vec3 u_LightPos1; // The position of the light in eye space.
uniform vec3 u_LightDir1; // The position of the light in eye space.
float l_spotCutOff=45.0;
uniform sampler2D u_Texture; // The input texture.
varying vec3 v_Position; // Interpolated position for this fragment.
varying vec3 v_Normal; // Interpolated normal for this fragment.
varying vec2 v_TexCoordinate; // Interpolated texture coordinate per fragment.
float cutoff = 0.1;
// The entry point for our fragment shader.
void main()
{
// Get a lighting direction vector from the light to the vertex.
vec3 lightVector1 = normalize(u_LightPos1 - v_Position);
// Will be used for attenuation.
float distance1 = length(u_LightPos1 - v_Position);
float diffuse=0.0;
// Calculate the dot product of the light vector and vertex normal. If the normal and light vector are
// pointing in the same direction then it will get max illumination.
float diffuse1 = max(dot(v_Normal, lightVector1), 0.1);
// Add attenuation.
diffuse1 = diffuse1 * (1.0 / (1.0+(0.25*distance1)));
// Add ambient lighting
diffuse = diffuse1+0.2;
// Multiply the color by the diffuse illumination level and texture value to get final output color.
vec4 color = (texture2D(u_Texture, v_TexCoordinate));
color.rgb *= (diffuse);
if( color.a < cutoff)
discard;
gl_FragColor = color;
}
Now the shaders are working perfectly but its behaving differently in different device:
Device 1: (moto x play)
1
Device 2: (Samsung S7)
2
Can anyone help?
The issue can be there in texture format/type which you have used. All devices doesn't support all texture formats.
for eg : if your output color can have negative values and the texture format of device doesn't support them, it will get clamped to 0 and might give different results.
Better to check capabilities of both devices using
GLES20.glGetString(GLES20.GL_EXTENSIONS));
I have written this simple shader to overlay texture over another (base) texture -
varying highp vec2 textureCoordinate;
varying highp vec2 textureCoordinate2;
uniform sampler2D inputImageTexture;
uniform sampler2D inputImageTexture2;
void main()
{
mediump vec4 base = texture2D(inputImageTexture, textureCoordinate);
mediump vec4 overlay = texture2D(inputImageTexture2, textureCoordinate2);
mediump float ra = (overlay.a) * overlay.r + (1.0 - overlay.a) * base.r;
mediump float ga = (overlay.a) * overlay.g + (1.0 - overlay.a) * base.g;
mediump float ba = (overlay.a) * overlay.b + (1.0 - overlay.a) * base.b;
gl_FragColor = vec4(ra, ga, ba, 1.0);
}
Issue - This works except for one issue. If the overlay image is smaller than the base image, the outside region of overlay image gives alpha value of 1.0, i.e overlay.a == 1.0. Due to this the base image is clipped by overlay image. The region outside overlay appears as black.
I am new to opengl, and was expecting that outside its bounds, the texture's alpha should appear as 0.0? How can I fix my shader code to achieve desired behaviour? Or do I need to modify my graphics pipeline?
EDIT Vertex shader below-
attribute vec4 inputTextureCoordinate2;
varying vec2 textureCoordinate;
varying vec2 textureCoordinate2;
void main()
{
gl_Position = pos;
textureCoordinate = uv;
textureCoordinate2 = inputTextureCoordinate2.xy;
}
I was expecting that outside its bounds, the texture's alpha should appear as 0.0
How are you sampling the texture outside of its bounds? When sampling a texture, the uv coordinates should range from 0 to 1. If the coordinates are outside of this range, then one of two things will happen:
If GL_CLAMP_TO_EDGE is set, then the cooridnate will be clamped to the (0, 1) range, and you'll sample an edge pixel
If GL_REPEAT is set, then the fractional part of the coordinate will be taken, and you'll sample somewhere in the middle of the texture
See the docs on glTexParameter for more details.
If your use case is simply overlaying images, perhaps you should try writing a pixel shader.
Set the viewport to the base image dimensions and draw a quad from (-1, 1).
Your fragment shader will now operate on each pixel, known as a texel. Get the texel with gl_FragCoord
Sample the base and overlay by texel e.g. using texelFetch
If the texel is outside of the overlay, set the overlay's rgba values to 0
For example
//fragment shader
uniform ivec2 overlayDim;
uniform sampler2D baseTexture;
uniform sampler2D overlayTexture;
void main() {
vec2 texelf = floor(gl_FragCoord.xy);
ivec2 texel = (int(texelf.x), int(texelf.y));
vec4 base = texelFetch(baseTexture, texel, 0);
vec4 overlay = texelFetch(overlayTexture, texel, 0);
float overlayIsValid = float(texel.x < overlayDim.x && texel.y < overlayDim.y);
overlay *= overlayIsValid;
//rest of code
}
What happens if you sample outside the range of the texture is controlled by the value you set for GL_TEXTURE_WRAP_S and GL_TEXTURE_WRAP_T using glTexParameteri().
In full OpenGL, you could set the value to GL_CLAMP_TO_BORDER, set the border color to a value with alpha 0.0, and be done with it. But texture borders are not available in OpenGL ES 2.0 (the option is introduced in ES 3.2, but not in earlier versions).
Without this, I can think of two options:
If you have control over the texture data, you could set a one pixel border to transparent values. The GL_CLAMP_TO_EDGE then gives you a transparent value when sampling outside the range.
Check the range in the fragment shader.
The fragment shader code for the second option could look something like this (untested):
mediump vec3 col = texture2D(inputImageTexture, textureCoordinate).xyz;
if (all(greaterThan(textureCoordinate2, vec2(0.0))) &&
all(lessThan(textureCoordinate2, vec2(1.0))))
{
mediump vec3 overlay = texture2D(inputImageTexture2, textureCoordinate2).xyz;
col = mix(col, overlay, overlay.a);
}
gl_FragColor = vec4(col, 1.0);
Compared to your original code, also note the use of vector operations. Whenever there is a good way of operating on vectors, it will make the code simpler. It will also make the job of the optimizer easier for GPUs with vector operations.
I found the issue in my code. I had
GLES20.glClearColor(0, 0, 0, 1);
in my code. Changing it to -
GLES20.glClearColor(0, 0, 0, 0);
fixed the issue.
Also as mentioned by #Reto, I have changed my fragment shader to use vector operations for optimisation.
void main()
{
mediump vec4 overlay = texture2D(inputImageTexture2, textureCoordinate2);
mediump vec3 col = texture2D(inputImageTexture, textureCoordinate).xyz;
col = mix(col, overlay.xyz, overlay.a);
gl_FragColor = vec4(col, 1.0);
}
I want to colorize a sprite so that RGB channels are all 1 and alpha remains unchanged.
I gather this should be done with shaders, but the two accepted answers on StackOverflow (Change sprite color into white and libgdx changing sprite color while hurt) don't work for me - the result is transparent, and they don't work on http://shdr.bkcore.com/ either
All you need is to replace the RGB each with 1.0 in the fragment shader.
Vertex shader-- This is like the one in SpriteBatch with vertex color removed since you aren't using it:
attribute vec4 a_position;
attribute vec2 a_texCoord0;
uniform mat4 u_projTrans;
void main()
{
v_texCoords = a_texCoord0;
gl_Position = u_projTrans * a_position;
}
Fragment shader-- grab just the alpha value from the texture.:
#ifdef GL_ES
precision lowp float; //since the only value we're storing is part of a color
#endif
varying vec2 v_texCoords;
uniform sampler2D u_texture;
void main()
{
float alpha = texture2D(u_texture, v_texCoords).a;
gl_FragColor = vec4(1.0, 1.0, 1.0, alpha);
}