I've been trying to figure out why my shader doesn't work on Android while it does on desktop. With not working I mean the shader makes the texture unable to be seen. Here's the code of my fragment shader:
precision mediump float;
varying vec4 v_color;
varying vec2 v_texCoord0;
varying float randomFloat;
uniform vec3 dif_color;
uniform sampler2D u_sampler2D;
vec4 color_right;
vec4 color_left;
vec4 color_bottom;
vec4 color_top;
vec3 colorNeighbors;
float neighbors;
void main(){
vec4 color = texture2D(u_sampler2D, v_texCoord0) * v_color;
ivec2 texSize = textureSize(u_sampler2D, 0);
ivec2 texIndex = ivec2(int(v_texCoord0.x * float(texSize.x)), int(v_texCoord0.y * float(texSize.y)));
color_right = texelFetch( u_sampler2D, texIndex + ivec2(1, 0), 0).rgba;
color_left = texelFetch( u_sampler2D, texIndex + ivec2(-1, 0), 0).rgba;
color_top = texelFetch( u_sampler2D, texIndex + ivec2(0, 1), 0).rgba;
color_bottom = texelFetch( u_sampler2D, texIndex + ivec2(0, -1), 0).rgba;
if(color_right.a != 0){ neighbors++; }
if(color_left.a != 0){ neighbors++; }
if(color_top.a != 0){ neighbors++; }
if(color_bottom.a != 0){ neighbors++; }
vec3 colorNeighbors = (color_right.rgb + color_left.rgb + color_top.rgb + color_bottom.rgb) / neighbors;
if((color_right.a != 0 || color_left.a != 0 || color_top.a != 0 || color_bottom.a != 0) && color.a == 0){
color.rgba = vec4(.1 + .3 * randomFloat + colorNeighbors,1.0);
}
gl_FragColor = color;
}
I've read many similar questions to mine. Answers to these advise adding a certain line: precision mediump float; and using floating points instead of integers. I've applied this to my code, but with no different result from not working.
Other answers are totally irrelevant to my code. I assume the problem in my code lies in this line:
ivec2 texSize = textureSize(u_sampler2D, 0);
When I remove this line (along with the lines dependent on this variable) the shader operates, but of course not in the way I want it to.
So, maybe someone knows a different way to get the texture size? But I'd also like to know what causes the shader to disfunction?
According to Khronos specification textureSize is available only in OpenGL ES SL 3.00. If you want to use it you should rewrite your code for OpenGL ES [SL] 3 format.
Also, check shader compilation log to see if there are other errors.
Don't use textureSize. Use the same values in a Vector2 uniform.
Be careful to send another float uniform with the devicePixelRatio. if don't, the texture in devices with retina display will looks to half size.
Related
I am trying to write an opengl shader that applies Vigenette Shader. But the issue I am facing is that it shows up a circle as you can see in the right image.
I want to find out what exactly must have been going wrong in my code.
I have also pasted the code below too.
Code For My Vigenette Shader
precision mediump float;
uniform sampler2D u_Texture;
uniform sampler2D u_Vigenette;
uniform sampler2D u_Map;
varying vec2 v_TexCoordinate;
void main()
{
vec3 texel = texture2D(u_Texture, v_TexCoordinate).rgb;
texel.r = (texel.r == 1.0)?.9961:texel.r;
texel.g = (texel.g == 1.0)?.9961:texel.g;
texel.b = (texel.b == 1.0)?.9961:texel.b;
texel = vec3(
texture2D(u_Map, vec2(texel.r, .16666)).r,
texture2D(u_Map, vec2(texel.g, .5)).g,
texture2D(u_Map, vec2(texel.b, .83333)).b);
texel.r = (texel.r == 1.0)?.9961:texel.r;
texel.g = (texel.g == 1.0)?.9961:texel.g;
texel.b = (texel.b == 1.0)?.9961:texel.b;
vec2 tc = (2.0 * v_TexCoordinate) - 1.0;
float d = dot(tc, tc);
vec2 lookup = vec2(d, texel.r);
texel.r = texture2D(u_Vigenette, lookup).r;
lookup.y = texel.g;
texel.g = texture2D(u_Vigenette, lookup).g;
lookup.y = texel.b;
texel.b = texture2D(u_Vigenette, lookup).b;
gl_FragColor = vec4(texel, 1.0);
}
Thanks in Advance for the help.
Bit of a guess because there's not enough information to be sure, but I think that your code is expecting the result of
float d = dot(tc, tc);
to be in the range 0 to 1, but it's actually in the range 0 to 2.
Perhaps you want to rescale it, or perhaps the u_Vigenette texture is set to repeat instead of clamp to edge.
I have written this simple shader to overlay texture over another (base) texture -
varying highp vec2 textureCoordinate;
varying highp vec2 textureCoordinate2;
uniform sampler2D inputImageTexture;
uniform sampler2D inputImageTexture2;
void main()
{
mediump vec4 base = texture2D(inputImageTexture, textureCoordinate);
mediump vec4 overlay = texture2D(inputImageTexture2, textureCoordinate2);
mediump float ra = (overlay.a) * overlay.r + (1.0 - overlay.a) * base.r;
mediump float ga = (overlay.a) * overlay.g + (1.0 - overlay.a) * base.g;
mediump float ba = (overlay.a) * overlay.b + (1.0 - overlay.a) * base.b;
gl_FragColor = vec4(ra, ga, ba, 1.0);
}
Issue - This works except for one issue. If the overlay image is smaller than the base image, the outside region of overlay image gives alpha value of 1.0, i.e overlay.a == 1.0. Due to this the base image is clipped by overlay image. The region outside overlay appears as black.
I am new to opengl, and was expecting that outside its bounds, the texture's alpha should appear as 0.0? How can I fix my shader code to achieve desired behaviour? Or do I need to modify my graphics pipeline?
EDIT Vertex shader below-
attribute vec4 inputTextureCoordinate2;
varying vec2 textureCoordinate;
varying vec2 textureCoordinate2;
void main()
{
gl_Position = pos;
textureCoordinate = uv;
textureCoordinate2 = inputTextureCoordinate2.xy;
}
I was expecting that outside its bounds, the texture's alpha should appear as 0.0
How are you sampling the texture outside of its bounds? When sampling a texture, the uv coordinates should range from 0 to 1. If the coordinates are outside of this range, then one of two things will happen:
If GL_CLAMP_TO_EDGE is set, then the cooridnate will be clamped to the (0, 1) range, and you'll sample an edge pixel
If GL_REPEAT is set, then the fractional part of the coordinate will be taken, and you'll sample somewhere in the middle of the texture
See the docs on glTexParameter for more details.
If your use case is simply overlaying images, perhaps you should try writing a pixel shader.
Set the viewport to the base image dimensions and draw a quad from (-1, 1).
Your fragment shader will now operate on each pixel, known as a texel. Get the texel with gl_FragCoord
Sample the base and overlay by texel e.g. using texelFetch
If the texel is outside of the overlay, set the overlay's rgba values to 0
For example
//fragment shader
uniform ivec2 overlayDim;
uniform sampler2D baseTexture;
uniform sampler2D overlayTexture;
void main() {
vec2 texelf = floor(gl_FragCoord.xy);
ivec2 texel = (int(texelf.x), int(texelf.y));
vec4 base = texelFetch(baseTexture, texel, 0);
vec4 overlay = texelFetch(overlayTexture, texel, 0);
float overlayIsValid = float(texel.x < overlayDim.x && texel.y < overlayDim.y);
overlay *= overlayIsValid;
//rest of code
}
What happens if you sample outside the range of the texture is controlled by the value you set for GL_TEXTURE_WRAP_S and GL_TEXTURE_WRAP_T using glTexParameteri().
In full OpenGL, you could set the value to GL_CLAMP_TO_BORDER, set the border color to a value with alpha 0.0, and be done with it. But texture borders are not available in OpenGL ES 2.0 (the option is introduced in ES 3.2, but not in earlier versions).
Without this, I can think of two options:
If you have control over the texture data, you could set a one pixel border to transparent values. The GL_CLAMP_TO_EDGE then gives you a transparent value when sampling outside the range.
Check the range in the fragment shader.
The fragment shader code for the second option could look something like this (untested):
mediump vec3 col = texture2D(inputImageTexture, textureCoordinate).xyz;
if (all(greaterThan(textureCoordinate2, vec2(0.0))) &&
all(lessThan(textureCoordinate2, vec2(1.0))))
{
mediump vec3 overlay = texture2D(inputImageTexture2, textureCoordinate2).xyz;
col = mix(col, overlay, overlay.a);
}
gl_FragColor = vec4(col, 1.0);
Compared to your original code, also note the use of vector operations. Whenever there is a good way of operating on vectors, it will make the code simpler. It will also make the job of the optimizer easier for GPUs with vector operations.
I found the issue in my code. I had
GLES20.glClearColor(0, 0, 0, 1);
in my code. Changing it to -
GLES20.glClearColor(0, 0, 0, 0);
fixed the issue.
Also as mentioned by #Reto, I have changed my fragment shader to use vector operations for optimisation.
void main()
{
mediump vec4 overlay = texture2D(inputImageTexture2, textureCoordinate2);
mediump vec3 col = texture2D(inputImageTexture, textureCoordinate).xyz;
col = mix(col, overlay.xyz, overlay.a);
gl_FragColor = vec4(col, 1.0);
}
I've done image warping using OpenCV in Python and C++, see the Coca Cola logo warped in place in the corners I had selected:
Using the following images:
and this:
Full album with transition pics and description here
I need to do exactly this, but in OpenGL. I'll have:
Corners inside which I've to map the warped image
A homography matrix that maps the transformation of the logo image
into the logo image you see inside the final image (using OpenCV's
warpPerspective), something like this:
[[ 2.59952324e+00, 3.33170976e-01, -2.17014066e+02],
[ 8.64133587e-01, 1.82580111e+00, -3.20053715e+02],
[ 2.78910149e-03, 4.47911310e-05, 1.00000000e+00]]
Main image (the running track image here)
Overlay image (the Coca Cola image here)
Is it possible ? I've read a lot and started OpenGL basics tutorials, but can it be done from just what I have? Would the OpenGL implementation be faster, say, around ~10ms?
I'm currently playing with this tutorial here:
http://ogldev.atspace.co.uk/www/tutorial12/tutorial12.html
Am I going in the right direction? Total OpenGL newbie here, please bear. Thanks.
After trying a number of solutions proposed here and elsewhere, I ended solving this by writing a fragment shader that replicates what 'warpPerspective' does.
The fragment shader code looks something like:
varying highp vec2 textureCoordinate;
uniform sampler2D inputImageTexture;
// NOTE: you will need to pass the INVERSE of the homography matrix, as well as
// the width and height of your image as uniforms!
uniform highp mat3 inverseHomographyMatrix;
uniform highp float width;
uniform highp float height;
void main()
{
// Texture coordinates will run [0,1],[0,1];
// Convert to "real world" coordinates
highp vec3 frameCoordinate = vec3(textureCoordinate.x * width, textureCoordinate.y * height, 1.0);
// Determine what 'z' is
highp vec3 m = inverseHomographyMatrix[2] * frameCoordinate;
highp float zed = 1.0 / (m.x + m.y + m.z);
frameCoordinate = frameCoordinate * zed;
// Determine translated x and y coordinates
highp float xTrans = inverseHomographyMatrix[0][0] * frameCoordinate.x + inverseHomographyMatrix[0][1] * frameCoordinate.y + inverseHomographyMatrix[0][2] * frameCoordinate.z;
highp float yTrans = inverseHomographyMatrix[1][0] * frameCoordinate.x + inverseHomographyMatrix[1][1] * frameCoordinate.y + inverseHomographyMatrix[1][2] * frameCoordinate.z;
// Normalize back to [0,1],[0,1] space
highp vec2 coords = vec2(xTrans / width, yTrans / height);
// Sample the texture if we're mapping within the image, otherwise set color to black
if (coords.x >= 0.0 && coords.x <= 1.0 && coords.y >= 0.0 && coords.y <= 1.0) {
gl_FragColor = texture2D(inputImageTexture, coords);
} else {
gl_FragColor = vec4(0.0,0.0,0.0,0.0);
}
}
Note that the homography matrix we are passing in here is the INVERSE HOMOGRAPHY MATRIX! You have to invert the homography matrix that you would pass into 'warpPerspective'- otherwise this code will not work.
The vertex shader does nothing but pass through the coordinates:
// Vertex shader
attribute vec4 position;
attribute vec4 inputTextureCoordinate;
varying vec2 textureCoordinate;
void main() {
// Nothing happens in the vertex shader
textureCoordinate = inputTextureCoordinate.xy;
gl_Position = position;
}
Pass in unaltered texture coordinates and position coordinates (i.e. textureCoordinates = [(0,0),(0,1),(1,0),(1,1)] and positionCoordinates = [(-1,-1),(-1,1),(1,-1),(1,1)], for a triangle strip), and this should work!
You can do perspective warping of the texture using texture2DProj(), or alternatively using texture2D() by dividing the st coordinates of the texture (which is what texture2DProj does).
Have a look here: Perspective correct texturing of trapezoid in OpenGL ES 2.0.
warpPerspective projects the (x,y,1) coordinate with the matrix and then divides (u,v) by w, like texture2DProj(). You'll have to modify the matrix so the resulting coordinates are properly normalised.
In terms of performance, if you want to read the data back to the CPU your bottleneck is glReadPixels. How long it will take depends on your device. If you're just displaying, the OpenGL ES calls will take much less than 10ms, assuming that you have both textures loaded to GPU memory.
[edit] This worked on my Galaxy S9 but on my car's Android it had an issue that the whole output texture was white. I've sticked to the original shader and it works :)
You can use mat3*vec3 ops in the fragment shader:
varying highp vec2 textureCoordinate;
uniform sampler2D inputImageTexture;
uniform highp mat3 inverseHomographyMatrix;
uniform highp float width;
uniform highp float height;
void main()
{
highp vec3 frameCoordinate = vec3(textureCoordinate.x * width, textureCoordinate.y * height, 1.0);
highp vec3 trans = inverseHomographyMatrix * frameCoordinate;
highp vec2 coords = vec2(trans.x / width, trans.y / height) / trans.z;
if (coords.x >= 0.0 && coords.x <= 1.0 && coords.y >= 0.0 && coords.y <= 1.0) {
gl_FragColor = texture2D(inputImageTexture, coords);
} else {
gl_FragColor = vec4(0.0,0.0,0.0,0.0);
}
};
If you want to have transparent background don't forget to add
GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
GLES20.glBlendEquation(GLES20.GL_FUNC_ADD);
And set transpose flag (in case you use the above shader):
GLES20.glUniformMatrix3fv(H_P2D, 1, true, homography, 0);
My terrain uses shader which itself uses four different textures. It runs fine on windows and linux machines, but on android it gets only ~25FPS on both galaxies. I thought, that textures are the problem, but no, as it appears the problem is with the part where I divide texture coordinates and use frac to get tiled coordinates. Without it, I get 60FPS.
// Material data.
//uniform vec3 uAmbient;
//uniform vec3 uDiffuse;
//uniform vec3 uLightPos[8];
//uniform vec3 uEyePos;
//uniform vec3 uFogColor;
uniform sampler2D terrain_blend;
uniform sampler2D grass;
uniform sampler2D rock;
uniform sampler2D dirt;
varying vec2 varTexCoords;
//varying vec3 varEyeNormal;
//varying float varFogWeight;
//------------------------------------------------------------------
// Name: fog
// Desc: applies calculated fog weight to fog color and mixes with
// specified color.
//------------------------------------------------------------------
//vec4 fog(vec4 color) {
// return mix(color, vec4(uFogColor, 1.0), varFogWeight);
//}
void main(void)
{
/*vec3 N = normalize(varEyeNormal);
vec3 L = normalize(uLightPos[0]);
vec3 H = normalize(L + normalize(uEyePos));
float df = max(0.0, dot(N, L));
vec3 col = uAmbient + uDiffuse * df;*/
// Take color information from textures and tile them.
vec2 tiledCoords = varTexCoords;
//vec2 tiledCoords = fract(varTexCoords / 0.05); // <========= HERE!!!!!!!!!
//vec4 colGrass = texture2D(grass, tiledCoords);
vec4 colGrass = texture2D(grass, tiledCoords);
//vec4 colDirt = texture2D(dirt, tiledCoords);
vec4 colDirt = texture2D(dirt, tiledCoords);
//vec4 colRock = texture2D(rock, tiledCoords);
vec4 colRock = texture2D(rock, tiledCoords);
// Take color information from not tiled blend map.
vec4 colBlend = texture2D(terrain_blend, varTexCoords);
// Find the inverse of all the blend weights.
float inverse = 1.0 / (colBlend.r + colBlend.g + colBlend.b);
// Scale colors by its corresponding weight.
colGrass *= colBlend.r * inverse;
colDirt *= colBlend.g * inverse;
colRock *= colBlend.b * inverse;
vec4 final = colGrass + colDirt + colRock;
//final = fog(final);
gl_FragColor = final;
}
Note: there's some more code for light calculation and fog, but it isn't used. I indicated the line that, when uncommented, causes massive lag. I tried using floor and calculating fractional part manually, but lag is the same. What might be wrong?
EDIT: Now here's what I don't understand.
This:
vec2 tiledCoords = fract(varTexCoords * 2.0);
Runs great.
This:
vec2 tiledCoords = fract(varTexCoords * 10.0);
Runs average on SIII.
This:
vec2 tiledCoords = fract(varTexCoords * 20.0);
Lags...
This:
vec2 tiledCoords = fract(varTexCoords * 100.0);
Well 5FPS is still better than I expected...
So what gives? Why is this happening? To my understanding this shouldn't make any difference. But it does. And a huge one.
I would run your code on a profiler (check Mali-400), but by the looks of it, you are killing the texture cache. For the first pixel computed, all those 4 texture look-ups are fetched but also the contiguous data is also fetched into the texture cache. For the next pixel, you are not accessing data in the cache but looking quite far (10, 20..etc) which completely defies the purpose of such a cache.
This of course a guess, without proper profiling is hard to tell.
EDIT: #harism also pointed you to that direction.
I have an application drawing some objects with OpenG-ES 2.0.The application fails to render on some of samsung devices. I tried debugging and it seems to be problem with the vertex and fragment shaders.
Here is my shader code:
Vertex Shader:
attribute vec3 Position;
attribute vec4 SourceColor;
varying vec4 DestinationColor;
uniform mat4 Projection;
uniform mat4 Modelview;
uniform mat4 CordTransform;
attribute float flag;
attribute float clubColorFlag;
attribute vec2 TexCoordIn; // New
varying vec2 TexCoordOut; // New
varying float flagS;
varying float flagClubColorS;
void main(void) {
gl_Position = Projection * Modelview * vec4(Position,1.0);
flagS = flag;
flagClubColorS = clubColorFlag;
if (clubColorFlag == 1.0) {
DestinationColor = vec4(0.190,0.309,0.309,1.0);
}
else {
DestinationColor = SourceColor;
}
exCoordOut = TexCoordIn;
gl_PointSize = 1.0;
}
Fragment Shader:
varying lowp vec4 DestinationColor;
varying lowp vec2 TexCoordOut; // New
uniform sampler2D Texture; // New
varying lowp float flagS;
varying lowp float flagClubColorS;
void main(void) {
gl_FragColor = DestinationColor;
if(flagS == 1.0){
gl_FragColor = DestinationColor;
}
else if (flagClubColorS == 1.0) {
gl_FragColor = DestinationColor;
}
else if (flagS == 0.0){
gl_FragColor = DestinationColor * texture2D(Texture, TexCoordOut); // New
}
}
I am not sure what is the problem but I get texture uniform -1 if I comment out the if-else part in fragment shader. In other scenario, it is zero. Both shaders compile without any errors.
Is it related to precision? Please help me to debug the issue.
I am answering my own question for people who are going to refer this later. I could kind of solve the issue.
The problem was not in either of the shaders. The issue was, I was using glDrawElements() method to draw objects with vertex and index buffers. I replaced this call with glDrawArrays() method using only vertex buffer and everything worked fine.
I am still not sure of the exact issue but it may help somebody struggling long for similar issue.