Develop Reference

GLSL - fragment shaders for image manipulation/blending

I'm trying to build a shader that allows you to combine two images by applying a gradient opacity mask to the one on top, like you do in photoshop. I've gotten to the point where I can overlay the masked image over the other but as a newbie am confused about a few things.
It seems that the images inside the shader are sometimes skewed to fit the canvas size, and they always start at position 0,0. I have played around with a few snippets I have found to try and scale the textures, but always end up with unsatisfactory results.
I am curious if there is a standard way to size, skew, and translate textures within a view, or if images in GLSL are necessarily limited in some way that will stop me from accomplishing my goal.
I'm also unsure of how I am applying the gradient/mask and if it is the right way to do it, because I do not have a lot of control over the shape of the gradient at the moment.
Here's what I have so far:
precision highp float;
varying vec2 uv;
uniform sampler2D originalImage;
uniform sampler2D image;
uniform vec2 resolution;
void main(){
float mask;
vec4 result;
vec2 position = gl_FragCoord.xy / ((resolution.x + resolution.y) * 2.0 );
mask = vec4(1.0,1.0,1.0,1.0 - position.x).a;
vec4 B = texture2D(image,uv);
vec4 A = texture2D(originalImage,uv) * (mask);
result = mix(B, A, A.a);
gl_FragColor = result;
}
Which produces an image like this:
What I would like to be able to do is change the positions of the images independently and also make sure that they conform to their proper dimensions.
I have tried naively shifting positions like this:
vec2 pos = uv;
pos.y = pos.y + 0.25;
texture2D(image, pos)
Which does shift the texture, but leads to a bunch of strange lines dragging:
I tried to get rid of them like this:
gl_FragColor = uv.y < 0.25 ? vec4(0.0,0.0,0.0,0.0) : result;
but it does nothing

You really need to decide what you want to happen when images are not the same size. What you probably want is for it to appear there's no image so check your UV coordinates and use 0,0,0,0 when outside of the image
//vec4 B = texture2D(image,uv);
vec4 getImage(sampler2D img, vec2 uv) {
if (uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0) {
return vec4(0);
}
return texture2D(img, uv);
}
vec4 B = getImage(image, uv);
As for a standard way to size/skew/translate images use a matrix
uniform mat4 u_imageMatrix;
...
vec2 newUv = u_imageMatrix * vec4(uv, 0, 1).xy;
An example of implementing canvas 2d's drawImage using a texture matrix.
In general though I don't think most image manipulations programs/library would try to do everything in the shader. Rather they'd build up the image with very very simple primitives. My best guess would be they use a shader that's just A * MASK then draw B followed by A * MASK with blending on.
To put it another way, if you have 30 layers in photoshop they wouldn't generate a single shader that computes the final image in one giant shader taking in all 30 layers at once. Instead each layer would be applied on its own with simpler shaders.
I also would expect them to create an texture for the mask instead of using math in the shader. That way the mask can be arbitrarily complex, not just a 2 stop ramp.
Note I'm not saying you're doing it wrong. You're free to do whatever you want. I'm only saying I suspect that if you want to build a generic image manipulation library you'll have more success with smaller building blocks you combine rather than trying to do more complex things in shaders.
ps: I think getImage can be simplified to
vec4 getImage(sampler2D img, vec2 uv) {
return texture2D(img, uv) * step(0.0, uv) * step(-1.0, -uv);
}

Vertex texture Fetch (Texture read in vertex shader)

New to Opengl and GLSL.
I am using OpenGL es 3.0 and my GLSL version #version 300 es.
i want to get pixel(ARGB data) at every position in my vertex shader(vertex texture fetch). i have verified that my Android tablet supports vertex texture fetch.
Now i pass in the texture(image) and Texture coordinates to the vertex shader
and execute
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, 4);
Is this the right way or should i use GL_POINTS .
if i am using GL_POINTS how to pass the texture cooordinate?
could you provide any samples/example code that does a full pixel read(ARGB) in the vertex shader.
attaching my vertex shader
uniform sampler2D sTexture;
in vec4 aTextureCoord;
out vec3 colorFactor;
vec2 vTextureCoord;
vec4 tex;
void main()
{
vTextureCoord = aTextureCoord.xy;
tex = texture(sTexture,vTextureCoord);
float luminance = 0.299 * tex.r + 0.587 * tex.g + 0.114 * tex.b;
colorFactor = vec3(1.0, 1.0, 1.0);
gl_Position = vec4(-1.0 + (luminance * 0.00784313725), 0.0, 0.0, 1.0);
gl_PointSize = 1.0;
};
My texture coordinates passed are
{0.f,1.f}
{1.f,1.f} {0.f,0.f}
{1.f,0.f}
and the shader is triggered by
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, 4);

Just declare a sampler and sample it exactly as per usual. E.g.
"#version 150\n"
in vec4 position;
in vec2 texCoordinate;
uniform sampler2D texID;
uniform mat4 modelViewProjection;
void main()
{
gl_Position = modelViewProjection * (position + texture(texID, texCoordinate));
}
That will sample a 4d vector from the texture unit texID at location texCoordinate, using that to perturb position prior to applying the model-view-projection matrix. The type of geometry you're drawing makes no difference.

gl_Position = vec4(-1.0 + (luminance * 0.00784313725), 0.0, 0.0, 1.0);
That's clever. It's not going to work, but that's clever.
The thing that is confusing everyone is what you haven't told us. That you're computing the histogram by using blending. That you compute the location for each fragment based on the luminance, so you get lots of overlap. And blending just adds everything together, thus producing your histogram.
FYI: It's always best to explain what it is you're actually trying to accomplish in your question, rather than hoping that someone can deduce what you're attempting to do.
That's not going to work because you only have four vertices in this case. You have lots of fragments, but they will be generated based on interpolation from your 4 vertices. And you can't change the position of a fragment from within a fragment shader.
If you want to do what you're trying to do, you still have to render one vertex for every texel you fetch. You still need to use GL_POINTS.

Alpha gradient in OpenGL ES shader is not smooth

I'm attempting to create an alpha radial gradient effect (kind of lighting) using a simple shader.
The effect is created correctly, however the gradient is not smooth.
The precision is set to highp, so I don't really know where to look.
This shader is currently running on Android, using OpenGL ES 2.0.
This is how the gradient currently looks like:
And this is my current shader:
Vertex:
precision highp float;
attribute vec4 vPosition;
attribute vec2 vStaticInterpolation;
varying vec2 interpolator;
void main() {
interpolator = vStaticInterpolation;
gl_Position = vPosition;
}
Fragment:
precision highp float;
uniform float alphaFactor;
varying vec2 interpolator;
float MAX_ALPHA = 0.75;
void main() {
float x = distance(interpolator, vec2(0.0, 0.0));
float alpha = MAX_ALPHA - MAX_ALPHA * x;
alpha = max(alpha, 0.0);
gl_FragColor = vec4(0.925, 0.921, 0.843, alpha);
gl_FragColor.a *= alphaFactor;
}
The shader receives constant attributes for interpolation (from -1.0 to 1.0) in vStaticInterpolation.
The actual color is currently hard-coded in the shader.

It looks to be related to a dithering problem.
This could depend on the OpenGL driver implementation of your mobile device (though I don't know which model you are currently using). In the past it used to be an issue.
Possible tests you could perform are:
Disable Opengl dithering:
GLES20.glDisable(GLES20.GL_DITHER);
Impose an RGB888 surface when you create the surface. It is usually done in the ConfigChooser function. I try to remember by hard, this is part of the code of my application:
new AndroidGL.ConfigChooser(8, 8, 8, 8, depth, stencil) :

android.grapics.matrix to OpenGL 2.0 ES texture translation

Currently, the application displays an ImageView that successfully zooms and pans around on the screen. On top of that image, I would like to provide a texture that I would like to update itself to zoom or pan when the image below it zooms or pans.
As I understand, this should be possible with using the getImageMatrix() of my current setup on the ImageView and then applying that to my textured bitmap that is on top of the original image.
Edit & Resolution: (with strong aide from the selected answer below)
Currently, the panning in the texture occurs at a different speed than that of the ImageView, but when that has been resolved I will update this posting with additional edits and provide the solution to the entirety of the application. Then only the solution and a quick problem description paragraph will remain. Perhaps I'll even post a surface view and renderer source code for using a zoomable surface view.
In order to accomplish a mapping of an ImageView to an OpenGL texture, there were a couple of things that needed to be done in order to accomplish this correctly. And hopefully, this will aide other people who might want to use a zoomable SurfaceView in the future.
The shader code is written below. The gl_FragColor takes a translation matrix of a 3x3 from the getImageMatrix of any ImageView in order to update the screen from within the onDraw() method.
private final String vertexShader_ =
"attribute vec4 a_position;\n" +
"attribute vec4 a_texCoord;\n" +
"varying vec2 v_texCoord;\n" +
"void main() {\n" +
" gl_Position = a_position;\n" +
" v_texCoord = a_texCoord.xy;\n" +
"}\n";
private final String fragmentShader_ =
"precision mediump float;\n" +
"varying vec2 v_texCoord;\n" +
"uniform sampler2D texture;\n" +
"uniform mat3 transform;\n" +
"void main() {\n" +
" vec2 uv = (transform * vec3(v_texCoord, 1.0)).xy;\n" +
" gl_FragColor = texture2D( texture, uv );\n" +
"}\n";
At the start, the translation matrix for the OpenGL is simply an identify matrix and our ImageView will update these shared values with the shader through a listener.
private float[] translationMatrix = {1.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 1.0f};
...
uniforms_[TRANSLATION_UNIFORM] = glGetUniformLocation(program_, "transform");
checkGlError("glGetUniformLocation transform");
if (uniforms_[TRANSLATION_UNIFORM] == -1) {
throw new RuntimeException("Could not get uniform location for transform");
}
....
glUniformMatrix3fv(uniforms_[TRANSLATION_UNIFORM], 1, false, FloatBuffer.wrap(translationMatrix));
....
However, the OpenGL code needs an inverse of the OpenGL matrix calculations that come from Android, as well as a transpose operation performed on them before they are handed off to the renderer for being displayed on the screen. This has to do with the way that that information is stored.
protected void onMatrixChanged()
{
//...
float[] matrixValues = new float[9];
Matrix imageViewMatrix = getImageViewMatrix();
Matrix invertedMatrix = new Matrix();
imageViewMatrix.invert(invertedMatrix);
invertedMatrix.getValues(matrixValues);
transpose(matrixValues);
matrixChangedListener.onTranslation(matrixValues);
//...
}
In addition to the fact that the values are in the wrong locations for input into the OpenGL renderer, we also have the problem that we are dealing with our translations on the scale of imageHeight and imageWidth instead of the normalized [0, 1] range that OpenGL expects. So, in order to correct with this we have to target the last column with divisible numbers of our width and height.
matrixValues[6] = matrixValues[6] / getImageWidth();
matrixValues[7] = matrixValues[7] / getImageHeight();
Then, after you have accomplished this, you can start mapping with the ImageView matrix functions or the updated ImageTouchView matrix functions (which have some additional nice features when handling images on Android).
The effect can be seen below with the ImageView behind it taking up the whole screen and the texture in front of it being updated based upon updates from the ImageView. This handles zooming and panning translations between the two.
The only other step in the process would be to change the size of the SurfaceView to match that of the ImageView in order to get the full screen experience when the user is zooming in. In order to get only a surfaceview shown, you can simply put an 'invisible' imageview behind it in the background. And just let the ImageTouchView updates change the way that your OpenGL SurfaceView is shown.
Vote up on the answer below, as they helped me offline through chat significantly to get to this point in understanding what needs to be done for linking an imageview with a texture. They deserve it.
If you do end up going with the vertex shader modification below, instead of the fragment shader resolution, then it would seem that the values no longer need to be inverted to work with the pinch zoom anymore (and in fact work backwards if you leave it as is). Additionally, panning seems to work backwards than is expected as well.

You could forge fragment shader for that, just set this matrix as uniform parameter of the fragment shaderand modify texture coordinates using this matrix.
precision mediump float;
uniform sampler2D samp;
uniform mat3 transform;
varying vec2 texCoord;
void main()
{
vec2 uv = (transform * vec3(texCoord, 1.0)).xy;
gl_FragColor = texture2D(samp, uv);
}
Note that matrix you get from image view is row-major order you should transform it to get OpenGL column-major order, also you would have divide x and y translation components by width and height respectively.
Update: It just occured to me that modifying vertex shader would be better, you can just scale your quad with the same matrix you used in fragment shader, and it would be clamped to your screen if to big, and it will not have a problem with edge clamping when small, and matrix multiplication will happen only 4 times instead of one for each pixel. Just do in vertex shader:
gl_Position = vec4((transform * vec3(a_position.xy, 1.0)).xy, 0.0, 1.0);
and use untransformed texCoord in fragment shader.

GLSL for OpenGL ES Android

I want to use shading on my OpenGL objects but can't seem to access GLSL functions in my opengl package. Is there a GLSL package available for OpenGL ES in Eclipse?
EDIT: As Tim pointed out. Shaders are written as text files and then loaded using glShaderSoure, I have a C++ shader file which I had written once for a ray tracing application. But, I am really confused as to how I would go about implementing in java. Suppose I have a 2D square drawn in my Renderer class using a gl object MySquare , how will I go about implementing the java equivalent of the shader file below.
Shader.vert
varying vec3 N;
varying vec3 v;
void main()
{
// Need to transform the normal into eye space.
N = normalize(gl_NormalMatrix * gl_Normal);
// Always have to transform vertex positions so they end
// up in the right place on the screen.
gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}
// Fragment shader for per-pixel Phong interpolation and shading.
Shader.Frag
// The "varying" keyword means that the parameter's value is interpolated
// between the nearby vertices.
varying vec3 N;
varying vec3 v;
//Used for Environmental mapping shader calculations
const vec3 xUnitVec=vec3(1.0, 0.0, 0.0), yUnitVec=vec3(1.0, 1.0, 0.0);
uniform vec3 BaseColor, MixRatio;
uniform sampler2D EnvMap;
void main()
{
// The scene's ambient light.
vec4 ambient = gl_LightModel.ambient * gl_FrontMaterial.ambient;
// The normal vectors is generally not normalized after being
// interpolated across a triangle. Here we normalize it.
vec3 Normal = normalize(N);
// Since the vertex is in eye space, the direction to the
// viewer is simply the normalized vector from v to the
// origin.
vec3 Viewer = -normalize(v);
// Get the lighting direction and normalize it.
vec3 Light = normalize(gl_LightSource[0].position.xyz);
// Compute halfway vector
vec3 Half = normalize(Viewer+Light);
// Compute factor to prevent light leakage from below the
// surface
float B = 1.0;
if(dot(Normal, Light)<0.0) B = 0.0;
// Compute geometric terms of diffuse and specular
float diffuseShade = max(dot(Normal, Light), 0.0);
float specularShade =
B * pow(max(dot(Half, Normal), 0.0), gl_FrontMaterial.shininess);
// Compute product of geometric terms with material and
// lighting values
vec4 diffuse = diffuseShade * gl_FrontLightProduct[0].diffuse;
vec4 specular = specularShade * gl_FrontLightProduct[0].specular;
ambient += gl_FrontLightProduct[0].ambient;
// Assign final color
gl_FragColor= ambient + diffuse + specular + gl_FrontMaterial.emission;
}

Check out the tutorials over at learnopengles.com. They'll answer all the questions you have.

There's no 'java equivalent' of a shader file. The shader is written in GLSL. The shader will be the same whether your opengl is wrapped in java, or c++, or python, or whatever. Aside from small API differences between OpenGL and OpenGLES, you can upload the exact same shader in Java as you used in C++, character for character.

You can use this source:
https://github.com/markusfisch/ShaderEditor
ShaderView class is the key!
GLSL files are in raw folder.