Develop Reference

How to make smoother borders using Fragment shader in OpenGL?

I have been trying to draw border of an Image with transparent background using OpenGL in Android. I am using Fragment Shader & Vertex Shader. (From the GPUImage Library)
Below I have added Fig. A & Fig B.
Fig A.
Fig B.
I have achieved Fig A. With the customised Fragment Shader. But Unable to make the border smoother as in Fig B. I am attaching the Shader code that I have used (to achieve rough border). Can someone here help me on how to make the border smoother?
Here is my Vertex Shader :
attribute vec4 position;
attribute vec4 inputTextureCoordinate;
varying vec2 textureCoordinate;
void main()
{
gl_Position = position;
textureCoordinate = inputTextureCoordinate.xy;
}
Here is my Fragment Shader :
I have calculated 8 pixels around the current pixel. If any one pixel of those 8 is opaque(having alpha greater than 0.4), It it drawn as a border color.
precision mediump float;
uniform sampler2D inputImageTexture;
varying vec2 textureCoordinate;
uniform lowp float thickness;
uniform lowp vec4 color;
void main() {
float x = textureCoordinate.x;
float y = textureCoordinate.y;
vec4 current = texture2D(inputImageTexture, vec2(x,y));
if ( current.a != 1.0 ) {
float offset = thickness * 0.5;
vec4 top = texture2D(inputImageTexture, vec2(x, y - offset));
vec4 topRight = texture2D(inputImageTexture, vec2(x + offset,y - offset));
vec4 topLeft = texture2D(inputImageTexture, vec2(x - offset, y - offset));
vec4 right = texture2D(inputImageTexture, vec2(x + offset, y ));
vec4 bottom = texture2D(inputImageTexture, vec2(x , y + offset));
vec4 bottomLeft = texture2D(inputImageTexture, vec2(x - offset, y + offset));
vec4 bottomRight = texture2D(inputImageTexture, vec2(x + offset, y + offset));
vec4 left = texture2D(inputImageTexture, vec2(x - offset, y ));
if ( top.a > 0.4 || bottom.a > 0.4 || left.a > 0.4 || right.a > 0.4 || topLeft.a > 0.4 || topRight.a > 0.4 || bottomLeft.a > 0.4 || bottomRight.a > 0.4 ) {
if (current.a != 0.0) {
current = mix(color , current , current.a);
} else {
current = color;
}
}
}
gl_FragColor = current;
}

You were almost on the right track.
The main algorithm is:
Blur the image.
Use pixels with opacity above a certain threshold as outline.
The main problem is the blur step. It needs to be a large and smooth blur to get the smooth outline you want. For blurring, we can use convolution filter Kernel. And to achieve a large blur, we should use a large kernel. And I suggest using the Gaussian Blur distribution, as it is very well known and used.
The Overview of he Algorithm is:
For each fragment, we sample many locations around it. The samples are made in an N by N grid. We average them together using weights that follow a 2D Gaussian Distribution.
This results in a blurred image.
With the blurred image, we paint the fragments that have alpha greater than a threshold with our outline color. And, of course, any opaque pixels in the original image should also appear in the result.
On a sidenote, your solution is almost a blur with a 3 x 3 kernel (you sample locations around the fragment in a 3 by 3 grid). However, a 3 x 3 kernel won't give you the amount of blur you need. You need more samples (e.g. 11 x 11). Also, the weights closer to the center should have a greater impact on the result. Thus, uniform weights won't work very well.
Oh, and one more important thing:
One single shader to acomplish this is NOT the fastest way to implement this. Usually, this would be acomplished with 2 separate renders. The first one would render the image as usual and the second render would blur and add the outline. I assumed that you want to do this with 1 single render.
The following is a vertex and fragment shader that accomplish this:
Vertex Shader
varying vec2 vecUV;
varying vec3 vecPos;
varying vec3 vecNormal;
void main() {
vecUV = uv * 3.0 - 1.0;
vecPos = (modelViewMatrix * vec4(position, 1.0)).xyz;
vecNormal = (modelViewMatrix * vec4(normal, 0.0)).xyz;
gl_Position = projectionMatrix * vec4(vecPos, 1.0);
}
Fragment Shader
precision highp float;
varying vec2 vecUV;
varying vec3 vecPos;
varying vec3 vecNormal;
uniform sampler2D inputImageTexture;
float normalProbabilityDensityFunction(in float x, in float sigma)
{
return 0.39894*exp(-0.5*x*x/(sigma*sigma))/sigma;
}
vec4 gaussianBlur()
{
// The gaussian operator size
// The higher this number, the better quality the outline will be
// But this number is expensive! O(n2)
const int matrixSize = 11;
// How far apart (in UV coordinates) are each cell in the Gaussian Blur
// Increase this for larger outlines!
vec2 offset = vec2(0.005, 0.005);
const int kernelSize = (matrixSize-1)/2;
float kernel[matrixSize];
// Create the 1-D kernel using a sigma
float sigma = 7.0;
for (int j = 0; j <= kernelSize; ++j)
{
kernel[kernelSize+j] = kernel[kernelSize-j] = normalProbabilityDensityFunction(float(j), sigma);
}
// Generate the normalization factor
float normalizationFactor = 0.0;
for (int j = 0; j < matrixSize; ++j)
{
normalizationFactor += kernel[j];
}
normalizationFactor = normalizationFactor * normalizationFactor;
// Apply the kernel to the fragment
vec4 outputColor = vec4(0.0);
for (int i=-kernelSize; i <= kernelSize; ++i)
{
for (int j=-kernelSize; j <= kernelSize; ++j)
{
float kernelValue = kernel[kernelSize+j]*kernel[kernelSize+i];
vec2 sampleLocation = vecUV.xy + vec2(float(i)*offset.x,float(j)*offset.y);
vec4 sample = texture2D(inputImageTexture, sampleLocation);
outputColor += kernelValue * sample;
}
}
// Divide by the normalization factor, so the weights sum to 1
outputColor = outputColor/(normalizationFactor*normalizationFactor);
return outputColor;
}
void main()
{
// After blurring, what alpha threshold should we define as outline?
float alphaTreshold = 0.3;
// How smooth the edges of the outline it should have?
float outlineSmoothness = 0.1;
// The outline color
vec4 outlineColor = vec4(1.0, 1.0, 1.0, 1.0);
// Sample the original image and generate a blurred version using a gaussian blur
vec4 originalImage = texture2D(inputImageTexture, vecUV);
vec4 blurredImage = gaussianBlur();
float alpha = smoothstep(alphaTreshold - outlineSmoothness, alphaTreshold + outlineSmoothness, blurredImage.a);
vec4 outlineFragmentColor = mix(vec4(0.0), outlineColor, alpha);
gl_FragColor = mix(outlineFragmentColor, originalImage, originalImage.a);
}
This is the result I got:
And for the same image as yours, with matrixSize = 33, alphaTreshold = 0.05
And to try to get crispier results we can tweak the parameters. Here is an example with matrixSize = 111, alphaTreshold = 0.05, offset = vec2(0.002, 0.002), alphaTreshold = 0.01, outlineSmoothness = 0.00. Note that increasing matrixSize will heavily impact performance, which is a limitation of rendering this outline with only one shader pass.
I tested the shader on this site. Hopefully you will be able to adapt it to your solution.
Regarding references, I have used quite a lot of this shadertoy example as basis for the code I wrote for this answer.

In my filters, the smoothness is achieved by a simple boxblur on the border.. You have decided that alpha > 0.4 is a border. The value of alpha between 0-0.4 in surrounding pixels gives an edge. Just blur this edge with a 3x3 window to get the smooth edge.
if ( current.a != 1.0 ) {
// other processing
if (current.a > 0.4) {
if ( (top.a < 0.4 || bottom.a < 0.4 || left.a < 0.4 || right.a < 0.4
|| topLeft.a < 0.4 || topRight.a < 0.4 || bottomLeft.a < 0.4 || bottomRight.a < 0.4 )
{
// Implement 3x3 box blur here
}
}
}
You need to tweak which edge pixels you blur. The basic issue is that it drops from an opaque to a transparent pixel - what you need is a gradual transition.
Another options is quick anti-aliasing. From my comment below - Scale up 200% and then scale down 50% to original method. Use nearest neighbour scaling. This technique is used for smooth edges on text sometimes.

How to use OpenGL to emulate OpenCV's warpPerspective functionality (perspective transform)

I've done image warping using OpenCV in Python and C++, see the Coca Cola logo warped in place in the corners I had selected:
Using the following images:
and this:
Full album with transition pics and description here
I need to do exactly this, but in OpenGL. I'll have:
Corners inside which I've to map the warped image
A homography matrix that maps the transformation of the logo image
into the logo image you see inside the final image (using OpenCV's
warpPerspective), something like this:
[[ 2.59952324e+00, 3.33170976e-01, -2.17014066e+02],
[ 8.64133587e-01, 1.82580111e+00, -3.20053715e+02],
[ 2.78910149e-03, 4.47911310e-05, 1.00000000e+00]]
Main image (the running track image here)
Overlay image (the Coca Cola image here)
Is it possible ? I've read a lot and started OpenGL basics tutorials, but can it be done from just what I have? Would the OpenGL implementation be faster, say, around ~10ms?
I'm currently playing with this tutorial here:
http://ogldev.atspace.co.uk/www/tutorial12/tutorial12.html
Am I going in the right direction? Total OpenGL newbie here, please bear. Thanks.

After trying a number of solutions proposed here and elsewhere, I ended solving this by writing a fragment shader that replicates what 'warpPerspective' does.
The fragment shader code looks something like:
varying highp vec2 textureCoordinate;
uniform sampler2D inputImageTexture;
// NOTE: you will need to pass the INVERSE of the homography matrix, as well as
// the width and height of your image as uniforms!
uniform highp mat3 inverseHomographyMatrix;
uniform highp float width;
uniform highp float height;
void main()
{
// Texture coordinates will run [0,1],[0,1];
// Convert to "real world" coordinates
highp vec3 frameCoordinate = vec3(textureCoordinate.x * width, textureCoordinate.y * height, 1.0);
// Determine what 'z' is
highp vec3 m = inverseHomographyMatrix[2] * frameCoordinate;
highp float zed = 1.0 / (m.x + m.y + m.z);
frameCoordinate = frameCoordinate * zed;
// Determine translated x and y coordinates
highp float xTrans = inverseHomographyMatrix[0][0] * frameCoordinate.x + inverseHomographyMatrix[0][1] * frameCoordinate.y + inverseHomographyMatrix[0][2] * frameCoordinate.z;
highp float yTrans = inverseHomographyMatrix[1][0] * frameCoordinate.x + inverseHomographyMatrix[1][1] * frameCoordinate.y + inverseHomographyMatrix[1][2] * frameCoordinate.z;
// Normalize back to [0,1],[0,1] space
highp vec2 coords = vec2(xTrans / width, yTrans / height);
// Sample the texture if we're mapping within the image, otherwise set color to black
if (coords.x >= 0.0 && coords.x <= 1.0 && coords.y >= 0.0 && coords.y <= 1.0) {
gl_FragColor = texture2D(inputImageTexture, coords);
} else {
gl_FragColor = vec4(0.0,0.0,0.0,0.0);
}
}
Note that the homography matrix we are passing in here is the INVERSE HOMOGRAPHY MATRIX! You have to invert the homography matrix that you would pass into 'warpPerspective'- otherwise this code will not work.
The vertex shader does nothing but pass through the coordinates:
// Vertex shader
attribute vec4 position;
attribute vec4 inputTextureCoordinate;
varying vec2 textureCoordinate;
void main() {
// Nothing happens in the vertex shader
textureCoordinate = inputTextureCoordinate.xy;
gl_Position = position;
}
Pass in unaltered texture coordinates and position coordinates (i.e. textureCoordinates = [(0,0),(0,1),(1,0),(1,1)] and positionCoordinates = [(-1,-1),(-1,1),(1,-1),(1,1)], for a triangle strip), and this should work!

You can do perspective warping of the texture using texture2DProj(), or alternatively using texture2D() by dividing the st coordinates of the texture (which is what texture2DProj does).
Have a look here: Perspective correct texturing of trapezoid in OpenGL ES 2.0.
warpPerspective projects the (x,y,1) coordinate with the matrix and then divides (u,v) by w, like texture2DProj(). You'll have to modify the matrix so the resulting coordinates are properly normalised.
In terms of performance, if you want to read the data back to the CPU your bottleneck is glReadPixels. How long it will take depends on your device. If you're just displaying, the OpenGL ES calls will take much less than 10ms, assuming that you have both textures loaded to GPU memory.

[edit] This worked on my Galaxy S9 but on my car's Android it had an issue that the whole output texture was white. I've sticked to the original shader and it works :)
You can use mat3*vec3 ops in the fragment shader:
varying highp vec2 textureCoordinate;
uniform sampler2D inputImageTexture;
uniform highp mat3 inverseHomographyMatrix;
uniform highp float width;
uniform highp float height;
void main()
{
highp vec3 frameCoordinate = vec3(textureCoordinate.x * width, textureCoordinate.y * height, 1.0);
highp vec3 trans = inverseHomographyMatrix * frameCoordinate;
highp vec2 coords = vec2(trans.x / width, trans.y / height) / trans.z;
if (coords.x >= 0.0 && coords.x <= 1.0 && coords.y >= 0.0 && coords.y <= 1.0) {
gl_FragColor = texture2D(inputImageTexture, coords);
} else {
gl_FragColor = vec4(0.0,0.0,0.0,0.0);
}
};
If you want to have transparent background don't forget to add
GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
GLES20.glBlendEquation(GLES20.GL_FUNC_ADD);
And set transpose flag (in case you use the above shader):
GLES20.glUniformMatrix3fv(H_P2D, 1, true, homography, 0);

Opengl ES 2.0 ortho projection blank screen

i'm quite new to opengl , and I'm trying to display a square with opengl es 2.0 with an orthographics projection ( on android using c++ and ndk ) , but all I get is a blank screen.
If i don't use the projection matrix I can get the square but it's stretched depending on surface's aspect ratio.
I'm building the matrix with the following code ( referring to https://www.opengl.org/sdk/docs/man2/xhtml/glOrtho.xml ):
GLfloat projMat[16];
void ortho(float left, float right, float top, float bottom, float near, float far){
float tx = ((right+left)/(right-left))*-1;
float ty = ((top+bottom)/(top-bottom))*-1;
float tz = ((far+near)/(far-near))*-1;
projMat[0] = 2/right-left;
projMat[1] = 0.0f;
projMat[2] = 0.0f;
projMat[3] = 0.0f;
projMat[4] = 0.0f;
projMat[5] = 2/top-bottom;
projMat[6] = 0.0f;
projMat[7] = 0.0f;
projMat[8] = 0.0f;
projMat[9] = 0.0f;
projMat[10] = -2/far-near;
projMat[11] = 0.0f;
projMat[12] = tx;
projMat[13] = ty;
projMat[14] = tz;
projMat[15] = 1.0f; }
And i call this function with : ortho(0.0,width,0.0, height,-1.0,1.0); , where width and height are surface's width and height.
My vertex shader :
attribute vec4 vPosition;
uniform mat4 projMatrix;
void main() {
gl_Position = projMatrix * vPosition;
};
My fragment shader :
precision mediump float;
void main() {
gl_FragColor = vec4(0.0, 1.0, 0.0, 1.0);
}
And my draw function :
// 3D drawing
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glUseProgram(mProgram);
glUniformMatrix4fv(mPerspectivehandler, 1, GL_FALSE, projMat);
glEnableVertexAttribArray(mvPositionHandle);
glVertexAttribPointer(mvPositionHandle, 3, GL_FLOAT, GL_FALSE, 0, quadverts);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glDisableVertexAttribArray(mvPositionHandle);
Do you know where's the mistake?
Edit :
I added parentheses as suggested, but it still doesn't work
GLfloat projMat[16];
void ortho(float left, float right, float top, float bottom, float near, float far){
float tx = ((right+left)/(right-left))*-1;
float ty = ((top+bottom)/(top-bottom))*-1;
float tz = ((far+near)/(far-near))*-1;
projMat[0] = 2/(right-left);
projMat[1] = 0.0f;
projMat[2] = 0.0f;
projMat[3] = 0.0f;
projMat[4] = 0.0f;
projMat[5] = 2/(top-bottom);
projMat[6] = 0.0f;
projMat[7] = 0.0f;
projMat[8] = 0.0f;
projMat[9] = 0.0f;
projMat[10] = -2/(far-near);
projMat[11] = 0.0f;
projMat[12] = tx;
projMat[13] = ty;
projMat[14] = tz;
projMat[15] = 1.0f; }

You're missing parentheses in the matrix calculation for the diagonal elements. They should be:
projMat[0] = 2.0f / (right - left);
projMat[5] = 2.0f / (top - bottom);
projMat[10] = -2.0f / (far - near);

It can be hard to debug issues where nothing is rendered. You should simplify until you can isolate what is going wrong.
You've declared vPosition as a vec4, but in your comment, your vertex position array only has 12 floats (3 per vertex). a vec4 needs 4 floats per vertex. this is also a problem here:
glVertexAttribPointer(mvPositionHandle, 3, GL_FLOAT, GL_FALSE, 0, quadverts);
You use 3 as the second parameter in the line above, but vec4 in your shader means use 4 here. Be sure that quadVerts holds 4 floats per vertex!
To debug, don't use a projection matrix - just render your geometry directly to clip-coordinates in your shader like so:
attribute vec4 vPosition;
uniform mat4 projMatrix;
void main() {
gl_Position = vPosition;
};
If the above shader renders (after you fix your vec4 problem), then you can start to debug your projection matrix.
Does your glClear() call actually take effect? try changing the color with glClearColor() and make sure the background color changes accordingly. If it doesn't, you might have set up your GL window incorrectly.

This question is 4 months old and I don't know if you still need an aswer.
I also had a lot of problems with orto projection matrices in OpenGL, but this works (for me):
result[0] = 2.0 / (right - left);
result[1] = 0.0;
result[2] = 0.0;
result[3] = 0.0;
result[4] = 0.0;
result[5] = 2.0 / (top - bottom);
result[6] = 0.0;
result[7] = 0.0;
result[8] = 0.0;
result[9] = 0.0;
result[10] = (1.0f / (near - far));
result[11] = 0.0;
result[12] = ((left + right) / (left - right));
result[13] = ((top + bottom) / (bottom - top));
result[14] = (near / (near - far));
result[15] = 1;

Apply linear impuls to pushed point - Box2D on Android

I would like to get a shape to move from middle bottom to the point I'm touching. Issue is to solve the angle (or degree?) to the touched point.
float angle = ?
float power = calculatePower(touchY);
Vec2 impulse = new Vec2(angle, power);
Vec2 point = body.getWorldCenter(); // to prevent rotation of shape
body.applyLinearImpulse(impulse, point);
Anyone got a suggestion?
EDIT: SOLVED
Thanks to Andrews answer. Here is the working code:
Point delta = new Point(touchX - bodyX, touchY - bodyY);
double angle = Math.atan2(delta.y, delta.x);
Vec2 direction = new Vec2((float)Math.cos(angle), (float)-Math.sin(angle));
float power = calculatePower(touchY);
Vec2 impulse = new Vec2(power * direction.x, power * direction.y);
Vec2 point = body.getWorldCenter();
body.applyLinearImpulse(impulse, point);

Pseudo code looks like this. atan2 is standard math function
Vec2 delta = touchPoint - body.position;
float angle = math.atan2(delta.y, delta.x);
This line Vec2 impulse = new Vec2(angle, power); looks very strange though. You are using angle as x value and power as y. That does not make much sense. I think your goal is to apply impulse of power power in direction of angle. In that case the code should look like this:
Vec2 dir = Vec2(math.cos(angle), math.sin(angle));
Vec2 impulse = power * dir;
But since delta and dir only differ in their magnitude you can simplify calculating the impulse as:
Vec2 delta = touchPoint - body.position;
Vec2 dir = delta.normalize();
Vec2 impulse = power * dir

GLSL ES fragment shader produces very different results on different devices

I am developing a game for Android using OpenGL ES 2.0 and have a problem with a fragment shader for drawing stars in the background. I've got the following code:
precision mediump float;
varying vec2 transformed_position;
float rand(vec2 co) {
return fract(sin(dot(co.xy ,vec2(12.9898,78.233))) * 43758.5453);
}
void main(void) {
float distance = 10.0;
float quantization_strength = 4.0;
vec3 background_color = vec3(0.09, 0.0, 0.288);
vec2 zero = vec2(0.0, 0.0);
vec2 distance_vec = vec2(distance, distance);
vec2 quantization_vec = vec2(quantization_strength, quantization_strength);
vec2 new_vec = floor(transformed_position / quantization_vec) * quantization_vec;
if(all(equal(mod(new_vec, distance_vec), zero))) {
float rand_val = rand(new_vec);
vec3 current_color = background_color * (1.0 + rand_val);
gl_FragColor = vec4(current_color.x, current_color.y, current_color.z, 1.0);
} else {
gl_FragColor = vec4(background_color.x, background_color.y, background_color.z, 1.0 );
}
}
My aim is to 'quantize' the fragment coordinates, so 'stars' are not 1px in size, and then light up quantized pixels that are distant enough by a random amount. This code, however, produces different results depending on where it is executed. I have used GLSL Sandbox (http://glsl.heroku.com), Nexus 7 and HTC Desire S to create comparison:
As you can see, GLSL Sandbox produces dense grid with many stars visible. On Nexus 7 stars are much fewer and distributed along lines (which may be not obvious on this small image) - the rand function does not work as expected. Desire S draws no stars at all.
Why does the rand function work so strangely on Nexus 7 (if I modify the vector used for dot product, stars are distributed along lines at different angle)? And what might cause Desire S not to render the stars?
I would also appreciate any optimization tips for this shader, as I am very inexperienced with GLSL. Or perhaps there is better way to draw 'stars' via fragment shader?
UPDATE
I changed the code to this (I used http://glsl.heroku.com/e#9364.0 as reference):
precision mediump float;
varying highp vec2 transformed_position;
highp float rand(vec2 co) {
highp float a = 1e3;
highp float b = 1e-3;
highp float c = 1e5;
return fract(sin((co.x+co.y*a)*b)*c);
}
void main(void) {
float size = 15.0;
float prob = 0.97;
lowp vec3 background_color = vec3(0.09, 0.0, 0.288);
highp vec2 world_pos = transformed_position;
vec2 pos = floor(1.0 / size * world_pos);
float color = 0.0;
highp float starValue = rand(pos);
if(starValue > prob) {
vec2 center = size * pos + vec2(size, size) * 0.5;
float xy_dist = abs(world_pos.x - center.x) * abs(world_pos.y - center.y) / 5.0;
color = 0.6 - distance(world_pos, center) / (0.5 * size) * xy_dist;
}
if(starValue < prob || color < 0.0) {
gl_FragColor = vec4(background_color, 1.0);
} else {
float starIntensity = fract(100.0 * starValue);
gl_FragColor = vec4(background_color * (1.0 + color * 3.0 * starIntensity), 1.0);
}
}
Desire S now gives me very nice, uniformly distributed stars. But the problem with Nexus 7 is still there. With prob = 0.97, no stars are displayed, and with very low prob = 0.01, they appear very sparsely placed along horizontal lines. Why does Tegra 3 behave so strangely?