Currently, the application displays an ImageView that successfully zooms and pans around on the screen. On top of that image, I would like to provide a texture that I would like to update itself to zoom or pan when the image below it zooms or pans.
As I understand, this should be possible with using the getImageMatrix() of my current setup on the ImageView and then applying that to my textured bitmap that is on top of the original image.
Edit & Resolution: (with strong aide from the selected answer below)
Currently, the panning in the texture occurs at a different speed than that of the ImageView, but when that has been resolved I will update this posting with additional edits and provide the solution to the entirety of the application. Then only the solution and a quick problem description paragraph will remain. Perhaps I'll even post a surface view and renderer source code for using a zoomable surface view.
In order to accomplish a mapping of an ImageView to an OpenGL texture, there were a couple of things that needed to be done in order to accomplish this correctly. And hopefully, this will aide other people who might want to use a zoomable SurfaceView in the future.
The shader code is written below. The gl_FragColor takes a translation matrix of a 3x3 from the getImageMatrix of any ImageView in order to update the screen from within the onDraw() method.
private final String vertexShader_ =
"attribute vec4 a_position;\n" +
"attribute vec4 a_texCoord;\n" +
"varying vec2 v_texCoord;\n" +
"void main() {\n" +
" gl_Position = a_position;\n" +
" v_texCoord = a_texCoord.xy;\n" +
"}\n";
private final String fragmentShader_ =
"precision mediump float;\n" +
"varying vec2 v_texCoord;\n" +
"uniform sampler2D texture;\n" +
"uniform mat3 transform;\n" +
"void main() {\n" +
" vec2 uv = (transform * vec3(v_texCoord, 1.0)).xy;\n" +
" gl_FragColor = texture2D( texture, uv );\n" +
"}\n";
At the start, the translation matrix for the OpenGL is simply an identify matrix and our ImageView will update these shared values with the shader through a listener.
private float[] translationMatrix = {1.0f, 0.0f, 0.0f,
0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 1.0f};
...
uniforms_[TRANSLATION_UNIFORM] = glGetUniformLocation(program_, "transform");
checkGlError("glGetUniformLocation transform");
if (uniforms_[TRANSLATION_UNIFORM] == -1) {
throw new RuntimeException("Could not get uniform location for transform");
}
....
glUniformMatrix3fv(uniforms_[TRANSLATION_UNIFORM], 1, false, FloatBuffer.wrap(translationMatrix));
....
However, the OpenGL code needs an inverse of the OpenGL matrix calculations that come from Android, as well as a transpose operation performed on them before they are handed off to the renderer for being displayed on the screen. This has to do with the way that that information is stored.
protected void onMatrixChanged()
{
//...
float[] matrixValues = new float[9];
Matrix imageViewMatrix = getImageViewMatrix();
Matrix invertedMatrix = new Matrix();
imageViewMatrix.invert(invertedMatrix);
invertedMatrix.getValues(matrixValues);
transpose(matrixValues);
matrixChangedListener.onTranslation(matrixValues);
//...
}
In addition to the fact that the values are in the wrong locations for input into the OpenGL renderer, we also have the problem that we are dealing with our translations on the scale of imageHeight and imageWidth instead of the normalized [0, 1] range that OpenGL expects. So, in order to correct with this we have to target the last column with divisible numbers of our width and height.
matrixValues[6] = matrixValues[6] / getImageWidth();
matrixValues[7] = matrixValues[7] / getImageHeight();
Then, after you have accomplished this, you can start mapping with the ImageView matrix functions or the updated ImageTouchView matrix functions (which have some additional nice features when handling images on Android).
The effect can be seen below with the ImageView behind it taking up the whole screen and the texture in front of it being updated based upon updates from the ImageView. This handles zooming and panning translations between the two.
The only other step in the process would be to change the size of the SurfaceView to match that of the ImageView in order to get the full screen experience when the user is zooming in. In order to get only a surfaceview shown, you can simply put an 'invisible' imageview behind it in the background. And just let the ImageTouchView updates change the way that your OpenGL SurfaceView is shown.
Vote up on the answer below, as they helped me offline through chat significantly to get to this point in understanding what needs to be done for linking an imageview with a texture. They deserve it.
If you do end up going with the vertex shader modification below, instead of the fragment shader resolution, then it would seem that the values no longer need to be inverted to work with the pinch zoom anymore (and in fact work backwards if you leave it as is). Additionally, panning seems to work backwards than is expected as well.
You could forge fragment shader for that, just set this matrix as uniform parameter of the fragment shaderand modify texture coordinates using this matrix.
precision mediump float;
uniform sampler2D samp;
uniform mat3 transform;
varying vec2 texCoord;
void main()
{
vec2 uv = (transform * vec3(texCoord, 1.0)).xy;
gl_FragColor = texture2D(samp, uv);
}
Note that matrix you get from image view is row-major order you should transform it to get OpenGL column-major order, also you would have divide x and y translation components by width and height respectively.
Update: It just occured to me that modifying vertex shader would be better, you can just scale your quad with the same matrix you used in fragment shader, and it would be clamped to your screen if to big, and it will not have a problem with edge clamping when small, and matrix multiplication will happen only 4 times instead of one for each pixel. Just do in vertex shader:
gl_Position = vec4((transform * vec3(a_position.xy, 1.0)).xy, 0.0, 1.0);
and use untransformed texCoord in fragment shader.
Related
I've recently started looking into OpenGL ES for Android and am working on a drawing app. I've implemented some basics such as point sprites, path smoothing and FBO for double buffering. At the moment I am playing around with the glBlendFunc, more specifically when I put two textures close to each other with the same color/alpha values, the alpha gets added so it appears darker at the intersection of the sprites. This is a problem because the stroke opacity is not preserved if a lot of points are close together, as the color tends to more opaque rather than staying with the same opacity. Is there a way to make the textures have the same color on the intersection, i.e. have the same alpha value for the intersecting pixels, but keep the alpha values for the rest of the pixels?
Here's how I've done the relevant parts of the app:
for drawing the list of point sprites I use blending like this:
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_ONE, GLES20.GL_ONE_MINUS_SRC_ALPHA);
the app uses an FBO with a texture, where it renders each brush stroke first and then this texture is rendered to the main screen. The blending func there is:
GLES20.glEnable(GLES20.GL_BLEND);
GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
OpenGL ES 2.0 doesn't support alpha masking;
there is no DEPTH_TEST function used anywhere in the app;
the textures for the point sprites are PNGs with transparent backgrounds;
the app supports texture masking which means one texture is used for the shape and one texture is used for the content;
my fragment shader looks like this:
precision mediump float;
uniform sampler2D uShapeTexture;
uniform sampler2D uFillTexture;
uniform float vFillScale;
varying vec4 vColor;
varying float vShapeRotation;
varying float vFillRotation;
varying vec4 vFillPosition;
vec2 calculateRotation(float rotationValue) {
float mid = 0.5;
return vec2(cos(rotationValue) * (gl_PointCoord.x - mid) + sin(rotationValue) * (gl_PointCoord.y - mid) + mid,
cos(rotationValue) * (gl_PointCoord.y - mid) - sin(rotationValue) * (gl_PointCoord.x - mid) + mid);
}
void main() {
// Calculations.
vec2 rotatedShape = calculateRotation(vShapeRotation);
vec2 rotatedFill = calculateRotation(vFillRotation);
vec2 scaleVector = vec2(vFillScale, vFillScale);
vec2 positionVector = vec2(vFillPosition[0], vFillPosition[1]);
// Obtain colors.
vec4 colorShape = texture2D(uShapeTexture, rotatedShape);
vec4 colorFill = texture2D(uFillTexture, (rotatedFill * scaleVector) + positionVector);
gl_FragColor = colorShape * colorFill * vColor;
}
my vertex shader is this:
attribute vec4 aPosition;
attribute vec4 aColor;
attribute vec4 aJitter;
attribute float aShapeRotation;
attribute float aFillRotation;
attribute vec4 aFillPosition;
attribute float aPointSize;
varying vec4 vColor;
varying float vShapeRotation;
varying float vFillRotation;
varying vec4 vFillPosition;
uniform mat4 uMVPMatrix;
void main() {
// Sey position and size.
gl_Position = uMVPMatrix * (aPosition + aJitter);
gl_PointSize = aPointSize;
// Pass values to fragment shader.
vColor = aColor;
vShapeRotation = aShapeRotation;
vFillRotation = aFillRotation;
vFillPosition = aFillPosition;
}
I've tried playing around with the glBlendFunc parameters but I can't find the right combination to draw what I want. I've attached some images showing what I would like to achieve and what I have at the moment. Any suggestions?
The Solution
Finally managed to get this working properly with a few lines thanks to # Rabbid76. First of all I had to configure my depth test function before I draw to the FBO:
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
GLES20.glDepthFunc(GLES20.GL_LESS);
// Drawing code for FBO.
GLES20.glDisable(GLES20.GL_DEPTH_TEST);
Then in my fragment shader I had to make sure that any pixels with alpha < 1 in the mask are discarded like this:
...
vec4 colorMask = texture2D(uMaskTexture, gl_PointCoord);
if (colorMask.a < 1.0)
discard;
else
gl_FragColor = calculatedColor;
And the result is (flickering is due to Android emulator and gif capture tool):
If you set the glBlendFunc
with the functions (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) and you use
glBlendEquation with the equation GL_FUNC_ADD then the destination color is
calculated as follows:
C_dest = C_src * A_src + C_dest * (1-A_src)
If you blend for example C_dest = 1 with C_src = 0.5 and A_src = 0.5 then:
C_dest = 0.75 = 1 * 0.5 + 0.5 * 0.5
If you repeat blending the same color C_src = 0.5 and A_src = 0.5 then the destination color becomes darker:
C_dest = 0.625 = 0.75 * 0.5 + 0.5 * 0.5
Since the new target color is always a function of the original target color and the source color, the color can not remain equel when blending 2 times, because the target color has already changed after the 1st time blending (except GL_ZERO).
You have to avoid that any fragment is blended twice. If all fragments are drawn to the same depth (2D) then you can use the depth test for this:
glEnable( GL_DEPTH_TEST );
glDepthFunc( GL_LESS );
// do the drawing with the color
glDisable( GL_DEPTH_TEST );
Or the stencil test can be used. For example, the stencil test can be set to pass only when the stencil buffer is equal to 0.
Every time a fragment is to be written the stencil buffer is incremented:
glClear( GL_STENCIL_BUFFER_BIT );
glEnable( GL_STENCIL_TEST );
glStencilOp( GL_KEEP, GL_KEEP, GL_INCR );
glStencilFunc( GL_EQUAL, 0, 255 );
// do the drawing with the color
glDisable( GL_STENCIL_TEST );
Extension to the answer
Note that you can discard fragments which should not be drawn.
If the fragment in your sprite texture has an alpha channel of 0 you should discard it.
Note, if you discard a fragment neither the color nor the depth and stencil buffer will be written.
Fragment shaders also have access to the discard command. When executed, this command causes the fragment's output values to be discarded. Thus, the fragment does not proceed on to the next pipeline stages, and any fragment shader outputs are lost.
Fragment shader
if ( color.a < 1.0/255.0 )
discard;
It's not possible to do this using the fixed-function blending in OpenGL ES 2.0, because what you want isn't actually alpha blending. What you want is a logical operation (e.g. max(src, dst)) which is rather different to how OpenGL ES blending works.
If you want to do path / stroke / fill rendering with pixel-exact edges you might get somewhere with using stencil masks and stencil tests, but you can't do transparency in this case - just boolean operators.
I'm trying to build a shader that allows you to combine two images by applying a gradient opacity mask to the one on top, like you do in photoshop. I've gotten to the point where I can overlay the masked image over the other but as a newbie am confused about a few things.
It seems that the images inside the shader are sometimes skewed to fit the canvas size, and they always start at position 0,0. I have played around with a few snippets I have found to try and scale the textures, but always end up with unsatisfactory results.
I am curious if there is a standard way to size, skew, and translate textures within a view, or if images in GLSL are necessarily limited in some way that will stop me from accomplishing my goal.
I'm also unsure of how I am applying the gradient/mask and if it is the right way to do it, because I do not have a lot of control over the shape of the gradient at the moment.
Here's what I have so far:
precision highp float;
varying vec2 uv;
uniform sampler2D originalImage;
uniform sampler2D image;
uniform vec2 resolution;
void main(){
float mask;
vec4 result;
vec2 position = gl_FragCoord.xy / ((resolution.x + resolution.y) * 2.0 );
mask = vec4(1.0,1.0,1.0,1.0 - position.x).a;
vec4 B = texture2D(image,uv);
vec4 A = texture2D(originalImage,uv) * (mask);
result = mix(B, A, A.a);
gl_FragColor = result;
}
Which produces an image like this:
What I would like to be able to do is change the positions of the images independently and also make sure that they conform to their proper dimensions.
I have tried naively shifting positions like this:
vec2 pos = uv;
pos.y = pos.y + 0.25;
texture2D(image, pos)
Which does shift the texture, but leads to a bunch of strange lines dragging:
I tried to get rid of them like this:
gl_FragColor = uv.y < 0.25 ? vec4(0.0,0.0,0.0,0.0) : result;
but it does nothing
You really need to decide what you want to happen when images are not the same size. What you probably want is for it to appear there's no image so check your UV coordinates and use 0,0,0,0 when outside of the image
//vec4 B = texture2D(image,uv);
vec4 getImage(sampler2D img, vec2 uv) {
if (uv.x < 0.0 || uv.x > 1.0 || uv.y < 0.0 || uv.y > 1.0) {
return vec4(0);
}
return texture2D(img, uv);
}
vec4 B = getImage(image, uv);
As for a standard way to size/skew/translate images use a matrix
uniform mat4 u_imageMatrix;
...
vec2 newUv = u_imageMatrix * vec4(uv, 0, 1).xy;
An example of implementing canvas 2d's drawImage using a texture matrix.
In general though I don't think most image manipulations programs/library would try to do everything in the shader. Rather they'd build up the image with very very simple primitives. My best guess would be they use a shader that's just A * MASK then draw B followed by A * MASK with blending on.
To put it another way, if you have 30 layers in photoshop they wouldn't generate a single shader that computes the final image in one giant shader taking in all 30 layers at once. Instead each layer would be applied on its own with simpler shaders.
I also would expect them to create an texture for the mask instead of using math in the shader. That way the mask can be arbitrarily complex, not just a 2 stop ramp.
Note I'm not saying you're doing it wrong. You're free to do whatever you want. I'm only saying I suspect that if you want to build a generic image manipulation library you'll have more success with smaller building blocks you combine rather than trying to do more complex things in shaders.
ps: I think getImage can be simplified to
vec4 getImage(sampler2D img, vec2 uv) {
return texture2D(img, uv) * step(0.0, uv) * step(-1.0, -uv);
}
New to Opengl and GLSL.
I am using OpenGL es 3.0 and my GLSL version #version 300 es.
i want to get pixel(ARGB data) at every position in my vertex shader(vertex texture fetch). i have verified that my Android tablet supports vertex texture fetch.
Now i pass in the texture(image) and Texture coordinates to the vertex shader
and execute
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, 4);
Is this the right way or should i use GL_POINTS .
if i am using GL_POINTS how to pass the texture cooordinate?
could you provide any samples/example code that does a full pixel read(ARGB) in the vertex shader.
attaching my vertex shader
uniform sampler2D sTexture;
in vec4 aTextureCoord;
out vec3 colorFactor;
vec2 vTextureCoord;
vec4 tex;
void main()
{
vTextureCoord = aTextureCoord.xy;
tex = texture(sTexture,vTextureCoord);
float luminance = 0.299 * tex.r + 0.587 * tex.g + 0.114 * tex.b;
colorFactor = vec3(1.0, 1.0, 1.0);
gl_Position = vec4(-1.0 + (luminance * 0.00784313725), 0.0, 0.0, 1.0);
gl_PointSize = 1.0;
};
My texture coordinates passed are
{0.f,1.f}
{1.f,1.f} {0.f,0.f}
{1.f,0.f}
and the shader is triggered by
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, 4);
Just declare a sampler and sample it exactly as per usual. E.g.
"#version 150\n"
in vec4 position;
in vec2 texCoordinate;
uniform sampler2D texID;
uniform mat4 modelViewProjection;
void main()
{
gl_Position = modelViewProjection * (position + texture(texID, texCoordinate));
}
That will sample a 4d vector from the texture unit texID at location texCoordinate, using that to perturb position prior to applying the model-view-projection matrix. The type of geometry you're drawing makes no difference.
gl_Position = vec4(-1.0 + (luminance * 0.00784313725), 0.0, 0.0, 1.0);
That's clever. It's not going to work, but that's clever.
The thing that is confusing everyone is what you haven't told us. That you're computing the histogram by using blending. That you compute the location for each fragment based on the luminance, so you get lots of overlap. And blending just adds everything together, thus producing your histogram.
FYI: It's always best to explain what it is you're actually trying to accomplish in your question, rather than hoping that someone can deduce what you're attempting to do.
That's not going to work because you only have four vertices in this case. You have lots of fragments, but they will be generated based on interpolation from your 4 vertices. And you can't change the position of a fragment from within a fragment shader.
If you want to do what you're trying to do, you still have to render one vertex for every texel you fetch. You still need to use GL_POINTS.
I am drawing a triangle in OpenGL like:
MyGLRenderer( )
{
fSampleVertices = ByteBuffer.allocateDirect( fSampleVerticesData.length * 4 )
.order ( ByteOrder.nativeOrder( ) ).asFloatBuffer( );
fSampleVertices.put( fSampleVerticesData ).position ( 0 );
Log.d( TAG, "MyGLRender( )" );
}
private FloatBuffer fSampleVertices;
private final float[] fSampleVerticesData =
{ .8f, .8f, 0.0f, -.8f, .8f, 0.0f, -.8f, -.8f, 0.0f };
public void onDrawFrame( GL10 unused )
{
GLES30.glViewport ( 0, 0, mWidth, mHeight );
GLES30.glClear ( GLES30.GL_COLOR_BUFFER_BIT );
GLES30.glUseProgram ( dProgramObject1 );
GLES30.glVertexAttribPointer ( 0, 3, GLES30.GL_FLOAT, false, 0, fSampleVertices );
GLES30.glEnableVertexAttribArray ( 0 );
GLES30.glDrawArrays( GLES30.GL_TRIANGLES, 0, 3 );
//Log.d( TAG, "onDrawFrame( )" );
}
So since I have experimented with the co-ordinates it doesn't take long to figure out that the visible area of the screen
is between -1,1. So then the triangle takes up 80% of the screen. As well I have determined that the pixel dimensions of my
GLSurfaceView are 2560 in width and 1600 in height.
So then given a triangle with these pixel based co-ordinates (fBoardOuter):
1112.0f
800.0f
0.0f
-1280.0f
800.0f
0.0f
-1280.0f
-800.0f
0.0f
I have to either convert those pixel co-ordinates to something between -1,1 or find out a way to have gl convert those co-ordinates
at the time they are drawn? Since I am very new to OpenGL I am looking for some guidance to do this?
My vertex shader is like:
String sVertexShader1 =
"#version 300 es \n"
+ "in vec4 vPosition; \n"
+ "void main() \n"
+ "{ \n"
+ " gl_Position = vPosition; \n"
+ "} \n";
Would I be correct then in saying that a pixels based system would be called world co-ordinates? What I am trying to do right now is just some 2D drawing for a board game.
I've discovered that Android has this function:
orthoM(float[] m, int mOffset, float left, float right, float bottom, float top, float near, float far)
However there is nothing in the documentation I've read so far that explain the usage of the matrix of how a float[] with pixel co-ordinates can be transformed to normalized co-ordinates with that matrix in GLES30.
I've also found the documentation here:
http://developer.android.com/guide/topics/graphics/opengl.html
Based off the documentation I have tried to create an example:
http://pastebin.com/5PTsfSdz
In the pastebin example fSampleVertices I thought would be much smaller and at the center of the screen but it isn't it's still almost the entire screen and fBoardOuter just shows me a black screen if I try to put it into glDrawArray.
You will probably need to find a book or some good tutorials to get a strong grasp on some of these concepts. But since there some specific items in your question, I'll try and explain them as well as I can within this format.
The coordinate system you discovered, where the range is [-1.0, 1.0] in the x- and y coordinate directions, is officially called Normalized Device Coordinates, often abbreviated as NDC. Which is very similar to the name you came up with, so some of the OpenGL terminology is actually very logical. :)
At least as long as you're dealing with 2D coordinates, this is the coordinate range your vertex shader needs to produce. I.e. the coordinates you assign to the built-in gl_Position variable need to be within this range to be visible in the output. Things gets slightly more complicated if you're dealing with 3D coordinates and are applying perspective projections, but we'll skip over that part for now.
Now, as you already guessed, you have two main options if you want to specify your coordinates in a different coordinate system:
You transform them to NDC in your code before you pass them to OpenGL.
You have OpenGL apply transformations to your input coordinates.
Option 2 is clearly the better one, since GPUs are very efficient at performing this job.
On a very simple level, this means that you modify the coordinates in your vertex shader. If you look at your very simple first vertex shader:
in vec4 vPosition;
void main()
{
gl_Position = vPosition;
}
you get the coordinates provided by your app code in the vPosition input variable, and you assign exactly he same coordinates to the vertex shader output gl_Position.
If you want to use a different coordinate system, you process the input coordinates in the vertex shader code, and assign those processed coordinates to the output instead.
Modern versions of OpenGL don't really have a name for those coordinate systems anymore. There used to be "model coordinates" and "world coordinates" when some of this stuff was still hardwired into a fixed pipeline. Now that this is done with programmable shader code, those concepts are not relevant anymore from the OpenGL point of view. All it cares about are the coordinates that come out of the vertex shader. Everything that happens before that is your own business.
The canonical way of applying linear transformations, which includes the translations and scaling you need for your intended use, is by multiplying the coordinates with a transformation matrix. You already discovered the android.opengl.Matrix package that contains some utility functions for building transformation matrices if you don't want to write the (simple) code yourself.
Once you have a transformation matrix, you pass it into the vertex shader as a uniform variable, and apply the matrix in your shader code. The way this looks in the shader code is for example:
in vec4 vPosition;
uniform mat4 TransformMat;
void main()
{
gl_Position = TransformMat * vPosition;
}
To set the value of this matrix, you need to get the location of the uniform variable once after linking the shader, with prog your shader program:
GLint transformLoc = GLES20.glGetUniformLocation(prog, "TransformMat");
Then, at least once, and every time you want to change the matrix, you call:
GLES20.glUniformMatrix4fv(transformLoc, 1, GL_FALSE, mat, 0);
where mat is the matrix you either built yourself, or got from one of the utility functions in android.opengl.Matrix. Note that this call needs to be after you make the program current with glUseProgram().
In certain circumstances, there is a desire to expand a texture when the user is zooming into an image until the shortest edge fills the entire screen (as seen with standard galleries). However, the input to the OpenGL shader is a matrix from an ImageView, so certain translations need to be put into place as well as corrective calculations for different spaces.
EDIT:
The vertex shader code is simplistic and is as follows:
private final String vertexShader_ =
"attribute vec4 a_position;\n" +
"attribute vec4 a_texCoord;\n" +
"varying vec2 v_texCoord;\n" +
"void main() {\n" +
" gl_Position = a_position;\n" +
" v_texCoord = a_texCoord.xy;\n" +
"}\n";
where a_position references verticesData_ below. When loading onto the screen, the position of the image is accurate based upon calculations from the display width and a portion of the screen that is occupies.
Additionally, I have the following fragment shader code:
private final String fragmentShader_ =
"precision mediump float;\n" +
"varying vec2 v_texCoord;\n" +
"uniform sampler2D texture;\n" +
"uniform mat3 transform;\n" +
"void main() {\n" +
" vec2 uv = (transform * vec3(v_texCoord, 1.0)).xy;\n" +
" gl_FragColor = texture2D( texture, uv );\n" +
"}\n";
where the input of the mat3 transform is input that is coming from an ImageView. Essentially, there is an ImageView underneath the OpenGLSurfaceView. It is a much lower resolution image and when the user swipes, the SurfaceView is hidden and the ImageView is beneath it at the same position that the user was in the SurfaceView.
However, later on when I want to expand upon this texture, I am finding myself will unexpected results.
When panning back and forth, the screen moves where it is expected to move. So, the translation in the x and y coordinates from the matrix are coming across ok. However, when zooming in, the image is tearing. It is only tearing when the bounds of the texture have grown beyond the screen dimensions. As can be seen below, the height is not introducing extra pixels when growing, but the width is tearing as it progresses.
In order to pass an appropriate matrix, the values from the ImageView to the OpenGL SurfaceView were inverted and transposed, since the space required it for an appropriate conversion. A scale factor is passed into the a listener that the Activity is listening in on.
#Override
public void onTranslation(float imageScaleF, float[] matrixTranslationF)
{
currentScaleForImage_ = imageScaleF;
//If we are currently using the entire screen, then there is not need anymore to resize the texture itself.
if(surfaceViewWidth_ * imageScaleF > displayViewWidth_ && surfaceViewHeight_ * imageScaleF > displayViewHeight_)
{
}
else //Expand the size of the texture to be displayed on the screen.
{
maximumScaleForImage_ = imageScaleF;
photoViewRenderer_.updateTextureSize(imageScaleF);
photoSurfaceView_.requestRender();
}
matrixTranslationF = updateTranslationBasedOnTextureSize(matrixTranslationF);
photoViewRenderer_.updateTranslationOfTexture(matrixTranslationF);
photoSurfaceView_.requestRender();
}
But, with the following code above, the portion of the image that scrolls is always cut short, and if I attempt to correct it based upon the scale by uncommenting the line there for updating the Translation, it causes tears. So, it seems as though at this point, the user below has gotten me into a better position with their input, but I am still a single step away and I think that it is within this function. The following now updated code below has made the corrections necessary in order to provide an appropriate translation between the matricies in the ImageView and the OpenGL texture/coordinates.
private float[] updateTranslationBasedOnTextureSize(float[] matrixTranslationsF)
{
if(scaleDirection_ == ScaleDirection.WIDTH)
{
matrixTranslationsF[0] = matrixTranslationsF[0] * maximumScaleForImage_;
}
else if(scaleDirection_ == ScaleDirection.HEIGHT)
{
matrixTranslationsF[4] = matrixTranslationsF[4] * maximumScaleForImage_;
}
return matrixTranslationsF;
}
I was causing tearing in the photographs in the first iteration as seen below. Or it was clipping the photograph due to improper conversions between the two spaces.
The width seems to be maintained, so it suggests to me that maybe something happens with values outside the standard bounds, but I am not sure what to try.
This streaking is happening because you have the texture wrap set to clamp to edge. You have something like:
glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );
glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );
GL_REPEAT is the default and will repeat the image. But you're looking to not have any "effect" at the bottom. This bottom is happening because of your MATH. You are getting the streaking only on the bottom and not the sides. As someone said in the comment you're vertex range isn't correct. The min and max are 0 and 1.. not -1 and 1. So at the bottom of your picture it's streaking because you're calculating a texture coordinate number out of the 0-1 bounds and probably getting in to the <0 area or the >1 area. When the texture2D(u_textureSampler, v_fragmentTexCoord0) gets a texture coordinate outside of the 0-1 range it's going to fall on that WRAP_S / WRAP_T setting to know what value to give. It's repeating the edge pixel because you have GL_CLAMP_TO_EDGE.
Your actual problem though isn't this setting. It's that you are calculating an S or T outside of the range to begin with. That's not good form cause even if you were writing black in there it's wasted cycles on the fragment shader and will slow down your app. Basically, your formulas are wrong.
As for your zooming in math... it's math. You'll have to work it out for whatever you're trying to do. (Or ask for help.) You could do something like this, which is a simplified version of what I'm doing in my new photo editing app:
GLfloat aspect = photo.width / photo.height;
if (aspect > 1) {
displayWidth = panelWidth;
displayHeight = panelWidth/aspect;
}
else {
displayHeight = panelHeight;
displayWidth = panelHeight*destAspect;
}
Where panelWidth and panelHeight are the size of the area you're displaying your photo - which could be the full screen or something smaller. Then build your vertex based on the display height and width. If you zoom in beyond the size of the panel you'll need to then do a crop to keep the image inside the panel, both by reducing the display width or height (which ever one is too large) and also changing your texture coordinates to match. This is very rough but something like this:
GLfloat sSize = (destSMax-destSMin) / panelZoom;
GLfloat tSize = (destTMax-destTMin) / panelZoom;
displaySMin = destSMin + panelPanX/coreTextureSize;
displayTMin = destTMin + panelPanY/coreTextureSize;
displaySMax = displaySMin + sSize;
displayTMax = displayTMin + tSize;
Then of course build your vertex data, use matrix math to move the square where you want it, and draw it.
I recommend rethinking your approach to using something more like what I'm doing above. Don't remove a percentage of the photo when it goes off screen. Instead calculate what is going to be seen and build your data based on that.
EDIT - Also, I realized my code is ios code and this is an android question. Same concept applies though. The glTexParameteri call might be slightly different and it might be float instead of GLfloat.