I'm learning OpenGL programming on Android at the moment and now I'm at the point where I have to use a ModelMatrix and a ProjectionMatrix. If I understand it right I have three different matrix in OpenGL:
ProjectionMatrix which creates the 3D efect (does not change while running the app)
ModelMatrix with this I can move objects around
ViewMatrix with which I can move all objects around (Camera)
Now in my tutorial book OpenGL ES 2 for Androud: A Quick-Start Guide) the ProjectionMatrix and the ModelMatrix are multiplied to one single matrix. But I do not understand why. Is that the correct way? With the ModelMatrix I can move objects around so why should I multiplay it with an other matrix? Would be great if you can help me to better understand the different matrix in OpenGL.
Right now, you're used to taking your vertex, left-multiplying it by a model matrix to rotate it/scale it/move it around in the world, then left-multiplying it by a view matrix to get its position relative to the camera, then left-multiplying it by a projection matrix to get that "3d effect".
In other words, you're taking your vertex v, then doing this to it:
v' = P * V * M * v, where in v', x and y is the pixel location on the screen, and z is the "depth" of the vertex.
But matrix multiplication (whether its a 4x4 matrix or a 4x1 matrix/vector) is associative, meaning:
A * B * v is the same thing as (A * B) * v, where A and B are matrices and v is some vector.
So instead of taking a thousand or so vertices and then multiplying it by M, then V, then P, why not pre-multiply M, V, and P once to create an MVP matrix then multiply every vertex by that? That way, when your vertex shader runs on each vertex, it only has to do one matrix multiplication rather than three.
Though I am not sure why your tutorial would be multiplying only the model matrix and the perspective matrix. Is there a view matrix present?
Related
I saw from OpenGL ES Android development guide said "...The projection matrix recalculates the coordinates of your graphics so that they map correctly to Android device screens."
And it's true, a simple square renders correctly on my phone after I multiply project matrix, but I am curious how the magic works. (And my intuition is model matrix should do the scale job)
Can anyone explained it in simply language without involving too much computer graphics background?
If you target OpenGL ES 1.0 there is the OpenGL fixed-function pipeline. And here we have the GL_PROJECTION and the GL_MODELVIEW matrix stacks (actually there is another one for texture operations, but that is irrelevant here).
What happens is that the fixed-function pipeline simply concatenates/multiplies the top matrix of those two matrix stacks (let's refer to those as P and MV for projection matrix and modelview matrix, respectively) as P * MV during each draw call and transforms any drawn vertex v by P * MV * v.
When you target OpenGL ES 2.0 or higher, there are no such fixed-function matrix stacks anymore, but you would be using shaders and uploading your own matrices (e.g. as uniforms or by other means). There, you are completely free as to how you decomposition the final transformation (previously P * MV in the fixed-function pipeline). In the end, all that you want to do is to transform vertices from their space (typically called "model space") into "clip space" coordinates. Clip space is the only thing that OpenGL ES 2.0 (or higher) actually cares about.
It's just that decompositing the final matrix (model -> clip space) into projection, view and model matrices is convenient, because of the frequency that they change during a whole scene render. The projection matrix would typically only contain those transformations that stay constant and only needs to be refreshed when you resize the window (or reconfigure the vertical field of view). That is to say, this matrix transforms from "view space" to "clip space" and represents the intrinsic properties of a camera.
Now, in most cases there is also a dedicated "view" matrix, which represents the external properties of a camera, which is typically the position and orientation of the camera in the world. This matrix thusly changes when you rotate/move the virtual camera. And this matrix transforms from "model" to "view" space.
Next, there would be the model matrices, which represent the transformations of each model in the world.
And finally, when you want to render anything, you'd use the decomposition P * V * M (projection matrix multiplied by view matrix multiplied by model matrix) to transform a vertex through the whole chain of model -> clip space.
Now to the question about why we do the scaling in the projection matrix:
The reason is that once you are in view space (after the GL_MODELVIEW transformation or your custom V * M transformation), you have a space whose axes are aligned with the screen. So, you know what to scale (the X or Y axes) and by how much. You cannot simply scale the model, since that might also be rotated and translated arbitrarily.
I'm currently trying to implement an AR-browser based on indoor maps, but I'm facing several problems, let's take a look at the figure:
In this figure, I've already changed the coordinate to OpenGL's right-handed coordinate system.
In our real-world scenario,
given the angle FOV/2 and the camera height h then I can get nearest visible point P(0,0,-n).
Given the angle B and the camera height h then I can get a point Q(0,0,-m) between nearest visible point and longest visible point.
Here comes a problem: when I finished setup my vertices(including P and Q) and use the method Matrix.setLookAtM like
Matrix.setLookAtM(modelMatrix, 0, 0f,h,0f,0f,-2000f,0f,0f,1f,0f);
the aspect ratio is incorrect.
If the camera height h is set to 0.92 and FOV is set to 68 degrees, n should be 1.43, But in OpenGL the coordinate of the nearest point is not (0,0,-1.43f). So I'm wondering how to fix this problem, how to map real-world coordinate to OpenGL's coordinate system?
In a rendering, each mesh of the scene usually is transformed by the model matrix, the view matrix and the projection matrix.
Model matrix:
The model matrix defines the location, oriantation and the relative size of a mesh in the scene. The model matrix transforms the vertex positions of the mesh to the world space.
View matrix:
The view matrix describes the direction and position from which the scene is looked at. The view matrix transforms from the wolrd space to the view (eye) space. In the coordinat system on the viewport, the X-axis points to the left, the Y-axis up and the Z-axis out of the view (Note in a right hand system the Z-Axis is the cross product of the X-Axis and the Y-Axis).
The view matrix can be set up by Matrix.setLookAtM
Projection matrix:
The projection matrix describes the mapping from 3D points of a scene, to 2D points of the viewport. The projection matrix transforms from view space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) in the range (-1, -1, -1) to (1, 1, 1) by dividing with the w component of the clip coordinates.At Perspective Projection the projection matrix describes the mapping from 3D points in the world as they are seen from of a pinhole camera, to 2D points of the viewport. The eye space coordinates in the camera frustum (a truncated pyramid) are mapped to a cube (the normalized device coordinates).
The perspective projection matrix can be set up by Matrix.perspectiveM
You can set up a separate view matrix and a separate projection matrix and finally multiply them. The aspect ratio and the field of view are parameters to [Matrix.perspectiveM]:
Matrix viewM = new Matrix();
Matrix.setLookAtM(viewM, 0, 0, 0, 0f, 0f,-2000f, 0f, 0f, 1.0f, 0.0f);
Matrix prjM = new Matrix();
Matrix.perspectiveM(prjM, 0, fovy, aspect, zNear, zFar);
Matrix viewPrjM = new Matrix();
Matrix.multiplyMM(viewPrjM, 0, prjM, 0, viewM, 0);
Thank to #Rabbid76's support, I finally figure it out myself.
Figure 1: Real-life scenario
Figure 2: OpenGL scenario
In real-life, if we are facing north, we coordinate system would be like:
x point to the east
y point to the north
z point to the sky
so given a camera held by a user, assuming its height is 1.5 meter and its field of view is 68 degrees, we can reference the nearest visible point is located at P(0,2.223,0). We can set the angle B to 89 degrees, so segment QP will be the visible ground on the smartphone screen.
How can we map the coordinate of real-life to OpenGL coordinate system? I found that we must go through several steps:
Assign the camera position to be the origin (e.g. C in figure2).
Due to OpenGL always draw from (1,1) to (-1,-1), we must assign the distance from C to C' to be 1, so that C' is (0, -1, 0).
Finally, we calculate the aspect ratio with camera height in real-life and segment C, C' in OpenGL, and apply it to other coordinates.
By doing stuff above, we can map real-world coordinate to OpenGL coordinate system magically.
I am learning OpenGL ES 2.0, without ever having learned OpenGL or OpenGL ES 1.x.
I'm applying non-uniform scaling to my modelViewMatrix, so the tutorials tell me that I need to take special steps to compute a normalMatrix. In my application the modelViewMatrix has dimension 4x4.
Some tutorials say that for the normalMatrix I need to simply calculate transpose(inverse(modelViewMatrix)).
Other instructions say that I need to first take the upper left 3x3 sub-matrix of my modelViewMatrix and then compute transpose(inverse(submatrix)).
Is there any difference? Do they lead to the same result?
Right now I'm using method 1, and then in the vertex shader I extract a vec3 after applying the transformation:
vec3 vNormalEyespace = vec3(normalMatrix * vec4(vertexNormal, 1.0));
I am doubting this because I see strange effects in my diffuse lighting. I'm thinking about trying method 2, but the android.opengl.Matrix class does not offer methods for inverting or transposing 3x3 matrices...
My actual code in renderFrame() is as follows:
final float[] normalMatrix=new float[16];
final float[] unscaledModelViewMatrix=modelViewMatrix_Vuforia.getData();
Matrix.invertM(normalMatrix, 0, unscaledModelViewMatrix, 0);
Matrix.transposeM(normalMatrix, 0, normalMatrix, 0);
// pass the normalMatrix to the shader
GLES20.glUniformMatrix4fv(normalMatrixHandleBottle, 1, false, normalMatrix, 0);
A 3x3 matrix is enough to transform the normals.
The primary purpose of using 4x4 matrices that operate on homogenous coordinates for positions is that they can express translations. A 3x3 matrix applied to a 3-member vector can not express translations. You can easily confirm that because it will always map the origin back to the origin.
Since normals are vectors, and not positions, we specifically do not want to apply the translation part of the modelview matrix to them. They describe directions, and directions do not change when a translation is applied.
So the cleanest approach is to use a 3x3 matrix for normals, and set it to the inverse-transpose of the top-left 3x3 elements of the modelview matrix.
In the case where you only have rotations and uniform scaling in the modelview matrix (which does not apply to your specific situation), people sometimes use the same modelview matrix that they also use for the positions. Which is correct, as long as the translation part is not applied. This can be done by setting the w component of the vector to zero for the multiplication:
vec3 transformedNormal = (modelViewMatrix * vec4(originalNormal, 0.0)).xyz
With the w component being zero, the matrix elements that encode the translation have no effect on the result, and this corresponds to using only the top-left 3x3 part of the matrix. As long as this is the same as the inverse-transpose of the matrix, which is the case for rotations and uniform scaling, this is a valid shortcut.
i´m new to openGL and want to draw lines on an GLSurfaceView.
I switched from canvas to openGL cause of the performance.
I know that OpenGL is more likely for 3D drawings, so pixel are not used because of this.
I want to draw 2D graphics like lines but need the dimensions in pixel, only x and y values without 3rd dimension.
Set properly your projection and view matrices. For first, use orthographic projection, second leave identity.
P = Diagonal(2 / W, 2 / H, 2 / (10 + 10), 1)
// Assume Znear=-10, ZFar=10, W and H are width and Height of the screen.
So coordinates you see will bew in cube -W/2 to W/2, -H/2 to H/2 and -10 to 10. You may even translate in view matrix to move origin from -W/2, -H/2 to 0, 0 translating view matrix: V = Translate(W/2, H/2, 0).
I'm am working on a basic augmented reality application on Android. What I did so far is detect a square with opencv and then using cvFindExtrinsicCameraParams2() I calculated a rotation and translation vector. For this I used 4 object points, which are just the corners of a square around (0,0,0) and the 4 corners of the square in the image.
This yields me a pretty good rotation and translation matrix. I also calculated the rotation matrix with cvRodrigues2() since using this is easier than the rotation vector. As long as I use these to draw some points in the image everything works fine. My next step is however to pass these vectors and the matrix back to java and then use them with OpenGL to draw a square in an OpenGLView. The square should be exactly around the square in the image which is displayed behind the OpenGLView.
My problem is that I cannot find the correct way of using the rotation matrix and translation vector in OpenGL. I started of with exactly the same object points as used for the openCV functions. Then I applied the rotation matrix and translation vector in pretty much any possible way I could think of. Sadly none of these approaches produce a result which is anyway near what I hoped for. Can anyone tell me how to use them correctly?
So far the "closest" results I have gotten, was when randomly multiplying the whole matrix with -1. But most of the time the squares still look mirror inverted or rotated for 180 degrees. So I guess it was just a lucky hit, but not the right approach.
Okay after some more testing I finally managed to get it to work. While I don't understand it... it does 'work'. For anyone who will need to do this in the future here is my solution.
float rv[3]; // the rotation vector
float rotMat[9]; // rotation matrix
float tv[3]; // translation vector.
rv[1]=-1.0f * rv[1]; rv[2]=-1.0f * rv[2];
//Convert the rotation vector into a matrix here.
//Complete matrix ready to use for OpenGL
float RTMat[] = {rotMat[0], rotMat[3], rotMat[6], 0.0f,
rotMat[1], rotMat[4], rotMat[7], 0.0f,
rotMat[2], rotMat[5], rotMat[8], 0.0f,
tv[0], -tv[1], -tv[2], 1.0f};
As genpfault said in his comment everything needs to be transposed since OpenGL since OpenGL needs a column-major order. (Thanks for the comment, I saw that page earlier already.) Furthermore the y and z rotation angle as well as the y and z translation need to be multiplied by -1. This is what I find a bit weird. Why only those and not the x values too?
This works as it should I guess. But corners the don't match exactly. I guess this is caused by some wrong openGLView configurations. So even though I am still not a 100% happy with my solution I guess it is the answer to my question.
Pandoro's method really works! In case someone wondering "how to convert the rotation vector into a rotation matrix" here's how I did it. By the way, I've used these in OpenGL 2, not ES.
// use the rotation vector generated from OpenCV's cvFindExtrinsicCameraParams2()
float rv[] = {rotation->data.fl[0], rotation->data.fl[1], rotation->data.fl[2] };
// use the translation vector generated from OpenCV's cvFindExtrinsicCameraParams2()
float tv[] = {translation->data.fl[0], translation->data.fl[1], translation->data.fl[2]} ;
float rm[9];
// rotation matrix
CvMat* rotMat = cvCreateMat (3, 3, CV_32FC1);
// rotation vectors can be converted to a 3-by-3 rotation matrix
// by calling cvRodrigues2() - Source: O'Reilly Learning OpenCV
cvRodrigues2(rotation, rotMat, NULL);
for(int i=0; i<9; i++){
rm[i] = rotMat->data.fl[i];
}
rv[1]=-1.0f * rv[1]; rv[2]=-1.0f * rv[2];
//Convert the rotation vector into a matrix here.
//Complete matrix ready to use for OpenGL
float RTMat[] = {rm[0], rm[3], rm[6], 0.0f,
rm[1], rm[4], rm[7], 0.0f,
rm[2], rm[5], rm[8], 0.0f,
tv[0], -tv[1], -tv[2], 1.0f};
Good luck!