I'm currently trying to implement an AR-browser based on indoor maps, but I'm facing several problems, let's take a look at the figure:
In this figure, I've already changed the coordinate to OpenGL's right-handed coordinate system.
In our real-world scenario,
given the angle FOV/2 and the camera height h then I can get nearest visible point P(0,0,-n).
Given the angle B and the camera height h then I can get a point Q(0,0,-m) between nearest visible point and longest visible point.
Here comes a problem: when I finished setup my vertices(including P and Q) and use the method Matrix.setLookAtM like
Matrix.setLookAtM(modelMatrix, 0, 0f,h,0f,0f,-2000f,0f,0f,1f,0f);
the aspect ratio is incorrect.
If the camera height h is set to 0.92 and FOV is set to 68 degrees, n should be 1.43, But in OpenGL the coordinate of the nearest point is not (0,0,-1.43f). So I'm wondering how to fix this problem, how to map real-world coordinate to OpenGL's coordinate system?
In a rendering, each mesh of the scene usually is transformed by the model matrix, the view matrix and the projection matrix.
Model matrix:
The model matrix defines the location, oriantation and the relative size of a mesh in the scene. The model matrix transforms the vertex positions of the mesh to the world space.
View matrix:
The view matrix describes the direction and position from which the scene is looked at. The view matrix transforms from the wolrd space to the view (eye) space. In the coordinat system on the viewport, the X-axis points to the left, the Y-axis up and the Z-axis out of the view (Note in a right hand system the Z-Axis is the cross product of the X-Axis and the Y-Axis).
The view matrix can be set up by Matrix.setLookAtM
Projection matrix:
The projection matrix describes the mapping from 3D points of a scene, to 2D points of the viewport. The projection matrix transforms from view space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) in the range (-1, -1, -1) to (1, 1, 1) by dividing with the w component of the clip coordinates.At Perspective Projection the projection matrix describes the mapping from 3D points in the world as they are seen from of a pinhole camera, to 2D points of the viewport. The eye space coordinates in the camera frustum (a truncated pyramid) are mapped to a cube (the normalized device coordinates).
The perspective projection matrix can be set up by Matrix.perspectiveM
You can set up a separate view matrix and a separate projection matrix and finally multiply them. The aspect ratio and the field of view are parameters to [Matrix.perspectiveM]:
Matrix viewM = new Matrix();
Matrix.setLookAtM(viewM, 0, 0, 0, 0f, 0f,-2000f, 0f, 0f, 1.0f, 0.0f);
Matrix prjM = new Matrix();
Matrix.perspectiveM(prjM, 0, fovy, aspect, zNear, zFar);
Matrix viewPrjM = new Matrix();
Matrix.multiplyMM(viewPrjM, 0, prjM, 0, viewM, 0);
Thank to #Rabbid76's support, I finally figure it out myself.
Figure 1: Real-life scenario
Figure 2: OpenGL scenario
In real-life, if we are facing north, we coordinate system would be like:
x point to the east
y point to the north
z point to the sky
so given a camera held by a user, assuming its height is 1.5 meter and its field of view is 68 degrees, we can reference the nearest visible point is located at P(0,2.223,0). We can set the angle B to 89 degrees, so segment QP will be the visible ground on the smartphone screen.
How can we map the coordinate of real-life to OpenGL coordinate system? I found that we must go through several steps:
Assign the camera position to be the origin (e.g. C in figure2).
Due to OpenGL always draw from (1,1) to (-1,-1), we must assign the distance from C to C' to be 1, so that C' is (0, -1, 0).
Finally, we calculate the aspect ratio with camera height in real-life and segment C, C' in OpenGL, and apply it to other coordinates.
By doing stuff above, we can map real-world coordinate to OpenGL coordinate system magically.
Related
I am developing a 2d game where I cant rotate my view using the rotation sensor and view different textures on screen.
I am scattering all the textures using this method :
public void position(ShaderProgram program, float[] rotationMatrix , float[] projectionMatrix , float longitude , float latitude , float radius)
{
this.radius = radius;
viewMat = new float[MATRIX_SIZE];
mvpMatrix = new float[MATRIX_SIZE];
// correct coordinate system to fit landscape orientation
SensorManager.remapCoordinateSystem(rotationMatrix, SensorManager.AXIS_Y, SensorManager.AXIS_MINUS_X, viewMat);
//correct the axis so that the direction of Y axis is to the sky and Z is to the front
Matrix.rotateM(viewMat, 0, -90f, 1f, 0f, 0f);
// first rotation - longitude
Matrix.rotateM(viewMat, 0, longitude, 0f, 1f, 0f);
//second rotation - latitude
Matrix.rotateM(viewMat, 0, latitude, 1f, 0f, 0f);
// used to control the distance of viewing the texture (currently only z translation is used)
Matrix.translateM(viewMat, 0 , 0f , 0f , radius);
//multiply the adjusted view matrix with projection matrix
Matrix.multiplyMM(mvpMatrix, 0, projectionMatrix, 0, viewMat, 0);
//send mvp matrix to shader
GLES20.glUniformMatrix4fv(program.getMatrixLocation(), 1, false, mvpMatrix, 0);
}
however when I render large amount of textures , the framerate becomes very laggy . so I thought about using culling.
how should I perform the culling test after I have a different view matrix for every texture?
what I mean is , how do I compare if the matrix that represent where I'm viewing right now intersects with the matrix represents each texture so I'll decide if to draw it or not ?
There are many ways on doing this but each of them will need more then just a matrix. A matrix (assuming the center of the object is at 0,0 without applying any matrix) alone will not handle cases where you may see only a part of the object.
You may define boundaries of the original object with 8 points such as a cube. Imagine if you draw these 8 points with the same matrix as the object the points will appear around the object so that they can define a surface which will box the object itself.
So these points may then be multiplied with your resulting matrix (the whole MVP matrix) which will project them to the openGL drawable part of the coordinate system. Now you only need to check that if any of these points is inside [-1,1] in every axis then you must draw the object. So x, y and z must be between -1 and 1.
Update:
Actually that will not be enough as the intersection may happen even if all of the 8 points are outside those coordinates. You will need a proper algorithm to find the intersection of the 2 shapes...
I am learning OpenGL ES 2.0, without ever having learned OpenGL or OpenGL ES 1.x.
I'm applying non-uniform scaling to my modelViewMatrix, so the tutorials tell me that I need to take special steps to compute a normalMatrix. In my application the modelViewMatrix has dimension 4x4.
Some tutorials say that for the normalMatrix I need to simply calculate transpose(inverse(modelViewMatrix)).
Other instructions say that I need to first take the upper left 3x3 sub-matrix of my modelViewMatrix and then compute transpose(inverse(submatrix)).
Is there any difference? Do they lead to the same result?
Right now I'm using method 1, and then in the vertex shader I extract a vec3 after applying the transformation:
vec3 vNormalEyespace = vec3(normalMatrix * vec4(vertexNormal, 1.0));
I am doubting this because I see strange effects in my diffuse lighting. I'm thinking about trying method 2, but the android.opengl.Matrix class does not offer methods for inverting or transposing 3x3 matrices...
My actual code in renderFrame() is as follows:
final float[] normalMatrix=new float[16];
final float[] unscaledModelViewMatrix=modelViewMatrix_Vuforia.getData();
Matrix.invertM(normalMatrix, 0, unscaledModelViewMatrix, 0);
Matrix.transposeM(normalMatrix, 0, normalMatrix, 0);
// pass the normalMatrix to the shader
GLES20.glUniformMatrix4fv(normalMatrixHandleBottle, 1, false, normalMatrix, 0);
A 3x3 matrix is enough to transform the normals.
The primary purpose of using 4x4 matrices that operate on homogenous coordinates for positions is that they can express translations. A 3x3 matrix applied to a 3-member vector can not express translations. You can easily confirm that because it will always map the origin back to the origin.
Since normals are vectors, and not positions, we specifically do not want to apply the translation part of the modelview matrix to them. They describe directions, and directions do not change when a translation is applied.
So the cleanest approach is to use a 3x3 matrix for normals, and set it to the inverse-transpose of the top-left 3x3 elements of the modelview matrix.
In the case where you only have rotations and uniform scaling in the modelview matrix (which does not apply to your specific situation), people sometimes use the same modelview matrix that they also use for the positions. Which is correct, as long as the translation part is not applied. This can be done by setting the w component of the vector to zero for the multiplication:
vec3 transformedNormal = (modelViewMatrix * vec4(originalNormal, 0.0)).xyz
With the w component being zero, the matrix elements that encode the translation have no effect on the result, and this corresponds to using only the top-left 3x3 part of the matrix. As long as this is the same as the inverse-transpose of the matrix, which is the case for rotations and uniform scaling, this is a valid shortcut.
I'm learning OpenGL programming on Android at the moment and now I'm at the point where I have to use a ModelMatrix and a ProjectionMatrix. If I understand it right I have three different matrix in OpenGL:
ProjectionMatrix which creates the 3D efect (does not change while running the app)
ModelMatrix with this I can move objects around
ViewMatrix with which I can move all objects around (Camera)
Now in my tutorial book OpenGL ES 2 for Androud: A Quick-Start Guide) the ProjectionMatrix and the ModelMatrix are multiplied to one single matrix. But I do not understand why. Is that the correct way? With the ModelMatrix I can move objects around so why should I multiplay it with an other matrix? Would be great if you can help me to better understand the different matrix in OpenGL.
Right now, you're used to taking your vertex, left-multiplying it by a model matrix to rotate it/scale it/move it around in the world, then left-multiplying it by a view matrix to get its position relative to the camera, then left-multiplying it by a projection matrix to get that "3d effect".
In other words, you're taking your vertex v, then doing this to it:
v' = P * V * M * v, where in v', x and y is the pixel location on the screen, and z is the "depth" of the vertex.
But matrix multiplication (whether its a 4x4 matrix or a 4x1 matrix/vector) is associative, meaning:
A * B * v is the same thing as (A * B) * v, where A and B are matrices and v is some vector.
So instead of taking a thousand or so vertices and then multiplying it by M, then V, then P, why not pre-multiply M, V, and P once to create an MVP matrix then multiply every vertex by that? That way, when your vertex shader runs on each vertex, it only has to do one matrix multiplication rather than three.
Though I am not sure why your tutorial would be multiplying only the model matrix and the perspective matrix. Is there a view matrix present?
I have been trying to do Cube of Rubik for android. I have one question about rotations. I want to rotate a figure visually correct. It means if user touch screen and after move his finger to right a figure rotate to right from side of observation point. But when I make some rotations the figure start move in not correct direction. I understand that it depends on that axis are change their situation. But I tried to use inverse model matrix to get necessary coordinates, but I haven't already result. Could anybody give me example or link of visually correct rotation of 3D figure with help of mouse or touch screen?
//get vector 3D of touch
Vector3f touchVector = getRubikSystemCoordinates(mTouchX,mTouchY,square.rubikRotationMatrix);
//Get vector 3D of move
Vector3f moveVector = getRubikSystemCoordinates(mMoveX,mMoveY,square.rubikRotationMatrix);
//get direction of motion
float direction = touchVector.substractFrom(moveVector);
//get axis for rotation
Vector3f axis = touchVector.vectorProductTo(moveVector);
//normalize axis
axis.normalize();
//get angle of rotation
float angle = direction.length;
//make identity Quad
Quaternion quad = new Quaternion();
//make rotation quad
quad.makeRotationKvaternion(angle,axis);
//from quad recieve matrix
Matrix4f matrix = quad.toMatrix();
//multiply to current modelview matrix
gl.glMultMatrixf(matrix.returnArray(),0);
//save rotation matrix
square.rotationMatrix = square.rotationMatrix.multiply(matrix);
//save modelView matrix
square.saveModelView(square.initMatrix.returnArray());
// touch coords to current modelView coords
private Vector3f getRubikSystemCoordinates(float x, float y, Matrix4f matrix){
// touch coords to normal coords of screen
Vector2f normalCoords = (new Vector2f(x,y)).toNormalScreenCoordinates(Settings.viewPort[2],Settings.viewPort[3]);
// to sphere coords in 3D
Vector3f sphereVector = new Vector3f(normalCoords.x,normalCoords.y, FloatMath.sqrt(2-normalCoords.x*normalCoords.x-normalCoords.y*normalCoords.y));
//Get inverse matrix from ModelView Matrix
Matrix4f m = matrix.inverseMatrix();
//Get vector for current modelView 3D coords
Vector3f vector = m.multiplyToVector(vector);
// make normalize vector
vector.normalize();
return vector;
}
What you're looking for is named arcball rotation. You'll find plenty of resources in java around the internet on this.
You are probably storing your rotation as three angles. Use matrix instead. Create a separate transformation matrix just for this rotation. Every time user rotates the object apply the rotation to this matrix. This way the movement will always be relative to current orientation.
i´m new to openGL and want to draw lines on an GLSurfaceView.
I switched from canvas to openGL cause of the performance.
I know that OpenGL is more likely for 3D drawings, so pixel are not used because of this.
I want to draw 2D graphics like lines but need the dimensions in pixel, only x and y values without 3rd dimension.
Set properly your projection and view matrices. For first, use orthographic projection, second leave identity.
P = Diagonal(2 / W, 2 / H, 2 / (10 + 10), 1)
// Assume Znear=-10, ZFar=10, W and H are width and Height of the screen.
So coordinates you see will bew in cube -W/2 to W/2, -H/2 to H/2 and -10 to 10. You may even translate in view matrix to move origin from -W/2, -H/2 to 0, 0 translating view matrix: V = Translate(W/2, H/2, 0).