I'm trying to create an app for Android using OpenGL ES, but I'm having trouble handling touch input.
I've created a class CubeGLRenderer which spawns a Cube. CubeGLRenderer is in charge of the projection and view matrix, and Cube is in charge of its model matrix. The Cube is moving along the positive X axis, with no movement in Y nor Z.
CubeGLRenderer updates the view matrix each frame in order to move along with the cube, making the cube look stationary on screen:
Matrix.setLookAtM(mViewMatrix, 0, 0.0f, cubePos.y, -10.0f, 0.0f, cubePos.y, 0.0f, 0.0f, 1.0f, 0.0f);
The projection matrix is calculated whenever the screen dimension changes (i.e. when the orientation of the device changes). The two matrices are then muliplied and passed to Cube.draw() where it applies its model matrix and renders itself to screen.
So far, so good. Let's move on to the problem.
I want to touch the screen and calculate an angle from the center of the cube's screen coordinates to the point of the screen that I touched.
I thought I'd just accomplish this using GLU.gluProject(), but I'm either not using it correctly or simply haven't understood it at all.
Here's the code I use to calculate the screen coordinates from the cube's world coordinates:
public boolean onTouchEvent(MotionEvent e) {
Vec3 cubePos = cube.getPos();
float[] modelMatrix = cube.getModelMatrix();
float[] modelViewMatrix = new float[16];
Matrix.multiplyMM(modelViewMatrix, 0, mViewMatrix, 0, modelMatrix, 0);
int[] view = {0, 0, width, height};
float[] screenCoordinates = new float[3];
GLU.gluProject(cubePos.x, cubePos.y, cubePos.z, modelViewMatrix, 0, mProjectionMatrix, 0, view, 0, screenCoordinates, 0);
switch (e.getAction()) {
case MotionEvent.ACTION_DOWN:
Log.d("CUBEAPP", "screenX: " + String.valueOf(screenCoordinates[0]));
break;
}
return true;
}
What am I doing wrong?
The same calculation you do in the vertex shader you use to render the cube should be used to translate the cube center into the screen space.
Normally you would multiply each each vertex of the cube by the modelViewProjection matrix and then send it to the fragment shader.
You should use the exact same matrix you use in the vertex shader and multiply the center of the cube with it.
However, multiplying a 4x4 matrix with a Vec4 vertex (x, y, z, 1) would give you a Vec4 result of (x2, y2, z2, w).
In order to get the screen space coordinates you need to divide x2 and y2 by w!
After you divide it by w your xy coordinates are suppose to be within [-1..1]x[-1..1] range.
In order to get the exact pixel you would need to normalize x2/w and y2/w into [0..1] and then multiply it by the screen resolution width and height.
Hope this helps.
Related
I'm trying to create a 3D Android game using the OpenGL library. From what I can tell, the Android platform (or OpenGL) doesn't provide a camera object, so I made my own. I managed to create a square that is drawn on the screen. I also managed to get the camera to move in all 3 directions without issue. The problem I'm having is when I turn the camera either in the x or y axis, the square is displayed in a very strange way. It seems highly distorted its perspective. Here's some pictures
Camera at origin looking forward: Position [0,0,0.55f], Forward [0,0,-1], Up [0,1,0]
Camera moved to the right: Position [3,0,0.55f], Forward [0,0,-1], Up [0,1,0]
Camera turned slightly to the right: Position [0,0,0.55f], Forward [0.05f,0,-1], Up [0,1,0]
I'm not sure where the problem is getting generated. Here are the vertices of the square:
static float vertices[] = {
// Front face (CC order)
-1.0f, 1.0f, 1.0f, // top left
-1.0f, -1.0f, 1.0f, // bottom left
1.0f, -1.0f, 1.0f, // bottom right
1.0f, 1.0f, 1.0f, // top right
};
From what I've read, my matrix multiplications are in the correct order. I pass the Matrices to the vertex shader and do the multiplications there:
private final String vertexShaderCode =
"uniform mat4 uVMatrix;" +
"uniform mat4 uMMatrix;" +
"uniform mat4 uPMatrix;" +
"attribute vec4 aVertexPosition;" + // passed in
"attribute vec4 aVertexColor;" +
"varying vec4 vColor;" +
"void main() {" +
" gl_Position = uPMatrix * uVMatrix * uMMatrix * aVertexPosition;" +
" vColor = aVertexColor;" + // pass the vertex's color to the pixel shader
"}";
The Model matrix just moves it to the origin and makes it scale 1:
Matrix.setIdentityM(ModelMatrix, 0);
Matrix.translateM(ModelMatrix, 0, 0, 0, 0);
Matrix.scaleM(ModelMatrix, 0, 1, 1, 1);
I use my Camera object to update the View Matrix:
Matrix.setLookAtM(ViewMatrix, 0,
position.x, position.y, position.z,
position.x + forward.x, position.y + forward.y, position.z + forward.z,
up.x, up.y, up.z);
Here is my ProjectionMatrix:
float ratio = width / (float) height;
Matrix.frustumM(ProjectionMatrix, 0, -ratio, ratio, -1, 1, 0.01f, 10000f);
What am I missing?
With the parameters you pass to frustrumM():
Matrix.frustumM(ProjectionMatrix, 0, -ratio, ratio, -1, 1, 0.01f, 10000f);
you have an extremely strong perspective. This is the reason why the geometry looks very distorted as soon as you rotate it just slightly.
The left, right, bottom, and top values are distances measured at the depth of the near clip plane. So for example for your top value of 1.0, with the near value at 0.01, the top plane of the view volume will move at distance of 1.0 away from the viewing direction at a forward distance of 0.01.
Doing the math, you get atan(top / near) = atan(1.0 / 0.01) = atan(100.0) = 89.42 degrees for half the vertical view angle, or 178.85 degrees for the whole view angle, which corresponds to an extreme fish eye lense on a camera, covering almost the whole space in front of the camera.
To use a more sane level of perspective, you can calculate the values based on the desired view angle. With alpha being the vertical view angle:
float near = 0.01f;
float top = tan(0.5f * alpha) * near;
float right = top * ratio;
Matrix.frustumM(ProjectionMatrix, 0, -right, right, -top, top, near, 10000f);
Start with view angles in the range of 45 to 60 degrees for a generally pleasing amount of perspective. And remember that the tan() function used above takes angles in radian, so you'll have to convert it first if your original angle is in degrees.
Or, if you're scared of math, you can always use perspectiveM() instead. ;)
Depending on your setup, your vertex buffer may be wrong . Your vertices array is a vec3. And the attribute position in your shader is a vec4.
Also you projection matrix is strange , depending on the aspect ratio you want you'll get bizarre results.
You should use perspectiveM instead.
I am rendering several square objects in sphere (I am in the center of the sphere and the objects are around me). I am using the phone's Rotation sensor to view the the objects in the sphere.
all the objects start from position around (0,0,0) , but at rendering time I'm rotating each object to a different angle in the sphere .
this is how I handle the matrices until from the moment I get the rotation matrix :
public void position(float[] rotationMatrix , float[] projectionMatrix , float rotationAngle , float xRotation , float yRotation , float zRotation , float xTranslation , float yTranslation , float zTranslation){
float[] MVP = new float[16];
float[] modelMatrix = new float[16];
System.arraycopy(rotationMatrix, 0, modelMatrix, 0, rotationMatrix.length);
Matrix.rotateM(modelMatrix, 0, -90f, 1f, 0f, 0f); //correct the axis so that the direction of Y axis is to the sky and Z is to the front
Matrix.rotateM(modelMatrix, 0, rotationAngle, xRotation, yRotation, zRotation); //rotate the object around axis
// used to control the distance of viewing the object (currently only z translation is used)
Matrix.translateM(modelMatrix, 0, xTranslation, yTranslation, zTranslation);
Matrix.multiplyMM(MVP , 0 , projectionMatrix , 0 , modelMatrix , 0);
textureShaderProgram.setUniforms(MVP , texture);
return;
}
then in the shader I multiply each object location(which is the same location basically) with this MVP matrix , and they are rendered around in a sphere like world.
this works good . now what I like to do is identify when an object is right in front of me . get each object's location at all times , and when I view a certain object make it selectable or lighted.
but since every object is multiplied several times , how can I know it's location and when I am actually viewing it now ?
The translation is always stored in the last column of a transformation matrix (or in the last row depending whether the matrix is stored column major or row major).
Thus the position of an object in worldspace is the last column of the model matrix.
I am trying to understand how camera works on OpenGL ES, so I am tryng to look at the same point with the two differents types, Matrix.frustumM and Matrix.orthoM
I will like to know what exactly I am doing when use Matrix.frustumM or orthoM, I know that I apply them to the ProjectionMatrix but I dont understand what defines the parameters(left,right,bottom,top,near,far of what? it is supposed to be the screen of the phone? ) same with orthoM
I want to draw a square on the screen on 0,0,0 with 1f of height and weight(like 2D just to test the cameras)
but if I do onSurfaceCreated
final float eyeX = 2f;
final float eyeY = 5f;
final float eyeZ = 8f;
final float lookX = 2f;
final float lookY = 5f;
final float lookZ = 0.0f;
final float upX = 0.0f;
final float upY = 1.0f;
final float upZ = 0.0f;
Matrix.setLookAtM(mViewMatrix, 0, eyeX, eyeY, eyeZ, lookX, lookY, lookZ, upX, upY, upZ);
onSurfaceChanged
GLES20.glViewport(0, 0, width, height);
// Create a new perspective projection matrix. The height will stay the
// same
// while the width will vary as per aspect ratio.
final float ratio = (float) width / height;
final float left = -ratio;
final float right = ratio;
final float bottom = -1.0f;
final float top = 1.0f;
final float near = 1.0f;
final float far = 25.0f;
Matrix.frustumM(mProjectionMatrix, 0, left, right, bottom, top, near, far);
That is what i saw onn phone
Draw function:
public void dibujarBackground()
{
// Draw a plane
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mBackgroundDataHandle);
Matrix.setIdentityM(mModelMatrix, 0);
Matrix.translateM(mModelMatrix, 0, 0.0f,2.0f, 0.0f);
drawBackground();
}
private void drawBackground()
{
coordinate.drawBackground(mPositionHandle, mNormalHandle, mTextureCoordinateHandle);
// This multiplies the view matrix by the model matrix, and stores the
// result in the MVP matrix
// (which currently contains model * view).
Matrix.multiplyMM(mMVPMatrix, 0, mViewMatrix, 0, mModelMatrix, 0);
GLES20.glUniformMatrix4fv(mMVMatrixHandle, 1, false, mMVPMatrix, 0);
Matrix.multiplyMM(mMVPMatrix, 0, mProjectionMatrix, 0, mMVPMatrix, 0);
GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, mMVPMatrix, 0);
GLES20.glUniform3f(mLightPosHandle,Light.mLightPosInEyeSpace[0], Light.mLightPosInEyeSpace[1], Light.mLightPosInEyeSpace[2]);
GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, 6);
}
Coords of the square:
final float[] backgroundPositionData = {
// In OpenGL counter-clockwise winding is default.
0f, 1f, 0.0f,
0f, 0f, 0.0f,
1f, 1f, 0.0f,
0f, 0f, 0.0f,
1f, 0f, 0.0f,
1f, 1f, 0.0f,
};
final float[] backgroundNormalData = {
0.0f, 0.0f, 1.0f,
0.0f, 0.0f, 1.0f,
0.0f, 0.0f, 1.0f,
0.0f, 0.0f, 1.0f,
0.0f, 0.0f, 1.0f,
0.0f, 0.0f, 1.0f, };
final float[] backgroundTextureCoordinateData = {
0.0f, 0.0f,
0.0f, 1.0f,
1.0f, 0.0f,
0.0f, 1.0f,
1.0f, 1.0f,
1.0f, 0.0f, };
Overall what you get in the end is a single matrix which is used to multiply the positions so that the visible fragments are in range [-1,1] in all 3 dimensions. That means if you use no matrix or use the identity the coordinates will need to be in this range to be visible. So the 3 matrix computations you are using are actually only conveniences to help you achieve a correct transformation:
Ortho is an orthographical transformation. This means the visual representation of x and y screen coordinates are not effected by the z coordinate at all. Visually that means the object does not appear smaller when it is further. The values you insert into this convenience method are border values (left, right, top, bottom) which means a rectangle with same coordinates will take exactly the full screen. These values are mostly used to be the same as your view coordinate system (left = 0, right = screenWidth, top = 0, bottom = screenHeight). Also there are near and far parameters which represent the clipping planes so that positions smaller then near or further then far are not visible. This projection is mostly used for 2D drawing.
Frustum matrix is designed so that the x and y coordinates are reduced with increasing z. This means an object will appear smaller when further. The border parameters are connected to the near parameter so that the rectangle with border coordinates having z at near will appear as full screen. The near must be larger then zero in this case or the result is unpredictable. The far promoter is just a clipping plane but same as with ortho the pixels are clipped if z value is smaller then near or larger then far. The border parameters are best computed with the field of view (angle) and screen aspect ratio. You use the tang function to compute border parameters to get the desired effect. This method is mostly used for 3D drawing.
LookAt is a convenience which is used to transform all the objects to such positions and orientations that they appear to be effected by the camera position. Though this method is defined with vectors you may imagine it having a vector position and rotations. What this does it creates a matrix that will rotate all the objects by -rotations and translate them by -position.
Overall the usage then is pretty simple. Each position should first be multiplied by the model matrix which is the matrix representing the model position in your scene. Then multiplied by the matrix received with lookAt to simulate the camera. Then multiplied by the projection matrix which in most cases is either the ortho or the frustum. The optimization then is to multiply the matrices first on the CPU and then have the positions multiplied by them on the GPU. Some variations then persist where you split the matrix to the "model view matrix" and the "projection matrix". This is used to compute things like lighting effect where the position must not be effected by the projection matrix.
As a beginner to android and openGL 2.0 es, I'm testing simple things and see how it goes.
I downloaded the sample at http://developer.android.com/training/graphics/opengl/touch.html .
I changed the code to check if I could animate a rotation of the camera around the (0,0,0) point, the center of the square.
So i did this:
public void onDrawFrame(GL10 unused) {
// Draw background color
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
// Set the camera position (View matrix)
long time = SystemClock.uptimeMillis() % 4000L;
float angle = ((float) (2*Math.PI)/ (float) 4000) * ((int) time);
Matrix.setLookAtM(mVMatrix, 0, (float) (3*Math.sin(angle)), 0, (float) (3.0f*Math.cos(angle)), 0 ,0, 0, 0f, 1.0f, 0.0f);
// Calculate the projection and view transformation
Matrix.multiplyMM(mMVPMatrix, 0, mProjMatrix, 0, mVMatrix, 0);
// Draw square
mSquare.draw(mMVPMatrix);
}
I expected the camera to look always to the center of the square (the (0,0,0) point) but that's not what happens. The camera is indeed rotating around the square but the square does not stay in the center of the screen.. instead it is moving along the X axis...:
I also expected that if we gave the eyeX and eyeY the same values as centerX and centerY,like this:
Matrix.setLookAtM(mVMatrix, 0, 1, 1, -3, 1 ,1, 0, 0f, 1.0f, 0.0f);
the square would keep it's shape (I mean, your field of vision would be dragged but along a plane which would be paralel to the square), but that's also not what happens:
This is my projection matrix:
float ratio = (float) width / height;
// this projection matrix is applied to object coordinates
// in the onDrawFrame() method
Matrix.frustumM(mProjMatrix, 0, -ratio, ratio, -1, 1, 2, 7);
What is going on here?
Looking at the source code to the example you downloaded, I can see why you're having that problem, it has to do with the order of the matrix multiplication.
Typically in OpenGL source you see matrices set up such that
transformed vertex = projMatrix * viewMatrix * modelMatrix * input vertex
However in the source example program that you downloaded, their shader is setup like this:
" gl_Position = vPosition * uMVPMatrix;"
With the position on the other side of the matrix. You can work with OpenGL in this way, but it requires that you reverse the lhs/rhs of your matrix multiplications.
Long story short, in your case, you should change your shader to read:
" gl_Position = uMVPMatrix * vPosition;"
and then I believe you will get the expected behavior.
I'm trying to migrate graphics in my game to OpenGL for performance reasons.
I need to draw an object using exact screen coordinates. Say a box 100x100 pixels in the center of 240x320 screen.
I need to rotate it around Z axis, preserving its size.
I need to rotate it around X axis, with perspective effect, preserving (or close to) its size.
I need to rotate it around Y axis, with perspective effect, preserving (or close to) its size.
Here's a picture.
So far I managed to achieve first 2 tasks:
public void onDrawFrame(GL10 gl) {
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
gl.glLoadIdentity();
gl.glTranslatef(120, 160, 0); // move rotation point
gl.glRotatef(angle, 0.0f, 0.0f, 1.0f); // rotate
gl.glTranslatef(-120, -160, 0); // restore rotation point
mesh.draw(gl); // draws 100x100 px rectangle with the following coordinates: (70, 110, 170, 210)
}
public void onSurfaceChanged(GL10 gl, int width, int height) {
gl.glViewport(0, 0, width, height);
gl.glMatrixMode(GL10.GL_PROJECTION);
gl.glLoadIdentity();
gl.glOrthof(0f, (float)width, (float)height, 0f, -1f, 1f);
gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadIdentity();
}
But when I'm trying to rotate my box around x or y, nasty thing are happening with my box and there is no perspective effect. I tried to use some other function instead of glRotate (glFrustum, glPerspective, gluLookAt, applying "skewing" matrix), but I couldn't make them work properly.
I'm trying to migrate graphics in my game to OpenGL for performance reasons.
I need to draw an object using exact screen coordinates. Say a box 100x100 pixels in the center of 240x320 screen.
For a perspective you also need some length for the lens, which determines the FOV. The FOV is the ratio of viewing plane distance to visible extents. In the case of the near plane it thus becomes {left,right,top,bottom}/near. For the sake of simplicity we assume horizontal FOV and a symmetric projection i.e.
FOV = 2*|left|/near = 2*|right|/near = extent/distance
or if you're more into angles
FOV = 2*tan(angular FOV / 2)
For a 90° FOV the length of the lens is half the width of the focal plane. Your focal plane is 240x320 pixels, so 120 to the left and right and 160 to the top and bottom. OpenGL does not really have a focus, but we can say that the middle plane between near and far is the "focal".
So let's say the object will have in average a extent of about the order of magnitude of visible plane limits, i.e. for a visible plane of 240x360, an object will have in average a size of ~200px. It thus makes sense the distance of near to far clipping to be 200, so +- 100 about the focal plane. So for a FOV of 90° the focal plane has distance
2*tan(90°/2) = extent/distance
2*tan(45°) = 2 = 240/distance
2*distance = 240
distance = 120
120, thus near and far clipping distances are 20 and 220.
Last but not least the near clip plane limits must be scaled by near_distance/focal_distance = 20/120
So
left = -120 * 20/120 = -20
right = 120 * 20/120 = 20
bottom = -180 * 20/120 = -30
top = 180 * 20/120 = 30
So this gives us the glFrustum parameters:
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glFrustum(-20, 20, -30, 30, 20, 220);
And last but not least we must move the world origin into the "focal" plane
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glTranslatef(0, 0, -120);
I need to rotate it around Z axis, preserving its size.
done.
I need to rotate it around X axis, with perspective effect, preserving (or close to) its size.
I need to rotate it around Y axis, with perspective effect, preserving (or close to) its size.
Perspective does not preserve size. That's what's makes it a perspective. You can use a very long lens, i.e. small FOV.
Code Update
As a general pro-tip: Do all OpenGL operations in the drawing handler. Don't set the projection in the reshape handler. It's ugly and as soon as you want to have some HUD or other kind of overlay you'll have to discard it anyway. So here's how to change it:
public void onDrawFrame(GL10 gl) {
// fov, extents are parameters set somewhere else
// 2*tan(fov/2.) = width/distance =>
float distance = width/(2.*tan(fov));
float near = distance - extent/2;
float far = distance + extent/2;
if(near < 1.) {
near = 1.;
}
float left = (-width/2) * near/distance;
float right = ( width/2) * near/distance;
float bottom = (-height/2) * near/distance;
float top = ( height/2) * near/distance;
gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);
gl.glViewport(0, 0, width, height);
gl.glMatrixMode(GL10.GL_PROJECTION);
gl.glLoadIdentity();
gl.glFrustum(left, right, bottom, top, near, far);
gl.glMatrixMode(GL10.GL_MODELVIEW);
gl.glLoadIdentity();
gl.glTranslatef(0, 0, -focal);
gl.glTranslatef(120, 160, 0); // move rotation point
gl.glRotatef(angle, 0.0f, 0.0f, 1.0f); // rotate
gl.glTranslatef(-120, -160, 0); // restore rotation point
mesh.draw(gl); // draws 100x100 px rectangle with the following coordinates: (70, 110, 170, 210)
}
public void onSurfaceChanged(GL10 gl, int new_width, int new_height) {
width = new_width;
height = new_height;
}
You need to use a perspective projection matrix and then use your model-view matrix to get the position and scaling right.
You're using an orthogonal projection (glOrthof()), which explicitly disables perspective.
It's opposite is glFrustum(), often wrapped by gluPerspective()/ which is easier to use but requires the GLU library.