I'm trying to implement color picking using GLES20.glReadPixels function in android OpenGL ES. The problem is that this function is always returning 0,0,0,0 as color and not the correct color values. Any idea why? My code looks like this:
public boolean onTouchEvent(MotionEvent event)
{
if (event != null)
{
float x = event.getX();
float y = event.getY();
if (event.getAction() == MotionEvent.ACTION_UP)
{
int newX = (int)x;
int newY = (int)y;
ByteBuffer pixel = ByteBuffer.allocate(4);
pixel.order(ByteOrder.nativeOrder());
pixel.position(0);
GLES20.glReadPixels(newX, (int)mRenderer.viewport[3] - newY, 1, 1,
GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE, pixel);
}
return true;
}
else
{
return super.onTouchEvent(event);
}
So as I said before... the result of the pixel array is always 0,0,0,0. Dont know why :/ What am I doing wrong? I was using the lighthouse tutorial as a reference:
http://www.lighthouse3d.com/opengl/picking/index.php3?color2
And I really can't see the mistake at this point :/
Oh I forgot to tell that my scene contains a 3D cube which is fully BLUE so the result should be something like 0,0,1,0 when I click on it but it isn't :(
EDIT:
The code from the Renderer where the Cube is drawn (it rotates arround its y-axis)
public void onDrawFrame(GL10 unused) {
float[] scratch = new float[16];
GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
Matrix.setLookAtM(mViewMatrix, 0, 0, 0, -3.0f, 0f, -0.3f, 0.0f, 0.0f, 1.0f, 0.0f);
Matrix.multiplyMM(mMVPMatrix, 0, mProjectionMatrix, 0, mViewMatrix, 0);
float[] mModelMatrix = new float[16];
Matrix.setIdentityM(mModelMatrix, 0);
Matrix.translateM(mModelMatrix, 0, 0, 0, -0.5f);
Matrix.setIdentityM(mRotationMatrix, 0);
Matrix.rotateM(mRotationMatrix, 0, mDeltaX, 0, 1.0f, 0);
Matrix.rotateM(mRotationMatrix, 0, -mDeltaY, 1.0f, 0, 0);
mDeltaX = 0.2f;
mDeltaY = 0.2f;
float[] mTempMatrix = new float[16];
Matrix.multiplyMM(mTempMatrix, 0, mRotationMatrix, 0, mAccumulatedRotation, 0);
System.arraycopy(mTempMatrix, 0, mAccumulatedRotation, 0, 16);
float[] temp = new float[16];
Matrix.multiplyMM(temp, 0, mModelMatrix, 0 , mAccumulatedRotation, 0);
Matrix.multiplyMM(scratch, 0, mMVPMatrix, 0, temp, 0);
Matrix.setIdentityM(mModelMatrix, 0);
Matrix.translateM(mModelMatrix, 0, 0, 0, 0.5f);
float[] temp2 = new float[16];
Matrix.multiplyMM(temp2, 0, scratch, 0, mModelMatrix, 0);
mCube.drawCube(temp2);
}
Like all other OpenGL calls, glReadPixels() only works if there is a current OpenGL context.
In Android, OpenGL rendering is mostly done using a GLSurfaceView, which takes care of spawning a secondary thread for rendering, creating an OpenGL context, and making that context current in the secondary rendering thread while invoking the methods in your GLSurfaceView.Renderer implementation.
onTouchEvent() is invoked in the UI thread, so you won't have a current OpenGL context here. To use glReadPixels(), you can forward the request to your rendering thread using the GLSurfaceView.queueEvent() method, and then process it asynchronously the next time your Renderer.onDraw() method is invoked.
Related
Hi I am working on a AR android app. I am using ARToolkit6. In this app I want to view my 3D object( A Cube) on left half of the screen. With this eventually I want to display 3 cubes on the screen each on 1/3 of the screen area.
I was able to scale the 3D object by tweaking ModelView Matrix. What I read so far, I think I need to tweak projection matrix to achieve my goal. I tried looking solutions online. But Couldn't get it to work. Can anyone direct me to right path?
for (int trackableUID : trackableUIDs) {
// If the marker is visible, apply its transformation, and render a cube
if (ARToolKit.getInstance().queryMarkerVisible(trackableUID)) {
float[] projectionMatrix = ARToolKit.getInstance().getProjectionMatrix();
float[] modelViewMatrix = ARToolKit.getInstance().queryMarkerTransformation(trackableUID);
float[] scalingMat = {1, 0, 0, 0, 0, 3.0f, 0, 0, 0, 0, 1.0f, 0, 0.0f, 0, 0, 1};
float[] newModelView = modelViewMatrix;
multiplyMM(newModelView, 0, modelViewMatrix, 0, scalingMat, 0);
cube.draw(projectionMatrix, newModelView);
}
I followed the this link Set origin to top-left corner of screen in OpenGL ES 2 and (OpenGL ES) Objects away from view centre are stretched. So I translated the modelView Matrix but it doesn't solve the problem, the 3D object appears at the center of the screen. Can you explain how should I approach this problem? Thanks
#Override
public void draw() {
super.draw();
GLES20.glEnable(GLES20.GL_CULL_FACE);
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
GLES20.glFrontFace(GLES20.GL_CCW);
// Look for trackables, and draw on each found one.
for (int trackableUID : trackableUIDs) {
// If the marker is visible, apply its transformation, and render a cube
if (ARToolKit.getInstance().queryMarkerVisible(trackableUID)) {
float[] projectionMatrix = ARToolKit.getInstance().getProjectionMatrix();
float[] modelViewMatrix = ARToolKit.getInstance().queryMarkerTransformation(trackableUID);
float[] scalingMat = {1, 0, 0, 0, 0, 3.0f, 0, 0, 0, 0, 1.0f, 0, 0.0f, 0, 0, 1};
multiplyMM(modelViewMatrix, 0, scalingMat, 0, modelViewMatrix, 0);
float[] rightModelMatrix = new float[16];
Matrix.setIdentityM(rightModelMatrix, 0);
// Translate outer sphere by 5 in x.
Matrix.translateM(rightModelMatrix, 0, 5.0f, 0.0f, 0.0f);
Matrix.multiplyMM(modelViewMatrix, 0, rightModelMatrix, 0, modelViewMatrix, 0);
cube.draw(projectionMatrix, modelViewMatrix);
}
}
Also tried this but the object gets displayed at the center of the screen.
glMatrixMode(GL_PROJECTION);
glTranslatef(5f, 0f, 0f);
I am having some problems with proper scene manipulations with OpenGL ES 2.0. On my screen I want to draw a rectangle and a cube. Rectangle should be moved to the bottom and scaled while cube should be moved to the top and scaled. What's more, I want to rotate this cube in all 3 axis.
This is what I've created so far:
#Override
public void onDrawFrame(GL10 gl) {
float[] scratch = new float[16];
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
Matrix.setLookAtM(viewMatrix, 0, 0, 0, -6, 0f, 0f, 0f, 0f, 1.0f, 0.0f);
Matrix.multiplyMM(MVPMatrix, 0, projectionMatrix, 0, viewMatrix, 0);
Matrix.translateM(scratchMatrix1, 0, 0, -1.1f, 0);
Matrix.scaleM(scratchMatrix3, 0, 0.5f, 0.5f, 0.5f);
Matrix.multiplyMM(scratchMatrix2, 0, scratchMatrix1, 0, scratchMatrix3, 0);
Matrix.multiplyMM(scratch, 0, scratchMatrix2, 0, MVPMatrix, 0);
rectangle.draw(scratch);
Matrix.setRotateM(scratchMatrix1, 0, angleXVal, 1.0f, 0, 0);
Matrix.setRotateM(rotationMatrix, 0, angleYVal, 0, 1.0f, 0);
Matrix.multiplyMM(scratchMatrix2, 0, rotationMatrix, 0, scratchMatrix1, 0);
Matrix.setRotateM(rotationMatrix, 0, angleZVal, 0, 0, 1.0f);
Matrix.multiplyMM(scratchMatrix3, 0, rotationMatrix, 0, scratchMatrix2, 0);
Matrix.translateM(scratchMatrix1, 0, 0, 0.3f, 0);
Matrix.multiplyMM(scratchMatrix2, 0, scratchMatrix3, 0, scratchMatrix1, 0);
Matrix.scaleM(scratchMatrix1, 0, 0.8f, 0.8f, 0.8f);
Matrix.multiplyMM(scratchMatrix3, 0, scratchMatrix2, 0, scratchMatrix1, 0);
Matrix.multiplyMM(scratch, 0, scratchMatrix2, 0, MVPMatrix, 0);
cube.draw(scratch);
}
The problem is that when I am rotating the cube also the rectangle is moving. It is this same with scale and translating of cube - it is pulling the rectangle.
For me it looks like the matrices are connected somewhere but I don't see where, because the rectangle is drawn first and scratch matrix is overwritten later.
Assuming that is your complete onDrawFrame() code, scratchMatrix1 etc are class member variables that persist from frame to frame.
That means that translations from one frame will carry over to the next, because Matrix.translateM() applies a translation to an existing matrix transform by adding to it, rather than setting the matrix to a given translation matrix and overwriting the previous values. So the translations will accumulate over several frames.
That means that when you translate the cube, the translation will still be sitting in scratchMatrix1 the next time onDrawFrame is called, and the rectangle translation will be added to it rather than be set afresh.
The same goes for Matrix.scaleM().
You can rectify this by initializing all the scratch matrices to identity at the start of the function, and again before beginning the calculation of the cube transform matrix:
Matrix.setIdentityM(scratchMatrix1, 0);
Matrix.setIdentityM(scratchMatrix2, 0);
Matrix.setIdentityM(scratchMatrix3, 0);
Matrix.setIdentityM(scratch, 0);
I want to rotate an element around a specific point defined by me and dynamically changed.
I am orientating myself at the guidelines from the google developers site.
My first approach is this:
scratch = new float[16];
Matrix.setIdentityM(mRotationMatrix, 0);
Matrix.setRotateM(mRotationMatrix, 0, angle, 0, 0, 1f);
Matrix.multiplyMM(scratch, 0, mMVPMatrix, 0, mRotationMatrix, 0);
element.draw(scratch);
This rotates the object around the center of the screen.
What do I have to add/change to make the object rotate around some other point?
Add a translation operation.
Grafika's Sprite2d class provides an example:
/**
* Re-computes mModelViewMatrix, based on the current values for rotation, scale, and
* translation.
*/
private void recomputeMatrix() {
float[] modelView = mModelViewMatrix;
Matrix.setIdentityM(modelView, 0);
Matrix.translateM(modelView, 0, mPosX, mPosY, 0.0f);
if (mAngle != 0.0f) {
Matrix.rotateM(modelView, 0, mAngle, 0.0f, 0.0f, 1.0f);
}
Matrix.scaleM(modelView, 0, mScaleX, mScaleY, 1.0f);
mMatrixReady = true;
}
This positions the object, then rotates it around the center of the object.
You need to translate the matrix in the reverse direction of the point first, then rotate and then translate it back. Look at it as if the rotation is always rotating around the center of the world, and the translation moves the center of the world.
Something like this (untested):
scratch = new float[16];
Matrix.setIdentityM(mRotationMatrix, 0);
Matrix.translateM(mRotationMatrix, 0, -x, -y, -z);
Matrix.rotateM(mRotationMatrix, 0, angle, 0, 0, 1f);
Matrix.translateM(mRotationMatrix, 0, x, y, z);
Matrix.multiplyMM(scratch, 0, mMVPMatrix, 0, mRotationMatrix, 0);
element.draw(scratch);
The x, y and z values need to be calculated as the delta between the current position of the object in the world and the position of the point you want to rotate around. You need to do that calculation yourself, but that's pretty trivial.
I have, a problem with the setLookAtM function. My goal is to create a cube within a cube something like this (yep, it's paint :P ):
So basically everything works... almoust... I have the smaller cube and I have the bigger one.
However, there is a problem. I created the bigger one with coords from -1 to 1 and now I want to upscale it. With scale 1.0f i have something like this (the inner cube is rotating):
And thats good, but now... when I try to scale the bigger cube (so that it looks like in the paint drawing) the image goes black or white (i guess it's because the "camera" looks at the white cube but still i dont know why does my inner cube disappear :/ I don't understand what I'm doing wrong. Here is my code:
public void onDrawFrame(GL10 unused) {
float[] scratch = new float[16];
GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
Matrix.setLookAtM(mViewMatrix, 0, 0, 0, -5.0f, 0f, 0f, -1.0f, 0f, 1.0f, 0.0f);
Matrix.multiplyMM(mMVPMatrix, 0, mProjectionMatrix, 0, mViewMatrix, 0);
mRoom.mScale = 1.0f;
Matrix.setIdentityM(mScaleMatrix, 0);
Matrix.scaleM(mScaleMatrix, 0, mRoom.mScale, mRoom.mScale, mRoom.mScale);
float[] scaleTempMatrix = new float[16];
Matrix.multiplyMM(scaleTempMatrix, 0, mMVPMatrix, 0, mScaleMatrix, 0);
mRoom.draw(scaleTempMatrix);
When I set for example:
mRoom.mScale = 3.0f;
And
Matrix.setLookAtM(mViewMatrix, 0, 0, 0, -2.0f, 0f, 0f, 0.0f, 1.0f, 1.0f, 0.0f);
My camera should be at (0, 0, -2) looking at (0,0, -1) and it should be inside the white cube (since scale is 3.0 so the coords should be from -3 to 3 right?) But all I get is a white screen without the smaller cube rotating inside :/
If your scale is 3x in this code, then your visible coordinate range is actually going to be [-1/3,1/3].
You are thinking about things backwards, it might help if you considered the order in which the scale operation is applied. Right now you are scaling the object-space coordinates, then applying the view matrix and then projection. It may not look that way, but that is how matrix multiplication in GL works; GL effectively flips the operands when it does matrix multiplication and matrix multiplication is not commutative.
I believe this is what you actually want:
public void onDrawFrame(GL10 unused) {
float[] scratch = new float[16];
GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
Matrix.setLookAtM(mViewMatrix, 0, 0, 0, -5.0f, 0f, 0f, -1.0f, 0f, 1.0f, 0.0f);
mRoom.mScale = 3.0f;
Matrix.setIdentityM(mScaleMatrix, 0);
Matrix.scaleM(mScaleMatrix, 0, mRoom.mScale, mRoom.mScale, mRoom.mScale);
Matrix.multiplyMM(mMVPMatrix, 0, mScaleMatrix, 0, mProjectionMatrix, 0);
Matrix.multiplyMM(mMVPMatrix, 0, mMVPMatrix, 0, mViewMatrix, 0);
mRoom.draw(mMVPMatrix);
I'm trying to rotate moving object, but it rotates aroud the center of the coordinates system. How to make it rotate around itself while moving? The code is:
Matrix.translateM(mMMatrix, 0, 0, -y, 0);
Matrix.setRotateM(mMMatrix, 0, mAngle, 0, 0, 1.0f);
y += speed;
Matrix.translateM(mMMatrix, 0, 0, y, 0);
Don`t use the view matrix to rotate objects, this matrix is used as the camera for all the scene, To transform an object you should use the model matrix. To rotate if around its own center, you can use the following method:
public void transform(float[] mModelMatrix) {
Matrix.setIdentityM(mModelMatrix, 0);
Matrix.translateM(mModelMatrix, 0, 0, y, 0);
Matrix.rotateM(mModelMatrix, 0, mAngle, 0.0f, 0.0f, 1.0f);
}
Don`t forget use the identity matrix to reset the transformations in every loop.
I think your code is worng. You shoud update the value of 'y' before to apply any transformation.
public void onDrawFrame(GL10 gl) {
...
y += speed;
transform(mModelMatrix);
updateMVP(mModelMatrix, mViewMatrix, mProjectionMatrix, mMVPMatrix);
renderObject(mMVPMatrix);
...
}
The updateMVP method, will combine the model, view and projection matrices:
private void updateMVP(
float[] mModelMatrix,
float[] mViewMatrix,
float[] mProjectionMatrix,
float[] mMVPMatrix) {
// combine the model with the view matrix
Matrix.multiplyMM(mMVPMatrix, 0, mViewMatrix, 0, mModelMatrix, 0);
// combine the model-view with the projection matrix
Matrix.multiplyMM(mMVPMatrix, 0, mProjectionMatrix, 0, mMVPMatrix, 0);
}
And at last the render method, will execute the Shaders to paint the object:
public void renderObject(float[] mMVPMatrix) {
GLES20.glUseProgram(mProgram);
...
// Pass the MVP data into the shader
GLES20.glUniformMatrix4fv(mMVPMatrixHandle, 1, false, mMVPMatrix, 0);
// Draw the shape
GLES20.glDrawElements (...);
}
I hope this will help you.
where do you make the object drawing?
I suppose it is after the code you have put up here, something like:
Matrix.translateM(mMMatrix, 0, 0, -y, 0);
Matrix.setRotateM(mMMatrix, 0, mAngle, 0, 0, 1.0f);
y += speed;
Matrix.translateM(mMMatrix, 0, 0, y, 0);
drawHere();//<<<<<<<<<<<<<<<<<<<
Then, the second translate call is the issue.
You should either move your draw call before the second translate.
or
the clean way to do it is:
Matrix.setIdentityM(mMMatrix, 0);//<<<<<<<<added
Matrix.translateM(mMMatrix, 0, 0, -y, 0);
Matrix.setRotateM(mMMatrix, 0, mAngle, 0, 0, 1.0f);
y += speed;
//Matrix.translateM(mMMatrix, 0, 0, y, 0); //<<<<<<<<<removed
drawHere();
I just used view matrix instead of model matrix and everything worked out. For details on model, view and projection matrices see.