In My OpenGL application I have created a SkyBox which works great if I render it from the origin. I render it with the PVMatrix (ProjectionMatrix * ViewMatrix [Camera]) and as I said if I render it from the origin all works great. But if I move the camera to let's say (0,6,-8) it does not work and the SkyBox is rendered as normal cube.
I thought that it is enought to create a ModelMatrix for the SkyBox and set the position in it to the position of the camera but this does not help. Of course I render the SkyBox with the new MVPMatrix now. Do you have any ideas why that does not work and what can I do to get it work?
In general translating the skybox to the camera should work out. Have you checked if you move it to the correct position?
One of the most common failures is that you move the camera in the same direction as your object, which results in moving the object twice away in the wrong direction. To explain this, let's have a look on the different coordinate systems:
Each model is defined in it's own model space. Lets call this space M. Using the model matrix (m), we are able to transform from model space to world space (W).
M ---m---> W
Now we have a second object in our scene, the camera, with its view space V and cameras model matrix c. Again we can transform
V ---c---> W
But since we need everything in view space instead of getting the camera to world space, we have to invert this transformation such that
W ---v---> V
In general this is given by v = c^-1, which is the view-matrix one specifies in the application. From knowing this, it should be quite clear why you have to move your object by z=-8 when your view-matrix contains a translation to z=8 (since T(8) = T(-8)^-1). For more details have a look at this presentation (starting from slide 6)
Related
I need to load a 3d model to my app (is not a game, not that it makes any difference) and detect when the user touches specific parts of this model, to take different actions.
How can I do this? Is it possible?
I'm not familiar with Rajawali, but GitHub describes it as an OpenGL ES framework. As you described it in the comment above, you'll need to consider two basic user actions, and one action I'll add as helpful:
Swipe across the screen in some direction: change in X, change in Y.
Touch at some (x,y) point on screen with the car in some orientation.
(Optional) Zoom in/out to make it easier for a user to select small features such as side mirrors.
Depending on what OpenGL ES details Rajawali exposes, you'll need to do one or both of the following:
Learn about the four matrices that determine how a 3D scene is rendered on a 2D screen.
Find the Rajawali functions with names such as "lookAt" or "setViewpoint," and learn how to pass screen gesture info to these functions.
You can read about the four OpenGL matrices at length elsewhere. Even if Rajawali simplifies the coding a bit you should learn a bit about those matrices. Although your first inclination is to change the "model" matrix that affects the object's position and orientation, it's more likely that you'll be manipulating the "view" matrix that determines the point and direction in space from which the user sees the car. That is, the car will actually remain centered at (0,0,0), and the user's swipes, touches, and pinches will change the viewpoint.
Constraining movements so that the vehicle is always centered is nice both because your code can be a little simpler, and also because the user can't "lose" the car by sliding the viewpoint too far to one side.
The simplest change of viewpoint is a zoom, which in most iterations means simply changing the Z translation of the viewpoint matrix. Rajawali may make this simpler by providing zoomIn() and zoomOut() functions. Otherwise you'll need to do this:
In the callback or "event handler" provided by Rajawali/Android for a pinch, get the pinch-in or pinch-out value.
Call the Rajawali zoomIn() or zoomOut() function, if it exists. You will likely need to scale the value so that the amount of pinch matches expectations for zooming in and out of a car model.
Alternately, set the Z translation component of the view matrix.
Converting an (x,y) 2D screen touch point to a ray cast into 3D space can be tricky if Rajawali doesn't provide an appropriate function called something like "screenToWorld" that accepts a point in 2D screen space and a 3D point or 3D ray in world space. Spend time googling for "ray casting" for Rajawali. Here's a brief overview of what the code will need to do:
Convert a 2D touch point into a 3D ray pointed into the screen.
Check for the intersection(s) of the 3D ray and various subobjects.
(Optional) Change the color or otherwise highlight the selected object.
OpenGL does not provide a ray casting function, and I don't recommend implementing it on your own unless you have no choice. Various frameworks that wrap around or supplement OpenGL may provide this function. OpenGL coders will fault me for this description, but from memory here's how to convert a 2D touchpoint into a 3D ray pointing into the screen:
Get the (x,y) 2D screen touch point from a "touch" or "click" callback or event handler in Rajawali or Android.
Convert the 2D touch point to a 3D point. If I remember, this means setting Z to some value such as -1, 0, or 1. This is the base point of the ray.
Define a second 3D point with a different Z value. This is a far point of the ray.
Use the screen, projection, and view matrices to transform the 3D points into "world" space.
Given the 3D world coordinates for your base point and far point, use ray-object intersection to determine what object is intersected.
Again, Rajawali may provide some function that determines which object(s) are intersected by the ray. If multiple objects are returned, then pick the closest object. Since your vehicle is already subdivided into multiple subobjects this shouldn't be too hard. Implementing pinch-to-zoom can make it easier for a user to select a small object.
Swiping is analogous to a mouse move for OpenGL, and many starter projects for OpenGL describe how to convert a mouse move to a rotation. Assuming for the moment that the model rotates only about the vertical axis from the ground through the roof, then you simply need to change left/right swipes to positive/negative rotations about what in OpenGL is typically the Y-axis.
From Android/Rajawali, handle the "swipe" event handler or callback. This is analogous to a "mouseMove" function.
Translate the left/right swipe into a negative/positive value.
Call the rotateAboutY() function, if available, OR apply a rotation to the viewpoint matrix (which I won't describe here).
Given all that, I would suggest the following approach:
See if Rajawali provides convenience functions to convert screen coordinates to a world ray, to convert a screen swipe to a rotation, and to test a ray intersection with a series of objects.
Even if Rajawali provides these functions, read a little bit about the low-level OpenGL ES underneath, and the four matrices: screen, perspective, viewpoint, and model.
If Rajawali doesn't provide the convenience functions, look for a framework that does OR see if some other library that works with Rajawali can provide these convenience functions.
If you can't change frameworks or find a framework that hides the messy details, plan to spend a week or more studying OpenGL closely. You probably don't need to know about shaders, textures, etc., but you will need to understand the OpenGL 3D space, the four matrices, and so on.
I am very new in terms of Augmented Reality with some experience on OpenGL. I wanna draw something such as a triangle with OpenGL ES(Android or iOS, doesn't matter for the platform). The background needs to be captured by mobile's camera. Basically the result likes the Pokemon Go, via the camera you got the real world as the background and a Pokemon inserts into the real world.
So, where should I begin?
One approach.
Capture the image and load into a 2D texture map.
Render a quadrilateral with this 2D texture. The quadrilateral will need to be far enough from your virtual camera so that it forms a background (you will want to enable depth testing). The quadrilateral will have to be large enough to cover the background (which depends on both the distance from the camera and your perspective field of view).
Now render your scene (triangles).
The figure below shows the view frustum in the xz-plane once you have things in view coordinates. N and F are the distances to the near and far clipping planes and θ is the vertical field of view -- let a = aspect ratio = w/h of your image (which should match the aspect ratio of your viewport). H is the height of the quad you wish to render and Q is the distance to the quad from the camera. The height of the quad should then be H = 2*Qtan(θ/2). The width of your quad is W = aH.
The distances from the camera of the objects you wish to be in the foreground should be between N and Q.
I am assuming you know how to set of the view matrix (via the "look-at transform") to position of your camera, and set the projection matrix to specify the perspective projection. Also assuming many other things (like how to load a texture map, draw a filled quad with texture coordinates, enable depth testing, etc...). If you need more details let me know.
If you wish to embed the object within the scene that will require some computer vision techniques (ascertaining depth via stereo image pairs). This is a non-trivial problem.
I am working on an AR app that needs to move an image depending on device's position and orientation.
It seems that Game Rotation Vector should provide the necessary data to achieve this.
However I cant seem to understand what the values that I get from GRV sensor show. For instance in order to reach the same value on the Z axis I have to rotate the device 720 degrees. This seems odd.
If I could somehow convert these numbers to angles from the reference frame of the device towards the x,y,z coordinates my problem would be solved.
I have googled this issue for days and didn't find any sensible information on the meaning of GRV coordinates, and how to use them.
TL:DR What do the numbers of the GRV sensor show? And how to convert them to angles?
As the docs state, the GRV sensor gives back a 3D rotation vector. This is represented as three component numbers which make this up, given by:
x axis (x * sin(θ/2))
y axis (y * sin(θ/2))
z axis (z * sin(θ/2))
This is confusing however. Each component is a rotation around that axis, so each angle (θ which is pronounced theta) is actually a different angle, which isn't clear at all.
Note also that when working with angles, especially in 3D, we generally use radians, not degrees, so theta is in radians. This looks like a good introductory explanation.
But the reason why it's given to us in the format is that it can easily be used in matrix rotations, especially as a quaternion. In fact, these are the first three components of a quaternion, the components which specify rotation. The 4th component specifies magnitude, i.e. how far away from the origin (0, 0) a point it. So a quaternion turns general rotation information into an actual point in space.
These are directly usable in OpenGL which is the Android (and the rest of the world's) 3D library of choice. Check this tutorial out for some OpenGL rotations info, this one for some general quaternion theory as applied to 3D programming in general, and this example by Google for Android which shows exactly how to use this information directly.
If you read the articles, you can see why you get it in this form and why it's called Game Rotation Vector - it's what's been used by 3D programmers for games for decades at this point.
TLDR; This example is excellent.
Edit - How to use this to show a 2D image which is rotated by this vector in 3D space.
In the example above, SensorManage.getRotationMatrixFromVector converts the Game Rotation Vector into a rotation matrix which can be applied to rotate anything in 3D. To apply this rotation a 2D image, you have to think of the image in 3D, so it's actually a segment of a plane, like a sheet of paper. So you'd map your image, which in the jargon is called a texture, onto this plane segment.
Here is a tutorial on texturing cubes in OpenGL for Android with example code and an in depth discussion. From cubes it's a short step to a plane segment - it's just one face of a cube! In fact that's a good resource for getting to grips with OpenGL on Android, I'd recommend reading the previous and subsequent tutorial steps too.
As you mentioned translation also. Look at the onDrawFrame method in the Google code example. Note that there is a translation using gl.glTranslatef and then a rotation using gl.glMultMatrixf. This is how you translate and rotate.
It matters the order in which these operations are applied. Here's a fun way to experiment with that, check out Livecodelab, a live 3D sketch coding environment which runs inside your browser. In particular this tutorial encourages reflection on the ordering of operations. Obviously the command move is a translation.
I have imported a model (e.g. a teapot) using Rajawali into my scene.
What I would like is to label parts of the model (e.g. the lid, body, foot, handle and the spout)
using plain Android views, but I have no idea how this could be achieved. Specifically, positioning
the labels on the right place seems challenging. The idea is that when I transform my model's position in the scene, the tips of the labels are still correctly positioned
Rajawali tutorial show how Android views can be placed on top of the scene here https://github.com/Rajawali/Rajawali/wiki/Tutorial-08-Adding-User-Interface-Elements
. I also understand how using the transformation matrices a 3D coordinate on the model can be
transformed into a 2D coordinate on the screen, but I have no idea how to determine the exact 3D coordinates
on the model itself. The model is exported to OBJ format using Blender, so I assume there is some clever way of determining
the coordinates in Blender and exporting them to a separate file or include them somehow in the OBJ file (but not
render those points, only include them as metadata), but I have no idea how I could do that.
Any ideas are very appreciated! :)
I would use a screenquad, not a view. This is a general GL solution, and will also work with iOS.
You must determine the indices of the desired model vertices. Using the text rendering algo below, you can just fiddle them until you hit the right ones.
Create a reasonable ARGB bitmap with same aspect ratio as the screen.
Create the screenquad texture using this bitmap
Create a canvas using this bitmap
The rest happens in onDrawFrame(). Clear the canvas using clear paint.
Use the MVP matrix to convert desired model vertices to canvas coordinates.
Draw your desired text at the canvas coordinates
Update the texture.
Your text will render very precisely at the vertices you specfied. The GL thread will double-buffer and loop you back to #4. Super smooth 3D text animation!
Use double floating point math to avoid loss of precision during coordinate conversion, which results in wobbly text. You could even use the z value of the vertex to scale the text. Fancy!
The performance bottleneck is #7 since the entire bitmap must be copied to GL texture memory, every frame. Try to keep the bitmap as small as possible, maintaining aspect ratio. Maybe let the user toggle the labels.
Note that the copy to GL texture memory is redundant since in OpenGL-ES, GL memory is just regular memory. For compatibility reasons, a redundant chunk of regular memory is reserved to artificially enforce the copy.
I have a 3D world that works well with a camera and game objects. When the camera 'renders' it positions the matrix in the location of the camera. To do this, I call the following code:
gl.glMultMatrixf(rotationArray,0);
gl.glTranslatef(position.getX(), position.getY(), position.getZ());
Now when each object gets rendered, I call this for each object. (Note that I push and pop the matrices appropriately)
gl.glTranslatef(position.getX(), position.getY(), position.getZ());
gl.glMultMatrixf(rotationArray,0);
Now my question comes into play of when I bring models into the world, I need certain Meshes to pivot not around the 0,0,0 point, but around a different point. I have an object called Mesh that has an ArrayList of submeshes. Each mesh (and submesh) have a pivot point location. (an x,y and z float).
Where do I translate the matricies so I pivot around the designated point? I've tried all sorts of combinations and nothing works!
Example:
gl.glTranslatef(position.getX(), position.getY(), position.getZ());
gl.glMultMatrixf(rotationArray,0);
gl.glTranslatef(pivotPoint.getX(),pivotPoint.getY(),pivotPoint.getZ());
and I've tried
gl.glTranslatef(pivotPoint.getX(),pivotPoint.getY(),pivotPoint.getZ());
gl.glMultMatrixf(rotationArray,0);
gl.glTranslatef(position.getX(), position.getY(), position.getZ());
This comes down to rotation around an arbitrary point. Traditionally when you rotate an object, you usually rotate it around the object's center but in this case you have a separate pivot point.
The process of rotating an object around any arbitrary point is:
Translate by -(RotationPoint)
Rotate
Translate by +(RotationPoint)