I need to load a 3d model to my app (is not a game, not that it makes any difference) and detect when the user touches specific parts of this model, to take different actions.
How can I do this? Is it possible?
I'm not familiar with Rajawali, but GitHub describes it as an OpenGL ES framework. As you described it in the comment above, you'll need to consider two basic user actions, and one action I'll add as helpful:
Swipe across the screen in some direction: change in X, change in Y.
Touch at some (x,y) point on screen with the car in some orientation.
(Optional) Zoom in/out to make it easier for a user to select small features such as side mirrors.
Depending on what OpenGL ES details Rajawali exposes, you'll need to do one or both of the following:
Learn about the four matrices that determine how a 3D scene is rendered on a 2D screen.
Find the Rajawali functions with names such as "lookAt" or "setViewpoint," and learn how to pass screen gesture info to these functions.
You can read about the four OpenGL matrices at length elsewhere. Even if Rajawali simplifies the coding a bit you should learn a bit about those matrices. Although your first inclination is to change the "model" matrix that affects the object's position and orientation, it's more likely that you'll be manipulating the "view" matrix that determines the point and direction in space from which the user sees the car. That is, the car will actually remain centered at (0,0,0), and the user's swipes, touches, and pinches will change the viewpoint.
Constraining movements so that the vehicle is always centered is nice both because your code can be a little simpler, and also because the user can't "lose" the car by sliding the viewpoint too far to one side.
The simplest change of viewpoint is a zoom, which in most iterations means simply changing the Z translation of the viewpoint matrix. Rajawali may make this simpler by providing zoomIn() and zoomOut() functions. Otherwise you'll need to do this:
In the callback or "event handler" provided by Rajawali/Android for a pinch, get the pinch-in or pinch-out value.
Call the Rajawali zoomIn() or zoomOut() function, if it exists. You will likely need to scale the value so that the amount of pinch matches expectations for zooming in and out of a car model.
Alternately, set the Z translation component of the view matrix.
Converting an (x,y) 2D screen touch point to a ray cast into 3D space can be tricky if Rajawali doesn't provide an appropriate function called something like "screenToWorld" that accepts a point in 2D screen space and a 3D point or 3D ray in world space. Spend time googling for "ray casting" for Rajawali. Here's a brief overview of what the code will need to do:
Convert a 2D touch point into a 3D ray pointed into the screen.
Check for the intersection(s) of the 3D ray and various subobjects.
(Optional) Change the color or otherwise highlight the selected object.
OpenGL does not provide a ray casting function, and I don't recommend implementing it on your own unless you have no choice. Various frameworks that wrap around or supplement OpenGL may provide this function. OpenGL coders will fault me for this description, but from memory here's how to convert a 2D touchpoint into a 3D ray pointing into the screen:
Get the (x,y) 2D screen touch point from a "touch" or "click" callback or event handler in Rajawali or Android.
Convert the 2D touch point to a 3D point. If I remember, this means setting Z to some value such as -1, 0, or 1. This is the base point of the ray.
Define a second 3D point with a different Z value. This is a far point of the ray.
Use the screen, projection, and view matrices to transform the 3D points into "world" space.
Given the 3D world coordinates for your base point and far point, use ray-object intersection to determine what object is intersected.
Again, Rajawali may provide some function that determines which object(s) are intersected by the ray. If multiple objects are returned, then pick the closest object. Since your vehicle is already subdivided into multiple subobjects this shouldn't be too hard. Implementing pinch-to-zoom can make it easier for a user to select a small object.
Swiping is analogous to a mouse move for OpenGL, and many starter projects for OpenGL describe how to convert a mouse move to a rotation. Assuming for the moment that the model rotates only about the vertical axis from the ground through the roof, then you simply need to change left/right swipes to positive/negative rotations about what in OpenGL is typically the Y-axis.
From Android/Rajawali, handle the "swipe" event handler or callback. This is analogous to a "mouseMove" function.
Translate the left/right swipe into a negative/positive value.
Call the rotateAboutY() function, if available, OR apply a rotation to the viewpoint matrix (which I won't describe here).
Given all that, I would suggest the following approach:
See if Rajawali provides convenience functions to convert screen coordinates to a world ray, to convert a screen swipe to a rotation, and to test a ray intersection with a series of objects.
Even if Rajawali provides these functions, read a little bit about the low-level OpenGL ES underneath, and the four matrices: screen, perspective, viewpoint, and model.
If Rajawali doesn't provide the convenience functions, look for a framework that does OR see if some other library that works with Rajawali can provide these convenience functions.
If you can't change frameworks or find a framework that hides the messy details, plan to spend a week or more studying OpenGL closely. You probably don't need to know about shaders, textures, etc., but you will need to understand the OpenGL 3D space, the four matrices, and so on.
Related
I am working on an AR app that needs to move an image depending on device's position and orientation.
It seems that Game Rotation Vector should provide the necessary data to achieve this.
However I cant seem to understand what the values that I get from GRV sensor show. For instance in order to reach the same value on the Z axis I have to rotate the device 720 degrees. This seems odd.
If I could somehow convert these numbers to angles from the reference frame of the device towards the x,y,z coordinates my problem would be solved.
I have googled this issue for days and didn't find any sensible information on the meaning of GRV coordinates, and how to use them.
TL:DR What do the numbers of the GRV sensor show? And how to convert them to angles?
As the docs state, the GRV sensor gives back a 3D rotation vector. This is represented as three component numbers which make this up, given by:
x axis (x * sin(θ/2))
y axis (y * sin(θ/2))
z axis (z * sin(θ/2))
This is confusing however. Each component is a rotation around that axis, so each angle (θ which is pronounced theta) is actually a different angle, which isn't clear at all.
Note also that when working with angles, especially in 3D, we generally use radians, not degrees, so theta is in radians. This looks like a good introductory explanation.
But the reason why it's given to us in the format is that it can easily be used in matrix rotations, especially as a quaternion. In fact, these are the first three components of a quaternion, the components which specify rotation. The 4th component specifies magnitude, i.e. how far away from the origin (0, 0) a point it. So a quaternion turns general rotation information into an actual point in space.
These are directly usable in OpenGL which is the Android (and the rest of the world's) 3D library of choice. Check this tutorial out for some OpenGL rotations info, this one for some general quaternion theory as applied to 3D programming in general, and this example by Google for Android which shows exactly how to use this information directly.
If you read the articles, you can see why you get it in this form and why it's called Game Rotation Vector - it's what's been used by 3D programmers for games for decades at this point.
TLDR; This example is excellent.
Edit - How to use this to show a 2D image which is rotated by this vector in 3D space.
In the example above, SensorManage.getRotationMatrixFromVector converts the Game Rotation Vector into a rotation matrix which can be applied to rotate anything in 3D. To apply this rotation a 2D image, you have to think of the image in 3D, so it's actually a segment of a plane, like a sheet of paper. So you'd map your image, which in the jargon is called a texture, onto this plane segment.
Here is a tutorial on texturing cubes in OpenGL for Android with example code and an in depth discussion. From cubes it's a short step to a plane segment - it's just one face of a cube! In fact that's a good resource for getting to grips with OpenGL on Android, I'd recommend reading the previous and subsequent tutorial steps too.
As you mentioned translation also. Look at the onDrawFrame method in the Google code example. Note that there is a translation using gl.glTranslatef and then a rotation using gl.glMultMatrixf. This is how you translate and rotate.
It matters the order in which these operations are applied. Here's a fun way to experiment with that, check out Livecodelab, a live 3D sketch coding environment which runs inside your browser. In particular this tutorial encourages reflection on the ordering of operations. Obviously the command move is a translation.
I'm a newbie in the OpenGL ES world, and learning some basics on 3d graphics on Android OpenGL ES. I'm wondering how to create a image plane that emitting light? This is easy to be implemented in 3d model software like Blender (using the Cycles Render), see the image below for effects I'm looking for. Through some research, I learnt that they may be related to Blur or Bloom effect using shader. But I'm not very sure, and I don't know how to implement them.
As per Paul-Jan's comment, what you want is far from basic in OpenGL.
The default approach for OpenGL is forward rendering. i.e. every time you specify a piece of geometry the calculation goes forwards from triangle to pixels, a function is applied to determine the colour for each of those pixels and they're forwarded to the frame buffer. So the starting position is that each individual pixel has no concept of the world around it. Each exists in isolation.
In your scene, the floor below the box has no idea it should be blue because it has no idea that there is a box above it.
Programs like Blender use a different approach, which in this context could accurate be called backwards rendering. It starts from each pixel and asks what geometry lies behind it. In doing that it explicitly has an idea of all the geometry in the scene. So when it spots that the floor is behind a certain position it can then continue and ask "and which light sources can the floor see?" to establish lighting.
The default OpenGL approach is long established for real-time rendering. If you look at old video games you'll notice evidence of it all over the place: objects often don't cast shadows on each other (or such shadows are very rough approximations), there's only one source of light which is infinitely far away (i.e. it's in a fixed position as far as geometry is concerned; no need to know about the scene really).
So solutions are to invest the geometry with some knowledge of the whole scene. A common approach is to perform internal renderings of the scene from the point of view of the light source. That generates a depth buffer. By handing the light position and depth buffer off to every piece of geometry in the scene they can calculate whether they're visible to the light source. If so then they're illuminated by it. If not then they're not.
Another option is deferred rendering; you do a standard pass of your scene, populating at each pixel the depth, the surface colour, the surface normal, etc. So you get the full scene information broken down into pixel-by-pixel storage from the point of view of the camera. You then pretend that everything the camera can see is everything that there is. So you just need to pass that buffer around for pixels to be able to work out, approximately, which light sources they can and can't see. You can also have different parts of the screen only consider which lights they're close enough to by a broad-phase 2d distance check, which saves time.
In either case we're actually talking about relatively advanced OpenGL stuff.
I am making an Android game app and am using openGL to load 3D obj models into the app. I would like to know if anyone can help me on how to make these objects interactive. All I really need is to make an object clickable, but it would be cool to learn more such as dragging it around the screen and such.
Thanks for any help.
What you want is called picking.
This isn't a easy task an depending on want you want to do exaclty you have mutliple options to do it.
The Problem: You want to select an Object in 3D space based on a 2D coordinate (mouse/touch position). Because of the 2D mouse coordinates you miss one coordinate to determine the exact click position in the 3D space.
One possible option would be:
Render your object with a specific color (e.g. completely red)
After that save the current display buffer to a variable
Clear the display buffer and render your model again with the standard settings (this is the screen that is displayed to the user)
determine the click/touch position translate it to a coordinate on your display area
check the color at this coordinate in your saved display buffer. If the color at this position is red, the user clicked/touch an object
This approach isn't that flexible but the implementation is very simple compared to other solutions. It is limited because you can only detect if the used clicked/touched a certain object or not but you cannot determine the exact position on the object.
Another possible option is to compute a ray on the 3D world based on the 2D click position and then determine all objects in 3D space that collide with this ray. This is called ray picking.
You can find a OpenGl tutorial for ray picking here
The example uses glRenderMode, glLoadName, etc. which maybe isn't the best choice if you are not using the fixed function pipeline (e.g. you are using custom shaders, etc.).
Another option whould be to do the math and compute the ray vector yourself based on click position, viewport and projection matrices. If you want to do this the documentation of gluUnproject can help you.
Assume I have 3 cubes at random location/orientation and I want to detect if any of the cube is overlapping (or colliding) with another cube. This overlap or collision could also happen as the cubes location/rotation are changed in each frame. Please note that I am looking for Android based and OpenGL ES (1.0 or 1.1) based solution for this.
This isn't really an OpenGL problem - it just does rendering.
I don't know of any ready-made android libraries for 3D collision detection, so you might just have to do the maths yourself. Efficient collision detection is generally the art of using quick, cheap tests to avoid doing more expensive analysis. For your problem, a good approach to detecting if cube A intersects cube b would be to do a quick rejection test, either
Compute the bounding spheres for A and B - if the distance between the two sphere's centers is greater than the sum of radii, then A and B do not intersect
Compute the axis-aligned bounding boxes for A and B - if the bounds do not intersect (very easy to test), then neither do A and B
If the bounds test indicates possible collision it's time for some maths. There are two ways to go from here: testing for vertex inclusion and testing for edge/face intersection
Vertex inclusion is testing the vertices of A to see if they lie within B: either rotate the vertex into B's frame of reference to test for inclusion, or use the planes of B's faces directly in a frustum-culling style operation.
Edge/Face intersection is testing each of the edges of A for intersection with B's face triangles.
While the vertex inclusion test is a bit cheaper than Edge/Face testing, it's possible for cubes to intersect without encompassing each other's vertices, so a negative result does not mean no intersection. Similarly, it's possible for cubes to intersect without an intersection between an edge and a face (if one lies within the other). You'll have to do a little of both tests to catch every intersection. This can be avoided if you can make some assupmtions about how the cubes can move from frame to frame, i.e.: if the A and B were not touching last frame, it's unlikely that they are A is wholly within B now.
I'm developing an Android application with OpenGL and JNI (all OpenGL stuff is in C code).
Imagine I've drawn a cube. I want that user can push his finger over the cube and can rotate the cube and move it around the screen.
Is there any way to do that?
How can assign an event listener to touch and move events only when the user touch the cube?
UPDATE I want something like this:
Rotate cube with fingers
Thanks.
This is called "picking" in 3d-ville... There are a number of tutorials on the subject floating hither and yon. There's even another question (sans the JNI spin) here on StackOverflow.
Also, check out this google IO video on developing android games to see why your approach may not be faster than pure Java... It Depends.
It turns out that JNI calls are Quite Expensive, so a JNI-based renderer could end up slower than a pure-java one unless you are Very Careful. YM Will V.
I'm pretty sure you'll have to listen to all touches then change behavior based on what is being touched. I suppose you could compute your cube's bounding box on screen and then monkey with your listeners every frame (or every time it moves), but I seriously doubt that would be the most efficient course to take. Listen for all touches, react appropriately.