I'm new to 3D programming and have been playing around with OpenGL ES for Android for a little while now and I've seen some options of this questions beaning ray tracking/tracing and Object Picking and something about using the pixels to select 3D Objects. I'm trying to make something like a paint program with OpenGL ES for Android to where I can select a line from a cube and delete it or objects to be deleted or modified. Anyway, I'm unsure of where to start learning this I've tried Google and didn't really find anything helpful. Maybe if there's a video tutorial or a website that explains this better or any help to point me in the direction to go would be very grateful. Thank you so much in advanced.
Yes I know this is a possible duplicate Question.
I'm an iOS dev myself, but I recently implemented ray casting for my game, so I'll try to answer this in a platform agnostic way.
There are two steps to the ray-casting operation: firstly, you need to get the ray from the user's tap, and secondly, you need to test the triangles defining your model for intersections. Note that this requires you to still have them in memory or be able to recover them -- you can't just be keeping them in a vbo on the graphics card.
First, the conversion to world coordinates. Since you are no doubt using a projection matrix to get a 3-D perspective for your models, you need to unproject the point to get it in world coordinates. There are many libraries with this already implemented, such as glut's glunproject which I believe are available on Android. I believe that mathematically this amounts to taking the inverse of all the transformations which are currently acting on your models. Regardless, there are many implementations publicly available online you can copy from.
At this point, you are going to need a Z coordinate for the point you are trying to unproject. You actually want to unproject twice, once with a Z coord of 0 and once with a Z coord of 1. The vector which results from the z-Coord of 0 is the origin of the ray, and by subtracting this vector from your z-coord of 1 vector you will get the direction. Now you are ready to test for intersections of your model's polygons.
I have had success with the method presented in this paper (http://www.cs.virginia.edu/~gfx/Courses/2003/ImageSynthesis/papers/Acceleration/Fast%20MinimumStorage%20RayTriangle%20Intersection.pdf) for doing the actual intersection test. The algorithm is implemented in C at the end, but you should be able to convert it to Java with little trouble.
The selection features of OpenGL are not available with OpenGL ES, so you would have to build that yourself.
I recommend starting with OpenGL ES sample programs that are specific to Android. I think this article will help:
http://software.intel.com/en-us/articles/porting-opengl-games-to-android-on-intel-atom-processors-part-1
Related
as part of my project, I need to plot 2D and 3D functions in android using android studio. I know how to plot 2D functions but I'm struggling with 3D functions.
What is the best way to plot 3D functions? What do I need and where do I start?
I'm not looking for code or external libraries that do all the work, I just need to know what I need to learn to be able to do it myself.
Thanks in advance.
I know how to plot 2D functions but I'm struggling with 3D functions.
What is the best way to plot 3D functions? What do I need and where do I start?
I'm not looking for code or external libraries that do all the work, I just need to know what I need to learn to be able to do it myself.
Since you already understand 2D and want to advance to 3D there's a simple and non-optimal method:
Decide on how much z depth you desire:
EG: Currently your limits for x and y in your drawing functions are 1920 and 1080 (or even 4096x4096), if you want to save memory and have things a bit low resolution use a size of 1920x1080x1000 - that's going to use 1000x more memory and has the potential to increase the drawing time of some calls by 1000 times - but that's the cost of 3D.
A more practical limit is matrices of 8192,8192,16384 but be aware that video games at that resolution need 4-6GB graphic cards to work half decently (more being a bit better) - so you'll be chewing up some main memory starting at that size.
It's easy enough to implement a smaller drawing space and simply increase your allocation and limit variables later, not only does that test that future increases will go smoothly but it allows everything to run faster while you're ironing the bugs out.
Add a 3rd dimension to the functions:
EG: Instead of a function that is simply line_draw(x,y) change it to line_draw(x,y,z), use the global variable z_limit (or whatever you decide to name it) to test that z doesn't exceed the limit.
You'll need to decide if objects at the maximum distance are a dot or not visible. While testing having all z's past the limit changed to the limit value (thus making them a visible dot) is useful. For the finished product once it goes past the limit that you are implementing it's best that it isn't visible.
Start by allocating the drawing buffer and implementing a single function first, there's no point (and possibly great frustration) changing everything and hoping it's just going to work - it should but if it doesn't you'll have a lot on your plate if there's a common fault in every function.
Once you have this 3D buffer filled with an image (start with just a few 3D lines, such as a full screen sized "X" and "+") you draw to your 2D screen X,Y by starting at the largest value of Z first (EG: z=1000). Once you finish that layer decrement z and continue, drawing each layer until you reach zero, the objects closest to you.
That's the simplest (and slowest) way to make certain that closest objects obscure the furthest objects.
Now does it look OK? You don't want distant objects the same size (or larger) than the closest objects, you want to make certain that you scale them.
The reason to choose numbers such as 8192 is because after writing your functions in C (or whichever language you choose) you'll want to optimize them with several versions each, written in assembly language, optimized for specific CPUs and GPU architectures. Without specifically optimized versions everything will be extremely slow.
I understand that you don't want to use a library but looking at several should give you an idea of the work involved and how you might implement your own. No need to copy, improve instead.
There are similar questions and answers that might fill in the blanks:
Reddit - "I want to create a 3D engine from scratch. Where do I start?"
Davrous - "Tutorial series: learning how to write a 3D soft engine from scratch in C#, TypeScript or JavaScript"
GameDev.StackExchange - "How to write my own 3-D graphics library for Windows? [closed]"
I am trying to convert point clouds sampled and stored in XYZij data (which, according to the document, stores data in camera space) into a world coordinate system so that they can be merged. The frame pair I use for the Tango listener has COORDINATE_FRAME_START_OF_SERVICE as the base frame and COORDINATE_FRAME_DEVICE as the target frame.
This is the way I implement the transformation:
Retrieve the rotation quaternion from TangoPoseData.getRotationAsFloats() as q_r, and the point position from XYZij as p.
Apply the following rotation, where q_mult is a helper method computing the Hamilton product of two quaternions (I have verified this method against another math library):
p_transformed = q_mult(q_mult(q_r, p), q_r_conjugated);
Add the translate retrieved from TangoPoseData.getTranslationAsFloats() to p_transformed.
But eventually, points at p_transformed always seem to end up in clutter of partly overlapped point clouds instead of an aligned, merged point cloud.
Am I missing anything here? Is there a conceptual mistake in the transformation?
Thanks in advance.
Ken & Vincenzo, thanks for the reply.
I somehow get better results by performing ICP registration using CloudCompare on individual point clouds after they are transformed into world coordinates using pose data alone. Below is a sample result from ~30 scans of a computer desk. Points on the farther end are still a bit off, but with carefully tuned parameters this might be improved. Also CloudCompare's command line interface makes it suitable for batch processing.
Besides the inevitable integration error that needs to be corrected, a mistake I made earlier was wrongly taking the camera space frame (the camera on the device), which is described here in the documentation, to be the same as the OpenGL camera frame, which is the same as the device frame as described here. But they are not.
Also, moving the camera slowly to get more overlap between two adjacent frames also helps registration. And a good visible lighting setup of the scene is important, since besides the motion sensors, Tango also relies on the fish eye camera on its back for motion tracking.
Hope the tips also work for more general cases other than mine.
There are two different "standard" forms of the quaternion notation. One has the rotation angle first, i.e. x i j k, and one has the rotation angle last, i.e. x y z w. The Tango API docs list the TangoPoseData::orientation as x y z w. The Wikipedia page on quaternions lists them as x i j k. You might want to check what notation is assumed in your product method.
Where is your pose data coming from? Are you getting the most recent pose after you are in the callback for the point cloud data or are you asking for the pose that corresponds to the timestamp in the XYZij struct? You should be asking for the pose at time "timestamp" from the XYZij struct.
I tried it, it does not work.
I tried to queue the pose and get the nearest one to the XYZij.
Look at the blue wall
The real wall
we from roomplan.de created an opensource sample how to use pcl in project tango apps. It records pointclouds and transforms them into a common coordinate frame (the StartOf Service Frame). You can find the sample code here: https://github.com/roomplan/tango-examples-java/tree/master/PointCloudJava_with_PCL the specific funtion is in jni/jni_part.cpp function: Java_com_tangoproject_experiments_javapointcloud_PointCloudActivity_saveRotatedPointCloud
If you want the sample to compile, you need to clone the complete folder and integrate pcl into your project. A solution how this can be done can be found on our website.
sample pictures can be viewed at the demo app in the playstore. (Cant post them here yet) https://play.google.com/store/apps/details?id=com.tangoproject.experiments.javapointcloud&hl=en
So, I have scoured the net, looking for information on how one would do this. And so far, all I've come up with is....nohting.
Any one got a better starting point than http://www.paulsprojects.net/tutorials/simplebump/simplebump.html
If you are targeting OpenGL 2.0 is in fact quite easy and you just need a normal map, and a shader that handles the lighting equation per pixel. I found this for you: http://www.learnopengles.com/android-lesson-four-introducing-basic-texturing/
I need to write to the depth buffer on an android device (OpenGL ES 2.0). Since gl_FragDepth is not writable under OGL ES 2.0, I have to find a workaround. I actually want to render spheres via raycasting, similar to this: http://www.sunsetlakesoftware.com/2011/05/08/enhancing-molecules-using-opengl-es-20 .
However, the solution explained on this website (offscreen render pass writing the depth using a special glBlendEquation) is only working on Apple devices, not on Android, because GL_MIN_EXT-blending is not supported.
On my Tegra3 tablet I was able to implement this method: Android GLES20.glBlendEquation not working? (btw, I recommend using linearized depth values, they give better results!)
It works quite good, but of course this is only available on Nvidia GPUs.
In theory, there is the extension GL_EXT_frag_depth (see Can an OpenGL ES fragment shader change the depth value of a fragment?), but it is not available on Android devices as well.
Finally, you could of course write the depth buffer for just one sphere (in an offscreen render pass), then write the depth buffer for the next sphere in a second render pass and combine the two in a third render pass. In doing so, you would have 2*n+1 render passes for n spheres - which seems to be quite inefficient!
So since I am running out of ideas, my question is: Can you think of another, generic way/workaround to write the depth buffer on an OpenGL ES 2.0 Android device?
Well, you are sure running out of options here. I don't know of any further workaround because I don't know Opengl ES soooo well.
The only thing that comes into my mind would be combining the brute-force multi-pass approach with some preprocessing:
Sort your spheres into groups where the atoms are not overlapping each other. It should be possible to sort all your spheres from proteins in less then ten groups. Then render all spheres of each group in one pass. The vertex-depth is sufficient here, because the spheres do not overlap. Then you can "depth-blend" the results.
This requires some preprocessing which could be a problem.
My question comes from why OpenGL and/or Android does not have a way to simply grab the current matrix and store it as a float[]. All the research I have found suggests using these classes which it looks like I have to download and put in my project called MatrixGrabber to be able to grab the current state of the Open GL matrix.
My overall goal is to easily determine what the Open GL world location is of an event caused by touching the screen where I can retriever the X and Y coordinates by the event.
The best workaround I have found is Android OpenGL 3D picking. but I wonder why there isn't a way where you can simply retriever the matrices you want and then just call
GLU.gluUnProject(...);
My question comes from why OpenGL and/or Android does not have a way to simply grab the current matrix and store it as a float[].
Because OpenGL ES 2.0 (and core desktop GL 3.1 and above) do not necessarily have a "current matrix." All transforms are done via shader logic, so matrices don't even have to be involved. It could be doing anything in there.
There is no current matrix, so there is nothing to get. And nothing to unproject.
In ES 1 you can grab the current matrix and store it as a float using glGetFloatv, with the pname GL_MODELVIEW_MATRIX, GL_PROJECTION_MATRIX or GL_TEXTURE_MATRIX as applicable.
GLU is not inherently part of OpenGL ES because ES is intended to be a minimal specification and GLU is sort of an optional extra. But you can grab SGI's reference implementation of GLU and use its gluUnProject quite easily.
EDIT: and to round off the thought, as Nicol points out there's no such thing as the current matrix in ES 2. You supply to your shaders arbitrarily many matrices for arbitrarily many purposes, and since you supplied them in the first place you shouldn't need to ask GL to get them back again.
I just took a look at http://developer.android.com/resources/samples/ApiDemos/src/com/example/android/apis/graphics/spritetext/MatrixGrabber.html which looks like the MatrixGrabber you're referring to and it doesn't look especially complicated — in fact, it's overbuilt. You can just use gl2.getMatrix directly and plug that into gluUnProject.
The probable reason for the design of the MatrixGrabber code is that it caches the values for multiple uses — because the GPU runs asynchronously to the CPU, your code may have to wait for the getMatrix response, so it is more efficient to get it as few times as possible and reuse the data.
Another source of complexity in the general problem is that a touch specifies only two dimensions. A single touch does not indicate any depth, so you have to do that in some application-specific way. The obvious approacj is to read the depth buffer (though some OpenGL implementations don't support that), but that doesn't work if you have e.g things that should be "transparent" to touches. An alternative is to construct a ray (such as by unprojecting twice with two different depths) and then do raycasting into your scene.