Develop Reference

Sceneform plane rendering similar to the HelloAR sample

I was trying to cover the whole surface of the detected plane with a texture. Using OpenGl (like in the HelloAr sample) enables me do this like this screenshot . However i want to switch to Sceneform and i only could get something like this by following other questions in the Github.
These are the codes that i used currently for texture rendering. To conclude, i do not want this spotlight texture on the plane. I want to cover whole detected plane like my first screenshot. Can you give some information about how can i achieve this? Thank you!
Texture.Sampler sampler =
Texture.Sampler.builder()
.setMagFilter(Texture.Sampler.MagFilter.LINEAR)
.setMinFilter(Texture.Sampler.MinFilter.LINEAR)
.setWrapMode(Texture.Sampler.WrapMode.REPEAT)
.build();
CompletableFuture<Texture> trigrid = Texture.builder()
.setSource(this, R.drawable.gray)
.setSampler(sampler).build();
PlaneRenderer planeRenderer = arSceneView.getPlaneRenderer();
planeRenderer.getMaterial().thenAcceptBoth(trigrid, (material, texture) -> {
material.setTexture(PlaneRenderer.MATERIAL_TEXTURE, texture);
});`

I found the solution both using other answers and combine them. Proper plane rendering can be done like below. Important point is this renderable should be called at every frame otherwise ARCore overrides it and you don't see the effect that you want. Here is a code that solves my problem.
arSceneView.getScene().addOnUpdateListener(frameTime -> {
PlaneRenderer planeRenderer = arSceneView.getPlaneRenderer();
planeRenderer.getMaterial().thenAcceptBoth(trigrid, (material, texture) -> {
material.setTexture(PlaneRenderer.MATERIAL_TEXTURE, texture);
material.setFloat(PlaneRenderer.MATERIAL_SPOTLIGHT_RADIUS, Float.MAX_VALUE);
});
});

You can control the radius of the spotlight on the material. To make the spotlight effect go away, you can set the radius to a large number:
planeRenderer.getMaterial().thenAcceptBoth(trigrid, (material, texture) -> {
material.setTexture(PlaneRenderer.MATERIAL_TEXTURE, texture);
material.setFloat(PlaneRenderer.MATERIAL_SPOTLIGHT_RADIUS,1000);
});

From this, probably is the size of your texture that are a little strange.
Change the MATERIAL_UV_SCALE parameter. This parameter is used to control the size at which the texture.
planeRenderer.getMaterial()
.thenAcceptBoth(trigrid, (material, texture) -> {
material.setTexture(PlaneRenderer.MATERIAL_TEXTURE, texture);
material.setFloat2(PlaneRenderer.MATERIAL_UV_SCALE, 50.0f, 50.0f);
});

Depth test not working correctly with stencil test

I need to cut holes in the 3D surface using stencil buffer.
Currently, everything works as expected, but the main problem is that holes are also visible through hills. How to prevent this behavior and hide holes if they are behind hills?
Current code:
GLES20.glColorMask(false,false,false,false);
GLES20.glDepthMask(false);
GLES20.glEnable(GLES20.GL_STENCIL_TEST);
GLES20.glStencilFunc(GLES20.GL_ALWAYS, 1, 0xFF);
GLES20.glStencilOp(GLES20.GL_KEEP, GLES20.GL_KEEP, GLES20.GL_REPLACE);
GLES20.glStencilMask(0xFF);
GLES20.glClear(GLES20.GL_STENCIL_BUFFER_BIT);
//drawing holes
GLES20.glStencilFunc(GLES20.GL_EQUAL, 0, 0xFF);
GLES20.glStencilMask(0x00);
GLES20.glColorMask(true,true,true,true);
GLES20.glDepthMask(true);
//draw surface with elevation

A solution which provides the effect, can be achieved by Face Culling by a separated stencil test for front and back faces and (see glStencilFuncSeparate and glStencilOpSeparate).
Sadly the back faces of the geometry have to be drawn in a separated step.
The process can be described in the following steps:
Enable the depth test
Disable the color buffer and enable the stencil test for setting the stencil mask
Draw the "holes". This causes that the stencil buffer is set to 1 at the positions of the holes.
Setup the stencil test for clearing the the stencil mask by backfaces
Enable face culling for front faces
Draw the geometry. This causes that the stencil buffer is cleared at positions which are covered.
Enable face culling for back faces
Enable the color buffer
Draw the geometry
To make tha algorithm work, you have to draw all your primitives int the same winding order. And you have to tell OpenGL the direction
by glFrontFace.
Either clockwise GL_CW or counterclockwise GL_CCW.
GLES20.glStencilMask(0xFF);
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_STENCIL_BUFFER_BIT);
GLES20.glFrontFace(GLES20.GL_CCW); // depends on your geometry "GL_CCW" or "GL_CW"
GLES20.glDisable(GLES20.GL_CULL_FACE);
GLES20.glEnable(GLES20.GL_DEPTH_TEST);
GLES20.glDepthFunc(GLES20.GL_LESS); // default
GLES20.glColorMask(false,false,false,false);
GLES20.glEnable(GLES20.GL_STENCIL_TEST);
GLES20.glStencilFuncSeparate(GLES20.GL_FRONT, GLES20.GL_ALWAYS, 1, 0xFF);
GLES20.glStencilFuncSeparate(GLES20.GL_BACK, GLES20.GL_ALWAYS, 1, 0xFF);
GLES20.glStencilOpSeparate(GLES20.GL_FRONT, GLES20.GL_KEEP, GLES20.GL_KEEP, GLES20.GL_REPLACE);
GLES20.glStencilOpSeparate(GLES20.GL_BACK, GLES20.GL_KEEP, GLES20.GL_KEEP, GLES20.GL_KEEP);
// draw the holes
// ....
GLES20.glStencilFuncSeparate(GLES20.GL_FRONT, GLES20.GL_EQUAL, 0, 0xFF);
GLES20.glStencilOpSeparate(GLES20.GL_FRONT, GLES20.GL_KEEP, GLES20.GL_KEEP, GLES20.GL_KEEP);
GLES20.glStencilFuncSeparate(GLES20.GL_BACK, GLES20.GL_ALWAYS, 0, 0xFF);
GLES20.glStencilOpSeparate(GLES20.GL_BACK, GLES20.GL_KEEP, GLES20.GL_KEEP, GL_REPLACE);
GLES20.glEnable(GLES20.GL_CULL_FACE);
GLES20.glCullFace(GLES20.GL_FRONT);
// draw the geometry the 1. time ("draw" back faces)
// ....
GLES20.glCullFace(GLES20.GL_BACK);
GLES20.glColorMask(true,true,true,true);
// draw the geometry the 2. time (draw front faces)
// ....

OpenGL - use different polygons one drawcall - instancing

I am using OpenGL ES version 3 for my android game and have implemented instancing. It works well IF I use a polygon of the same size/dimension that is identical vertices. I can jump to different UV-coordinates of the texture atlas if I want to create change the sprites state for every frame.
v_TexCoordinate = a_TexCoordinate + uvCoordsOffset[gl_InstanceID];
that is .. I just change the texture coordinates with a uniform-vec which consists of offset coordinates.
BUT - here comes the issue.
What if I want to do the same but with sprites that have different dimensions?
in the drawcall ...
GLES30.glDrawElementsInstanced(GLES30.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, indexOffset, nFallingObj);
I can only send one polygon, that is the green dinosaur vertices in the screenshot. I have this as base polygon and I want to get to the RED dinosaur of the atlas. I can do this easily with texture offset as described above BUT you see how it becomes? the polygon of the green dinosaur is used but I want to be changed as of the red dinosaurs.
Is there any easy solution to this issue?
thanks in advance!!!
some source-code
the draw-method
public void drawFallingObjects() {
GLES30.glUseProgram(mProgramHandle);
GLES30.glEnableVertexAttribArray(mPositionHandle);
GLES30.glVertexAttribPointer(mPositionHandle, CreateGLContext.POSITION_DATA_SIZE, GLES20.GL_FLOAT, false, CreateGLContext.STRIDE, 0);
GLES30.glEnableVertexAttribArray(mTextureCoordinateHandle);
GLES30.glVertexAttribPointer(mTextureCoordinateHandle, CreateGLContext.TEXTURE_COORDINATE_DATA_SIZE, GLES20.GL_FLOAT, false,
CreateGLContext.STRIDE, CreateGLContext.POSITION_DATA_SIZE * CreateGLContext.BYTES_PER_FLOAT);
GLES30.glUniform2fv(uvCoordsOffsetLoc, nFallingObj, uvOffsetVec, 0);
GLES30.glUniformMatrix4fv(mMVPMatrixHandle, nFallingObj, false, mMVPMatrixMajor, 0);
GLES30.glDrawElementsInstanced(GLES30.GL_TRIANGLES, 6, GLES20.GL_UNSIGNED_SHORT, indexOffset, nFallingObj);
}
vertex-shader code
void main()
{
v_Color = a_Color;
v_TexCoordinate = a_TexCoordinate + uvCoordsOffset[gl_InstanceID];
gl_Position = u_MVPMatrix[gl_InstanceID] * a_Position;
}

What if I want to do the same but with sprites that have different
dimensions?
Your MVP matrix can encode scale and skew, which is all you need to adjust the on-screen sprite size.

OpenCV - Detect hand-drawing shapes

Could OpenCV detect the geometric shapes which is drawn by hand as below? The shape can be a rectangle, triangle, circle, curve, arc,polygon,...
I am going to develop an android application which detect these shapes.

Well, I tried it in a harry. Normally you need to skeletonize the input. Anyway. You can reason about the shapes based on their points. Normally a square has 4, a triangle 3, etc.
Effort results:
Canny results:
Polygonal approximation:
Console output:
contour points:11
contour points:6
contour points:4
contour points:5
Here is the code:
Mat src=imread("WyoKM.png");
Mat src_gray(src.size(),CV_8UC1);
if (src.empty()) exit(-10);
imshow("img",src);
/// Convert image to gray and blur it
cvtColor( src, src_gray, CV_BGR2GRAY );
threshold(src_gray,src_gray,100,255,src_gray.type());
imshow("img2",src_gray);
Mat canny_output;
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
/// Detect edges using canny
int thresh=100;
Canny( src_gray, canny_output, thresh, thresh*2, 3 );
imshow("canny",canny_output);
imwrite("canny.jpg",canny_output);
/// Find contours
findContours( canny_output, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );
// testing the approximate polygon
cv::Mat result(src_gray.size(),CV_8U,cv::Scalar(255));
for(int i=0;i<contours.size();i=i+4) //for testing reasons. Skeletonize input.
{
std::vector<cv::Point> poly;
poly.clear();
cv::approxPolyDP(cv::Mat(contours[i]),poly,
5, // accuracy of the approximation
true); // yes it is a closed shape
// Iterate over each segment and draw it
std::vector<cv::Point>::const_iterator itp= poly.begin();
cout<<"\ncontour points:"<<poly.size();
while (itp!=(poly.end()-1)) {
cv::line(result,*itp,*(itp+1),cv::Scalar(0),2);
++itp;
}
// last point linked to first point
cv::line(result,
*(poly.begin()),
*(poly.end()-1),cv::Scalar(20),2);
}
imshow("result",result);
imwrite("results.jpg",result);
cvWaitKey();

Detect touch on OpenGL object?

I am developing an application which uses OpenGL for rendering of the images.
Now I just want to determine the touch event on the opengl sphere object which I have drwn.
Here i draw 4 object on the screen. now how should I come to know that which object has been
touched. I have used onTouchEvent() method. But It gives me only x & y co-ordinates but my
object is drawn in 3D.
please help since I am new to OpenGL.
Best Regards,
~Anup

t Google IO there was a session on how OpenGL was used for Google Body on Android. The selecting of body parts was done by rendering each of them with a solid color into a hidden buffer, then based on the color that was on the touch x,y the corresponding object could be found. For performance purposes, only a small cropped area of 20x20 pixels around the touch point was rendered that way.

Both approach (1. hidden color buffer and 2. intersection test) has its own merit.
1. Hidden color buffer: pixel read-out is a very slow operation.
Certainly an overkill for a simple ray-sphere intersection test.
Ray-sphere intersection test: this is not that difficult.
Here is a simplified version of an implementation in Ogre3d.
std::pair<bool, m_real> Ray::intersects(const Sphere& sphere) const
{
const Ray& ray=*this;
const vector3& raydir = ray.direction();
// Adjust ray origin relative to sphere center
const vector3& rayorig = ray.origin() - sphere.center;
m_real radius = sphere.radius;
// Mmm, quadratics
// Build coeffs which can be used with std quadratic solver
// ie t = (-b +/- sqrt(b*b + 4ac)) / 2a
m_real a = raydir%raydir;
m_real b = 2 * rayorig%raydir;
m_real c = rayorig%rayorig - radius*radius;
// Calc determinant
m_real d = (b*b) - (4 * a * c);
if (d < 0)
{
// No intersection
return std::pair<bool, m_real>(false, 0);
}
else
{
// BTW, if d=0 there is one intersection, if d > 0 there are 2
// But we only want the closest one, so that's ok, just use the
// '-' version of the solver
m_real t = ( -b - sqrt(d) ) / (2 * a);
if (t < 0)
t = ( -b + sqrt(d) ) / (2 * a);
return std::pair<bool, m_real>(true, t);
}
}
Probably, a ray that corresponds to cursor position also needs to be calculated. Again you can refer to Ogre3d's source code: search for getCameraToViewportRay. Basically, you need the view and projection matrix to calculate a Ray (a 3D position and a 3D direction) from 2D position.

In my project, the solution I chose was:
Unproject your 2D screen coordinates to a virtual 3D line going through your scene.
Detect possible intersections of that line and your scene objects.
This is quite a complex tast.

I have only done this in Direct3D rather than OpenGL ES, but these are the steps:
Find your modelview and projection matrices. It seems that OpenGL ES has removed the ability to retrieve the matrices set by gluProject() etc. But you can use android.opengl.Matrix member functions to create these matrices instead, then set with glLoadMatrix().
Call gluUnproject() twice, once with winZ=0, then with winZ=1. Pass the matrices you calculated earlier.
This will output a 3d position from each call. This pair of positions define a ray in OpenGL "world space".
Perform a ray - sphere intersection test on each of your spheres in order. (Closest to camera first, otherwise you may select a sphere that is hidden behind another.) If you detect an intersection, you've touched the sphere.

for find touch point is inside circle or not..
public boolean checkInsideCircle(float x,float y, float centerX,float centerY, float Radius)
{
if(((x - centerX)*(x - centerX))+((y - centerY)*(y - centerY)) < (Radius*Radius))
return true;
else
return false;
}
where
1) centerX,centerY are center point of circle.
2) Radius is radius of circle.
3) x,y point of touch..