How to properly translate detected face coordinates from front camera - android

I'm using Google Vision for face detection on Android. Currently my code:
public void onPreviewFrame(byte[] data, Camera camera) {
// creating Google Vision frame from a camera frame for face recognition
com.google.android.gms.vision.Frame frame = new com.google.android.gms.vision.Frame.Builder()
.setImageData(ByteBuffer.wrap(data), previewWidth,
previewHeight, ImageFormat.NV21)
.setId(frameId++)
.setRotation(com.google.android.gms.vision.Frame.ROTATION_270)
.setTimestampMillis(lastTimestamp).build();
// recognize the face in the frame
SparseArray<Face> faces = detector.detect(frame);
// wrong coordinates
float x = faces.valueAt(0).getPosition().x;
float y = faces.valueAt(0).getPosition().y;
}
The problem is that x and y are not correct and even negative sometimes. I know that to get correct coordinates it should be rotated somehow, but how exactly?

These coordinates can be negative if the face extends beyond the top and/or the left edges of the image. Even though the head may not be entirely within the photo, the face detector will estimate the bounding box of the face beyond the image bounds based upon what is visible.
The coordinates should be correct relative to the image. However, if you are drawing on a preview from the front-facing camera, note that this preview is displayed reversed (like a mirror image). In this case, you'd need to reverse the coordinates on order to draw on the preview. See an example of how this is done here:
https://github.com/googlesamples/android-vision/blob/master/visionSamples/FaceTracker/app/src/main/java/com/google/android/gms/samples/vision/face/facetracker/ui/camera/GraphicOverlay.java#L101

Related

How to zoom Camera using android camera2 api with a float numbers

I developed a custom camera using camera2 api, i make zoom in and zoom out feature like this.
Camera.Parameters params = mCamera.getParameters();
int value = params.getMaxZoom();
params.setZoom(value);
It works for me, but how can I zoom camera by float number like 2.4f ?
Can any one help me, please?
https://developer.android.com/reference/android/hardware/Camera.Parameters.html#getMaxZoom()
Sets current zoom value. If the camera is zoomed (value > 0), the actual picture size may be smaller than picture size setting.
https://developer.android.com/reference/android/hardware/Camera.Parameters.html#setZoom(int)
setZoom takes an Integer (whole number) so you can't set the zoom to a float.
The newest Android CameraX api's can Zoom using a Rect: https://developer.android.com/reference/androidx/camera/core/Preview.html#zoom(android.graphics.Rect) (but under the hood it relies on Camera2 so I wouldn't hold out much hope for hardware float zoom.

3d object depth parameter affects perceived width

I have created one RectangularPrism with default camera in Rajawali 1.1.777:
RectangularPrism p = new RectangularPrism(2f, 2f, xf);
Depending on x (depth) the perceived width on the Android screen varies.
If the depth is 2f the width of the prism fills the screen.
If the depth is 0.5f & 1f the width of the prism does not fill the screen.
So from a Rajawali/OpenGL perspective how and why does the depth parameter affect the perceived width of the object on the screen? Since the width (2f) is kept static shouldn't the prism always fill the screen in the same way?
0.1f:
1f:
2f:
Answer from https://github.com/Rajawali/Rajawali/issues/1883
The default camera is a Perspective Camera, faces in the distance appear smaller than faces close by. The default camera is located at 0,0,4 and looks at 0,0,0. If one places a RectangularPrism of size 2,2,0.5 at 0,0,0, the distance between the camera and the closest face is 3.75. If one places a RectangularPrism of size 2,2,2 at 0,0,0, the distance between the camera and the closest face is only 3, and so the closest face appears bigger.
If perspective is not desired, one may instantiate an Orthographic Camera and getCurrentScene().addAndSwitchCamera(orthographicCamera).

draw a text on nose base with face detection API

I'm trying to draw a text on a face using android face detection API.
Right now, I did this
for(Landmark landmark : face.getLandmarks()){
if(landmark.getType() == Landmark.NOSE_BASE){
Bitmap moustache = BitmapFactory.decodeResource(resources, R.drawable.moustache);
canvas.drawText("=====", landmark.getPosition().x, landmark.getPosition().y, mIdPaint);
}
}
but turns out the text ===== is draw on top of the head, and I don't know why.
If someone need more code, just let me know
If you are drawing graphics over a live camera preview, you need to take a few things into account:
the device's rotation
the scale of the view relative to the size of the preview image
whether you are using the front facing camera (which will mirror the image)
The sample code for the face tracker demo has utility methods (translateX, translateY, scaleX, scaleY) to help with this:
https://github.com/googlesamples/android-vision/blob/master/visionSamples/FaceTracker/app/src/main/java/com/google/android/gms/samples/vision/face/facetracker/FaceGraphic.java#L99
https://github.com/googlesamples/android-vision/blob/master/visionSamples/FaceTracker/app/src/main/java/com/google/android/gms/samples/vision/face/facetracker/ui/camera/GraphicOverlay.java#L100

Convert 2d screen coordinates to 3d World Coordinates

I would like to convert 2d Screen Coordinates to 3d World Coordinates.
I put together a hack but would like a solid solution. If possible please use your equation/algorithm on the example below (if there is already a link, could you use the link to show the solution to the problem below)
My Environment is Java/C++ for Android and iOS using OpenGL 2.0
Problem to Solve
Where screen
(0,0) is top left and bottom right is screenWidth and screenHeight.
screenWidth=667; screenHeight=375;
//Camera's position,where camera is looking at, and camera direction
CamPos (0.0f,0.0f,11.0f); CamLookAt (0.0f, 0.0f, 10.5f); CamDir(0.0f,1.0f,0.0f);
//The Frustum
left =1.3f; right=1.3f; bottom=1.0f; top=1.0f; near=3.0f; far=1000.0f;
Obj
//object's position in 3d space x,y,z and scaling
objPos(1.0f, -.5f, 5.0f); objScale(.1f,.1f,0.0f);
The problem is how to convert (600,200) screen coordinates (with scaling of (.1f,.1f.0.0f))to the object's (1.0f,-.5f,5.0f) world coordinates to check if they are colliding(I am doing simple box collision, just need the 2d screen coordinates converted to world). I will put the hack that I have below but I am sure there is a better answer. Thank you in advance for any help. Please use the numbers above to show how your algorithm works.
Hack
//this is 0,0 in screen coordinates//-Cam_Offset =hardcoded to camera's pos
float convertedZeroY= (((mCam->right)*(Cam_Offset))); //2
float convertedZeroX= -(((Cam_Offset)+convertedZeroY));
convertedZeroY=convertedZeroY/2;
convertedZeroX=convertedZeroX/2;
//this is the length from top to bottom in game coordinates
float normX=convertedZeroX*2;
float normY=convertedZeroY*2;
//this is the screen coordinates converted to 3d world coordinates
mInput->mUserInputConverted->mPos.x=(-(((mInput->x)/mCam->mScreenW)*normX)+convertedZeroX);
mInput->mUserInputConverted->mPos.y=(-(((mInput->y)/mCam->mScreenH)*(normY))+convertedZeroY);

how to rotate 3d perspective camera for isometric tiled map in andenigne

i am working on isometric tiled map game.here in introduction i want to show complete game field .so i used
this.mCamera = new ZoomCamera(CAMERA_WIDTH, CAMERA_HEIGHT, CAMERA_WIDTH, CAMERA_HEIGHT) {
#Override
public void onApplySceneBackgroundMatrix(final GLState pGLState) {
final float widthRaw = this.getWidthRaw();
final float heightRaw = this.getHeightRaw();
pGLState.orthoProjectionGLMatrixf(0, widthRaw, heightRaw, 0, getZNear(), getZFar());
}
#Override
public void onUpdate(float pSecondsElapsed) {
if (timeCounter >= 1) {
mCamera.setRotation(i);
timeCounter = 0;
i=i+1;
}
timeCounter += pSecondsElapsed;
super.onUpdate(pSecondsElapsed);
}
};
but its rotating 2d view .i want to rotate in 3d perspective .how can i rotate camera in 3d perspective in andenigne GLES2.0.
please suggest me
You cannot rotate your camera in 3D. The appearance of 3D is caused by the fact that the artwork is drawn in perspective. Rotating the artwork does not cause the artwork to change the way its drawn any more than rotating a piece of paper with a drawing on it would cause a 3D transformation. To rotate in 3D you need to be using a 3D engine.
Several times I stumbled upon this tutorial in the AndEngine Forum, where it states that it is possible to rotate the Camera in a way to show distant objects further away (smaller). It is made for the AndEngine GLES1 but it should be possible to adapt it to GLES2.
AndEngine knows a z-axis. The position on that axis is set automatically based on the order in which you attach the Sprites to the Scene. However, you should set it manually. In most cases it will be sufficient to set the z-axis according to the y-position (z-axis = y-axis) every time a Sprite changes its position.
public class YourSprite extends Sprite{
...
#Override
setPosition(float x, float y){
super.setPosition(x,y);
this.setZIndex(y);
}
}
If you then manage to implement the camera rotation as described in the tutorial, together with the z-axis you should have a pretty realistic 3D effect.
However, I never tried that tutorial, because most games that use the Bird's-eye view (as the game from the youtube link you provided) don't need a real vanishing point, since the display is most likely so small, so the player wouldn't notice anyway. So I stick to changing the position on the z-axis. But I would certainly like to know more if anyone manages to rotate the camera!

Categories

Resources