Develop Reference

AndroidCamera2 for face detection and distance measurement

I'm building a camera that needs to detect the user's face/eyes and measure distance through the eyes.
I found that on this project https://github.com/IvanLudvig/Screen-to-face-distance, it works great but it doesn't happen to use a preview of the frontal camera (Really, I tested it on at least 10 people, all measurements were REALLY close or perfect).
My app already had a selfie camera part made by me, but using the old camera API, and I didn't find a solution to have both camera preview and the face distance to work together on that, always would receive an error that the camera was already in use.
I decided to move to the camera2 to use more than one camera stream, and I'm still learning this process of having two streams at the same time for different things. Btw documentation on this seems to be so scarce, I'm really lost on it.
Now, am I on the right path to this?
Also, on his project,Ivan uses this:
Camera camera = frontCam();
Camera.Parameters campar = camera.getParameters();
F = campar.getFocalLength();
angleX = campar.getHorizontalViewAngle();
angleY = campar.getVerticalViewAngle();
sensorX = (float) (Math.tan(Math.toRadians(angleX / 2)) * 2 * F);
sensorY = (float) (Math.tan(Math.toRadians(angleY / 2)) * 2 * F);
This is the old camera API, how can I call this on the new one?
Judging from this answer: Android camera2 API get focus distance in AF mode
Do I need to get min and max focal lenghts?
For the horizontal and vertical angles I found this one: What is the Android Camera2 API equivalent of Camera.Parameters.getHorizontalViewAngle() and Camera.Parameters.getVerticalViewAngle()?
The rest I believe is done by Google's Cloud Vision API
EDIT:
I got it to work on camera2, using GMS's own example, CameraSourcePreview and GraphicOverlay to display whatever I want to display together the preview and detect faces.
Now to get the camera characteristics:
CameraManager manager = (CameraManager) this.getSystemService(Context.CAMERA_SERVICE);
try {
character = manager.getCameraCharacteristics(String.valueOf(1));
} catch (CameraAccessException e) {
Log.e(TAG, "CamAcc1Error.", e);
}
angleX = character.get(CameraCharacteristics.SENSOR_INFO_PHYSICAL_SIZE).getWidth();
angleY = character.get(CameraCharacteristics.SENSOR_INFO_PHYSICAL_SIZE).getHeight();
sensorX = (float)(Math.tan(Math.toRadians(angleX / 2)) * 2 * F);
sensorY = (float)(Math.tan(Math.toRadians(angleY / 2)) * 2 * F);
This pretty much gives me mm accuracy to face distance, which is exactly what I needed.
Now what is left is getting a picture from this preview with GMS's CameraSourcePreview, so that I can use later.
Final Edit here:
I solved the picture issue, but I forgot to edit here. The thing is, all the examples using camera2 to take a picture are really complicated (rightly so, it's a better API than camera, has a lot of options), but it can be really simplyfied to what I did here:
mCameraSource.takePicture(null, bytes -> {
Bitmap bitmap;
bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
if (bitmap != null) {
Matrix matrix = new Matrix();
matrix.postRotate(180);
matrix.postScale(1, -1);
rotateBmp = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(),
bitmap.getHeight(), matrix, false);
saveBmp2SD(STORAGE_PATH, rotateBmp);
rotateBmp.recycle();
bitmap.recycle();
}
});
That's all I needed to take a picture and save to a location I specified, don't mind the recycling here, it's not right, I'm working on it

It looks like that bit of math is calculating the physical dimensions of the image sensor, via the angle-of-view equation:
The camera2 API has the sensor dimensions as part of the camera characteristics directly: SENSOR_INFO_PHYSICAL_SIZE.
In fact, if you want to get the field of view in camera2, you have to use the same equation in the other direction, since FOVs are not part of camera characteristics.
Beyond that, it looks like the example you linked just uses the old camera API to fetch that FOV information, and then closes the camera and uses the Vision API to actually drive the camera. So you'd have to look at the vision API docs to see how you can give it camera input instead of having it drive everything. Or you could use the camera API's built-in face detector, which on many devices gives you eye locations as well.

Face not detected using vision google service

https://developers.google.com/android/reference/com/google/android/gms/vision/face/FaceDetector.Builder
I'm using the above google service in my app for face detection. I made sure my phone has the minimum google play service version, which on my phone is 8.3, but still I can't get the face detection to work! I imported the library by importing the google play library in my eclipse project.... Here's the code:
#Override
protected void onPreExecute()
{
detector = new FaceDetector.Builder(MainContext)
.setTrackingEnabled(false)
//.setProminentFaceOnly(true)
.setLandmarkType(FaceDetector.ALL_LANDMARKS) //required
.build();
}
private void detectTheFace(Bitmap converted)
{
Frame frame = new Frame.Builder().setBitmap(converted).build();
faces = detector.detect(frame);
}
I don't know if it's necessary to convert the bitmap you use to detect the faces has to be RGB_565 configuration but I did it anyways. I tried with and without changing the RGB configuration and it yields the same results. Basically the faces sparse array is of size 0 meaning it doesnt detect a face.... ever. Btw just to give some context on the above code, I'm executing the face detection in a async task because I want to run it on the background.

I have the same problem ,i.e. it was working fine on nexus but not in galaxy. I resolved the problem by rotating the bitmap to 90 degree in case detector.detect() method gives faces of zero size. so maximum retry is 3 times after calling detector.detect() because 4th rotation gives you the same bitmap.
Bitmap rotateBitmap(Bitmap bitmapToRotate) {
Matrix matrix = new Matrix();
matrix.postRotate(90);
Bitmap rotatedBitmap = Bitmap.createBitmap(bitmapToRotate, 0, 0,
bitmapToRotate.getWidth(), bitmapToRotate.getHeight(), matrix,
true);
return rotatedBitmap;
}
check if the face return by detector.detect() has zero size then below code should run.
if(!faces.size()>0){
if (rotationCounter < 3) {
rotationCounter++;
bitmap= rotateBitmap(bitmapToRotate);
//again call detector.detect() here
}
}
you can check for the need of rotating bitmap without writing above code. From your original, code try to capture the image in landscape mode of phone or just rotate the image to 90 degree ans capture it.

To solve this problem, use the orientation specification from the EXIF of the photo.

Android: Rotating large image crashes without throwing an error

I have searched and found simple code to rotate an image. I am pulling the image out of an ImageView object into a bitmap, rotating it then putting it back. I realize this is not the most effective method but I don't think it should crash without giving an error message in the CATCH block.
Here is my code. The only value passed in is "r" or "l" depending on which direction I want to rotate. Smaler images (1500x1500 or smaller) work just fine. Things go bad around the 2500x2500 size.
public void rotate(String dir)
{
try
{
float angle = (dir.equals("r") ? 90 : -90);
Bitmap bitmap = ((BitmapDrawable) imageView.getDrawable()).getBitmap();
Matrix matrix = new Matrix();
matrix.reset();
matrix.postRotate(angle);
bitmap = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, false);
imageView.setImageBitmap(bitmap);
}
catch(Exception e)
{
Utilities.logError(e.toString());
}
}
Any clue as to why it is crashing and why it doesn't thow an exception? I just get a message "Unfortuantly process .... has stopped" and I get kicked back to the welcome screen of my app.
Oh, for kicks I set the angle to ZERO (hard coded) and it didn't crash. I suspect that it is just taking too long to rotate and Android is having a fit. But I am not sure how to confirm that as the problem or how to tell Android to wait a little longer.
Even if I reduce the preview image for the rotation, when I go to save I will have to rotate the full size image at least once and will hit this same issue. Won't I?

I can more or less guarantee without looking at the logs that you're getting an Out Of Memory Exception.
You need to use smaller images, or use a different method to rotate that doesn't use up so much memory (you're allocating 2 2500x2500 bitmaps at the same time here! that's tons!).
Try using a RotateAnimation to get your effect instead.
Hope this helps :)

Rotating an image bitmap with animation android

In my app I am giving a functionality to edit an image from the filepath.
Now I want to add a feature to change the image orientation with animation and then save it.
I am able to change the orientation but don't know how to add animation to it.
Here's my code.
ImageView img;
Bitmap bmp;
Bitmap rotatedBMP;
String imageFilePath="";
File imgFile;
public void rotate(){
if(rotatedBMP == null){
if(imgFile != null && imgFile.exists())
bmp = BitmapFactory.decodeFile(imgFile.getAbsolutePath());
}else
bmp = rotatedBMP;
// Getting width & height of the given image.
int w = bmp.getWidth();
int h = bmp.getHeight();
Matrix mtx = new Matrix();
mtx.preRotate(90);
rotatedBMP = Bitmap.createBitmap(bmp, 0, 0, w, h, mtx, true);
/*BitmapDrawable bmd = new BitmapDrawable(rotatedBMP);
img.setImageDrawable(bmd);*/
img.setImageBitmap(rotatedBMP);
}

There's a conflict between your desire to animate rotate a view, and your desire to rotate and save the actual bitmap.
If you want to rotate the image view, the good way to do that is to rotate the canvas in an onDraw method with an animation update step, or to use Android's helper for that like RotateAnimation. Drop us a comment if you want examples.
That's the "good way" because it doesn't create a new bitmap on every animation frame, which would be very expensive.
However, you've got a conflict that you also want to rotate the bitmap itself, rather than just the view, so as to save it. This is fine given it's not on every animation step.
You consequently will have to save the user's chosen transformations you want to apply to the bitmap as late as possible. Then animate the view however is appropriate in response to the user. On the save, you'd concat all the users transformations on writing to the file. Or give up on the idea of animating it.
My implementation idea would be to store a second matrix locally and have each user input both update the canvas transforms and that matrix. You'd then 'get hold' of the bitmap as it were when the save button is pressed and apply it to the updated transformation matrix.
Happy to elaborate as requested.
Some sources
Android: Rotate image in imageview by an angle (mentions issues with creating many bitmaps too)
Rotating a view in Android (gauravjain0102's answer is underrated!)

setRotation(90) to take picture in portrait mode does not work on samsung devices

According to the documentation, setRotation(90) should rotate the captured JPEG picture (takePicture in landscape mode.
This works fine on a HTC phone, but does not work on Samsung Google Nexus S and Samsung Galaxy S3. Is this a bug?
I know that I can use the matrix transform rotation, but wish the OS can do this more efficiently, and don't want to risk over-rotating on some other devices.
edit
Setting camera.setDisplayOrientation(90); made the preview to be in portrait mode, however it did not have any affect on the picture taken.
Further, Besides setRotation, I have also tried to set the picture size - where I flip h with w: parameters.setPictureSize(1200, 1600);. This also did not have any affect.
solution
Apparently Samsung phones set the EXIF orientation tag, rather than rotating individual pixels. As ariefbayu suggested, reading the Bitmap using BitmapFactory does not support this tag. His code sample is the solution, and this solution is also compatible with using inSampleSize.

I try to answer this in relation to the Exif tag. This is what I did:
Bitmap realImage = BitmapFactory.decodeStream(stream);
ExifInterface exif=new ExifInterface(getRealPathFromURI(imagePath));
Log.d("EXIF value", exif.getAttribute(ExifInterface.TAG_ORIENTATION));
if(exif.getAttribute(ExifInterface.TAG_ORIENTATION).equalsIgnoreCase("6")){
realImage=ImageUtil.rotate(realImage, 90);
}else if(exif.getAttribute(ExifInterface.TAG_ORIENTATION).equalsIgnoreCase("8")){
realImage=ImageUtil.rotate(realImage, 270);
}else if(exif.getAttribute(ExifInterface.TAG_ORIENTATION).equalsIgnoreCase("3")){
realImage=ImageUtil.rotate(realImage, 180);
}
The ImageUtil.rotate():
public static Bitmap rotate(Bitmap bitmap, int degree) {
int w = bitmap.getWidth();
int h = bitmap.getHeight();
Matrix mtx = new Matrix();
mtx.postRotate(degree);
return Bitmap.createBitmap(bitmap, 0, 0, w, h, mtx, true);
}

Liso22, the variable stream doesn't make a difference, just plug in your bitmap, however you got it (decodeFile etc). And if you're having problems with the 'ImageUtil.rotate(), just write the 'public static Bitmap rotate()' as a method with the same parameters and make 'real image' equal that. Anyways, this solution doesn't seem to be working for me, the exif tag returns 1 (normal) whether in portrait or landscape.