Correctly orient raw camera data - android

My camera app processes raw camera frames from onPreviewFrame() with OpenCV and then displays them on the screen. However, the raw frames are oriented the way the camera is mounted on the phone, which is usually not right-side up. To fix this, I rotate them manually with OpenCV, which is time-consuming.
I've looked into using setDisplayOrientation, but the documentation states that
This does not affect the order of byte array passed in onPreviewFrame
and I need the data to actually be right-side up, not just display right-side up. How can I correctly orient the raw camera data? If this is impossible, can I efficiently rotate the byte array passed to me in onPreviewFrame(), say by using OpenGL?

No, there's no way to get the API to do this for you, but to figure out how you need to rotate the data, take a look at the sample code at Camera.Parameters.setRotation. While setRotation() only affects JPEGs, you want to apply the exact same amount of rotation to the data you get in onPreviewFrame() as you would to the JPEGs.
Reproducing the sample code here, with minor changes:
public void onOrientationChanged(int orientation) {
if (orientation == ORIENTATION_UNKNOWN) return;
android.hardware.Camera.CameraInfo info =
new android.hardware.Camera.CameraInfo();
android.hardware.Camera.getCameraInfo(cameraId, info);
orientation = (orientation + 45) / 90 * 90;
int rotation = 0;
if (info.facing == CameraInfo.CAMERA_FACING_FRONT) {
rotation = (info.orientation - orientation + 360) % 360;
} else { // back-facing camera
rotation = (info.orientation + orientation) % 360;
}
mCameraOrientation = rotation; // Store rotation for later use
}
...
void onPreviewFrame(byte[] data, Camera camera) {
switch(mCameraOrientation) {
case 0:
// data is correctly rotated
break;
case 90:
// rotate data by 90 degrees clockwise
case 180:
// rotate data upside down
case 270:
// rotate data by 90 degrees counterclockwise
}
}
So you need to inherit from OrientationEventListener and override onOrientationChanged as above, and then use the calculated orientation value from there to rotate the preview frames when they come in.

setDisplayOrientation only rotates the camera vs. surface you are drawing into, if I recall correctly.
I believe you want to rotate the camera interpretation, what you may could achieve with setRotation

Related

Nexus 5x reverse landscape sensor fix in a android camera preview app

I am kind of newbie in Android development, so my apologies in advance if my question is trivial. In one part of my app I need a live preview of my rear camera, so I created a custom class which extends SurfaceView and implement SurfaceHolder.Callback (I followed basically the instructions in the android documentation).
Unfortunately, I am testing my app in a Nexus 5x, which I've just realized that it has installed the camera sensor in a reverse way. For that reason, the camera preview of my app when running on my Nexus 5x appears upside down, which is something I dont want.
It seems that the new android.hardware.camera2 API is able to handle this issue automatically. Eventually I'll need to update all my code using this new API, but for now what I need is a quick fix while using the old camera API.
So I was reading out there and I found a piece of code that I would need to introduce in the SurfaceChanged method to workaround this issue. Here it is:
Display display = ((WindowManager)getContext().getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
if(display.getRotation() == Surface.ROTATION_0)
{
parameters.setPreviewSize(capHeight, capWidth);
camera.setDisplayOrientation(90);
}
if(display.getRotation() == Surface.ROTATION_90)
{
parameters.setPreviewSize(capWidth, capHeight);
}
if(display.getRotation() == Surface.ROTATION_180)
{
parameters.setPreviewSize(capHeight, capWidth);
}
if(display.getRotation() == Surface.ROTATION_270)
{
parameters.setPreviewSize(capWidth, capHeight);
camera.setDisplayOrientation(180);
}
camera.setParameters(parameters);*/
camera.startPreview();
The problem is that I don't see that something has changed.
Any thoughts?
The 5X camera is not "reverse"; it does report the correct camera orientation in its parameters, so no special workarounds are needed, just make sure you're setting the display orientation correctly:
public static void setCameraDisplayOrientation(Activity activity,
int cameraId, android.hardware.Camera camera) {
android.hardware.Camera.CameraInfo info =
new android.hardware.Camera.CameraInfo();
android.hardware.Camera.getCameraInfo(cameraId, info);
int rotation = activity.getWindowManager().getDefaultDisplay()
.getRotation();
int degrees = 0;
switch (rotation) {
case Surface.ROTATION_0: degrees = 0; break;
case Surface.ROTATION_90: degrees = 90; break;
case Surface.ROTATION_180: degrees = 180; break;
case Surface.ROTATION_270: degrees = 270; break;
}
int result;
if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
result = (info.orientation + degrees) % 360;
result = (360 - result) % 360; // compensate the mirror
} else { // back-facing
result = (info.orientation - degrees + 360) % 360;
}
camera.setDisplayOrientation(result);
}
From http://developer.android.com/reference/android/hardware/Camera.html#setDisplayOrientation%28int%29
if (Build.MODEL.equals("Nexus 5X")){
// rotate camera 180°
mCamera.setDisplayOrientation(180);
}
Change 5x to 5X will be ok.
Same here.
Since camera2 doesn't support devices < API 21, and don't want to implement the two APIs, it would be nice do get a quick fix.
I'm trying somthing like :
if (Build.MODEL.equals("Nexus 5x")){
// rotate camera 180°
mCamera.setDisplayOrientation(180);
}
But can't rotate the display of the camera. Gonna check if there is something to do with the PreviewCallback
Is it a bad thing to use the 2 apis ?
As in this topic, it is not recommended :
How to use Android's camera or camera2 API to support old and new API versions without deprecation notes?

Android Camera Rotation Generic

Im trying to rotate my camera when the device rotates, iv tried several methods, but none of them seem to work on all devices.
//This Works on my nexus 5 but not on my Samsung galaxy tab
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height)
{
if (mHolder.getSurface() == null)
{
return;
}
try
{
mCamera.stopPreview();
Camera.Parameters parameters = mCamera.getParameters();
parameters.set("orientation", "landscape");
int rotation = ((WindowManager) getContext().getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay().getRotation();
int orientation = getContext().getResources().getConfiguration().orientation;
if (rotation == Surface.ROTATION_0) {
if (orientation == Configuration.ORIENTATION_LANDSCAPE) {
mCamera.setDisplayOrientation(0);
} else {
mCamera.setDisplayOrientation(90);
}
}
else if (rotation == Surface.ROTATION_90) {
if (orientation == Configuration.ORIENTATION_PORTRAIT) {
mCamera.setDisplayOrientation(270);
}
}
else if (rotation == Surface.ROTATION_180) {
if (orientation == Configuration.ORIENTATION_LANDSCAPE) {
mCamera.setDisplayOrientation(180);
}
}
else if (rotation == Surface.ROTATION_270) {
if (orientation == Configuration.ORIENTATION_PORTRAIT) {
mCamera.setDisplayOrientation(90);
}
}
mCamera.setPreviewDisplay(holder);
mCamera.startPreview();
}
catch (Exception e) {
}
}
// and i tried this, which is part of the documentation and it doesnt work on my nexus 5:
public static void setCameraDisplayOrientation(Activity activity,
int cameraId, android.hardware.Camera camera) {
android.hardware.Camera.CameraInfo info =
new android.hardware.Camera.CameraInfo();
android.hardware.Camera.getCameraInfo(cameraId, info);
int rotation = activity.getWindowManager().getDefaultDisplay()
.getRotation();
int degrees = 0;
switch (rotation) {
case Surface.ROTATION_0: degrees = 0; break;
case Surface.ROTATION_90: degrees = 90; break;
case Surface.ROTATION_180: degrees = 180; break;
case Surface.ROTATION_270: degrees = 270; break;
}
int result;
if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
result = (info.orientation + degrees) % 360;
result = (360 - result) % 360; // compensate the mirror
} else { // back-facing
result = (info.orientation - degrees + 360) % 360;
}
camera.setDisplayOrientation(result);
}
Does someone have a working solution for all devices?
Although I'm not exactly sure what's your issue from the code you provided, here are few things one has to got right when using Camera API and image rotation on Android (the old one - on API <21)
1. You have to take in account the orientation of the image sensor.
The orientation of the sensor means its angle to the device native use-mode. (that's what actually does the google snippet you are using) For phones this is usually being set to portrait, on tablets to landscape.
Here is an excerpt from the documentation, it can make pain in head for a while, but that's just how things work.
For example, suppose a device has a naturally tall screen. The back-facing camera sensor is mounted in landscape. You are looking at the screen. If the top side of the camera sensor is aligned with the right edge of the screen in natural orientation, the value should be 90. If the top side of a front-facing camera sensor is aligned with the right of the screen, the value should be 270.
2. Frontfacing camera preview is being mirrored
If you use frontfacing camera, you have to take in account the implicit mirroring.
3. These rules are only meant for rotating the Camera preview
This is important. All this is only meant to allow you to display correctly rotated preview frames on the device screen. Once you take an actual picture, you have to rotate it once again to suit your needs. For front-facing images this might mean mirroring it yourself,...You can see the result of this "weirdness" among all apps on Android. Just take picture in hangouts and share it right away. On various phones you gotta get somewhat weirdly rotated image as the result.
4. OEMs does not adhere to specs all the time
All I can say, is that probably no solution will be hasslefree, we got some issues with some particular devices. (as usual on Android:)
5. Use the CWAC Camera library
We found this library to be a superior wrapper around the official Camera APIs. Solving practically all of the issues I did mention.

Android camera image orientation without sufaceView

I am having a service in which I have to capture image without a surfaceView, everything is working perfect except the result image orientation, which I found to be miss-angled. On small device like HTC, I found it having issue or rotation so set rotation manually to see it working and it worked.
if (camInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
parameters.setRotation(270);
} else if (camInfo.facing ==
Camera.CameraInfo.CAMERA_FACING_BACK) {
parameters.setRotation(90);
}
But when checking over samsung and HTC one (large devices) it creates problem with the angle. I found some posts where there I have to put the image path and then try to set the rotation, but that didn't work for me i.e. this as I am taking picture without serfaceview and then immediately posting it to the server. I also tried the google portion of code for setCameraOrientation() but it requires the activity view to work and so I am failed there too.
All I need is to fix the angle of the image before sent to the server without any surfaceview or activity thing.
setRotation() may only choose to play with the EXIF flags. The result is an image which is still rotated 90°, but with a "flag" that describes the correct orientation. Not all viewers correctly account for this flag. Specifically, BitmapFactory ignores it. You can draw the bitmap rotated on a canvas, or rotate the bitmap acquired from BitmapFactory.decodeFile(), or manipulate the JPEG data before writing it into outStream using a 3rd party lib, e.g. MediaUtil. The Android port is on GitHub.
You can have access to the image orientation information through the ExifInterface object. It will give you different values depending on the phone and if you capture the image in Landscape or Portrait mode. You can then use a matrix to rotate the image according to the ExifInterface information.And finally send it to your server.
Knowing the path of your image ( imagePath ), use the following code:
Matrix matrix = new Matrix();
try{
ExifInterface exif = new ExifInterface(imagePath);
int orientation = exif.getAttributeInt(
ExifInterface.TAG_ORIENTATION,
ExifInterface.ORIENTATION_NORMAL);
switch (orientation) {
case ExifInterface.ORIENTATION_ROTATE_90:
// Change the image orientation
matrix.postRotate(90);
break;
case ExifInterface.ORIENTATION_ROTATE_180:
// Change the image orientation
matrix.postRotate(180);
break;
}catch (IOException e) {
e.printStackTrace();
}
Then you use your matrix object in order to rotate your bitmap using:
rotatedBitmap = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(),matrix, true);
You would have to save the bitmap somewhere ( external or internal temp storage ) before sending it to the server.
Hope it helps you.
Your rotation-setting code is a bit too simple, so it's possible that on some devices it'll do the wrong thing. It's not guaranteed that these are the right rotations - the correct answer is a function of your device's current orientation, and the orientation of the sensor on the device.
Take a look at the sample code for Camera.Parameters.setRotation:
public void onOrientationChanged(int orientation) {
if (orientation == ORIENTATION_UNKNOWN) return;
android.hardware.Camera.CameraInfo info =
new android.hardware.Camera.CameraInfo();
android.hardware.Camera.getCameraInfo(cameraId, info);
orientation = (orientation + 45) / 90 * 90;
int rotation = 0;
if (info.facing == CameraInfo.CAMERA_FACING_FRONT) {
rotation = (info.orientation - orientation + 360) % 360;
} else { // back-facing camera
rotation = (info.orientation + orientation) % 360;
}
mParameters.setRotation(rotation);
}
If you don't have an activity, you'll have to figure out how to get the device's current orientation some other way, but you do need to include info.orientation in your calculation.

Android camera rotation for portrait mode pictures

I'm trying to make a custom camera app. I'm taking pictures from the front facing camera without preview, but the resulting images are rotated according to the front camera orientation.
I searched through a lot of post here at stack, also the one described here in the android docs.
This is the solution that I came up with:
private void prepareCamera(Activity a){
//Rotation
Camera.CameraInfo info = new Camera.CameraInfo();
Camera.getCameraInfo(Camera.CameraInfo.CAMERA_FACING_FRONT,info);
int rotation = a.getWindowManager().getDefaultDisplay().getRotation();
int degrees = 0;
Log.d("TEST","rotation: "+rotation);
switch(rotation){
case Surface.ROTATION_0: degrees = 0; break; //Natural orientation
case Surface.ROTATION_90: degrees = 90; break; //Landscape left
case Surface.ROTATION_180: degrees = 180; break; //Upside down
case Surface.ROTATION_270: degrees = 270; break; //Landscape right
}
Log.d("TEST","degrees: "+degrees);
int result = (info.orientation + degrees) % 360;
Log.d("TEST","result: "+result);
result = (360 - result) % 360; //Compensate the mirror
Log.d("TEST","result: "+result);
//camera.setDisplayOrientation(result);
//Parameters
Camera.Parameters params = camera.getParameters();
params.setRotation(result);
params.setJpegQuality(100);
camera.setParameters(params);
//Preview
SurfaceView view = new SurfaceView(context);
try{
camera.setPreviewDisplay(view.getHolder());
}catch(IOException e){
throw new RuntimeException(e);
}
camera.startPreview();
}
It works, but I have two problems
1) I have the line, wich I took from the android docs, that "compensates the mirror". If I don't delete that line the pictures are well rotated according to the camera orientation, but they are overpassed by 180º. If I delete it the pictures are fine. I want to understand the why, if someone can please explain me.
I think its because that example code that I took from the docs its made for the SurfaceView, and it calls setDisplayOrientation(result);, when I'm using the results in .setRotation(result);. Maybe it has something to do with it?
2) When I'm testing I can rotate my phone to all directions but Upside down and get the pictures well rotated. But when I put the phone upside down, they are rotate 90, but upside down. As they were processed like Natural orientation. Maybe android wont track upside down rotation? If so, why would they with a case in the example provided by them.
Please look at setRotation() sample:
public void onOrientationChanged(int orientation) {
if (orientation == ORIENTATION_UNKNOWN) return;
android.hardware.Camera.CameraInfo info =
new android.hardware.Camera.CameraInfo();
android.hardware.Camera.getCameraInfo(cameraId, info);
orientation = (orientation + 45) / 90 * 90;
int rotation = 0;
if (info.facing == CameraInfo.CAMERA_FACING_FRONT) {
rotation = (info.orientation - orientation + 360) % 360;
} else { // back-facing camera
rotation = (info.orientation + orientation) % 360;
}
mParameters.setRotation(rotation);
}
Display.getRotation(), unlike orientation sensor, is limited to orientations that your current Activity supports. On some devices, the portrait upside-down orientation is not supported by system.

Is the setDisplayOrientation sample code correct?

The documentation page for Camera.setDisplayOrientation contains the following code sample stating that using it will "make the camera image show in the same orientation as the display":
public static void setCameraDisplayOrientation(Activity activity,
int cameraId, android.hardware.Camera camera) {
android.hardware.Camera.CameraInfo info =
new android.hardware.Camera.CameraInfo();
android.hardware.Camera.getCameraInfo(cameraId, info);
int rotation = activity.getWindowManager().getDefaultDisplay()
.getRotation();
int degrees = 0;
switch (rotation) {
case Surface.ROTATION_0: degrees = 0; break;
case Surface.ROTATION_90: degrees = 90; break;
case Surface.ROTATION_180: degrees = 180; break;
case Surface.ROTATION_270: degrees = 270; break;
}
int result;
if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
result = (info.orientation + degrees) % 360;
result = (360 - result) % 360; // compensate the mirror
} else { // back-facing
result = (info.orientation - degrees + 360) % 360;
}
camera.setDisplayOrientation(result);
}
However, I had problems using it, sometimes the image would appear upside down. After some trial-and-error, I found that the correct code would be (replacing the last 8 lines of the method):
int result = (360 + info.orientation - degrees) % 360;
camera.setDisplayOrientation(result);
(Which means the calculation for back-facing cameras is correct for front cameras too.) The "compensate the mirror" comment is a bit weird since mirroring cannot be undone by rotating, that operation only swaps 90° and 270° rotations which doesn't really make sense to me.
So the question is: is the sample wrong indeed or am I missing something? I tried it on multiple devices, both back and front cameras and all supported orientations, so I know that my code IS working. One little detail that might be worth mentioning: all my devices returned 90° as info.orientation.
EDIT: Here is my camera-related code, I have tested it on a Nexus One and a Samsung Galaxy S Plus. It is used in my head-tracking 3D app, the preview is shown in the lower left corner and should always have the correct orientation.
SOLUTION (sort of): It looks like the code is correct, but my testing phone (Samsung Galaxy S Plus) returns an incorrect value for the front camera's CameraInfo.orientation. There are many related discussions about the preview being shown upside down on this model (examples here and here). A way to fix is to include an option to manually rotate the image.
The snippet you've quoted, which I used and applied in my project, is no problem in my circumstances.
For all the devices (Galaxy Note, Galaxy S III, Galaxy Nexus, Galaxy Ace II, Nexus S) I used for test, info.Orientation all return 270 on front camera, and 90 on back camera.
After a few discuss with the question raiser, I found I misunderstood the questions, so I divide the answer in two parts.
For the wrong orientation in camera preview, please refer to this solution:
Solution for wrong orientation in camera preview:
First please ensure info.Orientation will return 270 on front camera, 90 on back camera.
Then please try set your camera preview activity (or similar class that handles the preview) orientation to landscape.
Thus, when you go through the code, you'll find:
degree = 90 for screen orientation, info.Orientation = 270 for the front camera. Then you'll get result = (270 - 90 + 360) % 360, result = 180, which means it will rotate clock wise 180 for your front camera view that will correct the front camera upside-down issue.
Solution for wrong orientation in camera photo results:
If this info.Orientation applies to you, then the problem may be:
For some Samsung (or other) devices (Like Galaxy Note, Galaxy SIII), the camera will only write the Orientation Tag instead of rotate the real pixels.
If you're using the front camera and using the code above, it will display the preview with correct orientation, but will show u the "upside down" picture if you take the shot.
Solution:
/**
*
* Get the orientation from EXIF
 * #param filepath
 * #return orientation
 */
public int getExifOrientation(String filepath) {
int degree = 0;
ExifInterface exif = null;
try {
exif = new ExifInterface(filepath);
} catch (IOException ex) {
Log.e("EXIF info", "cannot read exif", ex);
}
if (exif != null) {
int orientation = exif.getAttributeInt(ExifInterface.TAG_ORIENTATION, -1);
if (orientation != -1) {
// We only recognize a subset of orientation tag values.
switch(orientation) {
case ExifInterface.ORIENTATION_ROTATE_90:
degree = 90;
break;
case ExifInterface.ORIENTATION_ROTATE_180:
degree = 180;
break;
case ExifInterface.ORIENTATION_ROTATE_270:
degree = 270;
break;
}
}
} else {
degree = 1;
}
Log.i("EXIF info", "Orientation degrees: " + degree);
return degree;
}
Then
if (isFromCamera) {
if (fromFrontCam) {
// try change here if the orientation still wrong, -90 means rotate counter-clockwise 90 degrees.
matrix.preRotate(-90);
} else {
matrix.preRotate(90);
}
} else {
// read EXIF data
getExifOrientation(path)
// don't forget to handle the situation that some devices won't write exif data
matrix.preRotate(degree);
}

Categories

Resources