Although Camera API is deprecated, it's simpler to get started than Camera2. But the document seems not to be that easy to understand(with my reading ability). I've searched a lot trying to figure it out. But there're still bunch of questions.
1.What is camera's orientation? Is there any definition that I miss? There're 4 dimensions(left, top, right, bottom) of camera. When we say orientation, which one is used and what is the relative object?
2.Why it's different between camera's orientation and device's?
3.Could you please tell me how to understand the algorithm given in the document of Camera.setDisplayOrientation() and Camera.parameters.setRotation()? There're sample code given by document for these two methods. These two algorithm both use sum and difference of orientation/rotation. I may use it directly in my project. But what does that mean exactly?
Besides, taking back-facing camera as example, the former uses (info.orientation - degrees + 360) % 360, while the latter uses (info.orientation + orientation) % 360. Confusing:/
Thanks if you can answer any of these questions above or just share some of your thoughts and experiences.
1. it is the how the camera is orientated in your device
2. Device orientation is the current orientation/rotation of your device (landscape/portrait )
3. Camera.setDisplayOrientation()
affects the preview
Camera.parameters.setRotation()
affects the result pictures (jpeg etc.)
Documentation:
Camera.setDisplayOrientation()
Set the clockwise rotation of preview display in degrees. This affects
the preview frames and the picture displayed after snapshot. This
method is useful for portrait mode applications. Note that preview
display of front-facing cameras is flipped horizontally before the
rotation, that is, the image is reflected along the central vertical
axis of the camera sensor. So the users can see themselves as looking
into a mirror.
Camera.parameters.setRotation()
Sets the clockwise rotation angle in degrees relative to the orientation of the camera. This affects the pictures returned from JPEG Camera.PictureCallback. The camera driver may set orientation in the EXIF header without rotating the picture. Or the driver may rotate the picture and the EXIF thumbnail. If the Jpeg picture is rotated, the orientation in the EXIF header will be missing or 1 (row #0 is top and column #0 is left side).
Related
I want to horizontally invert frames coming to Surface object made from TextureView. What I can do is to set transformation matrix to this TextureView instance, whereas I postScale by -1 for x and 1 for y (leave unchanged), and than postTranslate dx on the full width of the view and leave dy unchanged (0F).
But the problem appears when I rotate my device by 90 degrees (horizontally) with screen rotation off: the image is rotated by 180 degrees, and it, of course, makes perfect sense, because x and y axises did not change.
How it can be solved? Is it possible to play with the translation matrix in such a way to resolve this problem? Or may be with OpenGL ES tools?
P.S.: strangely, but rotation is done two times two times more comparing to the device rotation itself, e.g. when I rotate the device by 90 degrees - preview is rotated by 180.
P.S.S.: I tried to invert the preview using Matrix's setPolyToPoly method... and have got exactly the same result.
P.S.S.S.: Also, played with open gl to achieve the goal using simple scale and rotation transformations for the model and projection matricies, and have got exact the same result!
Update:
These are screenshots that describe default behavior of the front camera - frame inversion is applied by HAL by default and I can't read the text; still, whether I rotate the device or not - "frame orientation" does not change:
And these are screenshots when I apply, e.g., Matrix.scaleM(modelMatrix, 0, -1F, 1F, 1F); and then apply this matrix to every coordinate that comes into vertex shader, so I can now read the text because I applied the inversion myself so HAL's inversion with my custom inversion will result into inversion absence, but when I rotate my device (with device orientation change on rotation disabled, of course, and that's the point) - I'll see myself flipped upside down, and that's, of course, make perfect sense, because device's coordinate system won't change. Still, I want to be able to avoid image rotation somehow on device rotation itself (like in the default mode - whether the device is rotated or not - the preview image just "does not care"), and to still be able to read the text (I mean, like in portrait mode).
Do you run your tests on front or back camera? Usually only the front camera frame needs to be flipped horizontally, not the back camera.
Anyway if you need different transformations based on device orientation, you need to detect orientation changes in your Activity. Camera doesn't do anything by itself when device is rotated, you will get the same results whatever transformation method you choose.
Update #2
I just realised that this problem can be solved if transformation is applied at the right moment in the pipeline. I will show a working example, it uses OpenGL extensively but I hope you can find something of use. See this code fragment, it uses a vertex shader to transform the coordinates. In drawFrame() there's a call to getTransformMatrix(). Instead of performing this call, get a matrix from this method I wrote when performing a similar task.
UPDATE : question is about Camera2
I'm trying to figure out who applies the rotation transform when a camera preview is drawn on a SurfaceTexture.
When requesting the preview sizes from the camera you always get pairs where the width is larger than the height (because landscape is the usual orientation when taking a picture).
When using the device on portrait mode and setting the preview size (for ex. 1600 x1200) the frames from the camera are correctly rotated but I can't find out where it's done, is it something CameraDevice does automatically based on Surface type ? or is it the SurfaceTexture who rotates the image?
The answer depends a bit on which camera API you're using; since you mention CameraDevice, I assume you mean camera2.
In camera2, one of the goals was to simplify the rotation handling where possible; so if you draw preview into a SurfaceView, the camera service and the hardware compositor / GPU cooperate and handle all the rotation for you. You always request landscape resolutions from the sensor (which is aligned with the long edge of the sensor matching the long edge of the device), and you just need to make sure your SurfaceView's aspect ratio matches that of the requested resolution (so in portrait, you need 9:16 View to display 1080p correctly; in landscape you need a 16:9 View).
For TextureView, and completely custom GL drawing with SurfaceTexture, the API can't do it all for you. The SurfaceTexture's transform is set by the camera API to rotate the camera output to the device's native orientation (generally portrait for phones, landscape for some tablets), but it can't handle the additional rotation if your app's UI is not in the native orientation.
In that case, you'll need to get the transform matrix from the SurfaceTexure, apply the extra rotation to match your app UI, and set that as the TextureView matrix.
It's been about a year since I worked on Camera stuff, so I apologize for a lack of specifics. But I do remember that the key when displaying the Camera preview was calculating the correct transform matrix to apply to the SurfaceView (or TextureView) to apply any scaling or rotation that is needed. So regarding "who is doing the rotation" I guess you can say it is the view, as instructed by the Transform that you supply. So the matrix will be based on the relationship of the preview resolution you request (from the list of what's capable for your device) compared to the actual laid out dimensions of your view, plus the physical orientation of the device sensor as compared to the current orientation of the device screen.
In the old camera API, Camera.CameraInfo has an orientation field that supplies the physical orientation with which the sensor is mounted. It's typically 90 for a landscape mounted sensor on most phones, but you'll also find some 270s out there (effectively "upside down"). 0 and 180 may exist but I've never seen them.
Why in the portrait screen orientation the camera surface is rotated by -90 degrees "90 degrees anticlockwise". I want the camera preview to be normal regardless of the screen orientation. How can I achieve that.
Did you tried this:
mCam.setDisplayOrientation(90); use it inside surfaceChanged(...)
setDisplayOrientation
Set the clockwise rotation of preview display in degrees. This affects the preview frames and the picture displayed after snapshot. This method is useful for portrait mode applications. Note that preview display of front-facing cameras is flipped horizontally before the rotation, that is, the image is reflected along the central vertical axis of the camera sensor. So the users can see themselves as looking into a mirror.
This does not affect the order of byte array passed in onPreviewFrame(byte[], Camera), JPEG pictures, or recorded videos. This method is not allowed to be called during preview.
Nexus 4.
I specially use System.out.println to print out the rotation on LogCat, to observe the value when I rotate my device.
int orientation_photo = ((WindowManager) mContext.getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay().getRotation();
System.out.println("For Debug: orientation_photo: " + orientation_photo);
The rotation (0, 1, 2 or 4) is not prcise at all. Sometimes I have rotated 90 degrees, but the rotation value does not change.
However, believe Nexus 4 device can detect it correctly, from the orientation performance of its default camera, which always record a correct orientation, so that we can view pictures in a correct way.
I need a correct rotation value, so that I could put in ExifInterface of the photo .jpg file.
Any help?
You are fetching the orientation of the UI, not the orientation of the phone itself. For various reasons they do not always match. In particular, many devices will never rotate their UI 180 degrees.
You probably want to look at the SensorManager class: http://developer.android.com/reference/android/hardware/SensorManager.html
I think I find the answer, that is: Yes, it is not precise, from the basic physical principle, because the orientation is subjective.
In my camera app, the UI orientation is fixed to portrait.
the preview displayed correctly on both landscape and portrait by setting camera.setdisplayorientation(90).
I can take picture and save it to file.
when I display it by ImageView,
the picture that was taken in landscape orientation could display correctly.
(the top of phone in my left side)
But... the one that was taken in portrait orientation is not so lucky...
It looks like turn 90 degrees to left.
I try to detect the device orientation through sensor so that I can set the EXIF header,
but... so far...that too hard to achieve for me...does any one can help me to solve this problem?
Use the getRotation method:
Display display = ((WindowManager)
context.getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
int rotation = display.getRotation();
From the documentation:
Returns the rotation of the screen from its "natural" orientation.
The returned value may be Surface.ROTATION_0 (no rotation), Surface.ROTATION_90, Surface.
ROTATION_180, or Surface.ROTATION_270.
For example, if a device has a naturally tall screen, and the user has turned it on its side to go into a landscape orientation, the value returned here may be either Surface.ROTATION_90 or Surface.ROTATION_270 depending on the direction it was turned.
The angle is the rotation of the drawn graphics on the screen, which is the opposite direction of the physical rotation of the device.
For example, if the device is rotated 90 degrees counter-clockwise, to compensate rendering will be rotated by 90 degrees clockwise and thus the returned value here will be Surface.ROTATION_90.
getRotation was introduced from Android 2.2. Use getOrientation if your target are older devices.
Got answer form here:
how to detect orientation of android device?
If you just want to detect orientation of the device then you can use an OrientationEventListener.
here's the official docs:
http://developer.android.com/reference/android/view/OrientationEventListener.html#onOrientationChanged(int)
But before you use anything, check out this excellent blog post about orientation handling in android:
http://android-developers.blogspot.in/2010/09/one-screen-turn-deserves-another.html