Develop Reference

Image Stabilization Issue in LGV30

Video suddenly zooming into the frame when recording starts in preview mode is known behavior, but the issue I am facing currently is the extreme jump in preview when using LGV30.
In pixel 1 XL and pixel 2 XL the jump is negligible
In Samsung S8+ the jump is visible but can be considered for the final smoother videos
But in LGV30 the jump is terrible, the video moves to the extreme top right corner of the preview and the final output quality of the video is worse than VGA.
I know that Image Stabilization is dependent on the combination of Sensor Orientation, Gyroscope and Accelerometer and the jump in video is decided by the quality of these sensors, but is there any way I can control the Jump in LGV30 or is it the hardware that has flaw.
I am writing down my configuration code for reference
mPreviewBuilder.set(CaptureRequest.COLOR_CORRECTION_MODE, CaptureRequest.COLOR_CORRECTION_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.CONTROL_CAPTURE_INTENT, CaptureRequest.CONTROL_CAPTURE_INTENT_VIDEO_RECORD);
mPreviewBuilder.set(CaptureRequest.HOT_PIXEL_MODE, CaptureRequest.HOT_PIXEL_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.EDGE_MODE, CaptureRequest.EDGE_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.NOISE_REDUCTION_MODE, CaptureRequest.NOISE_REDUCTION_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.TONEMAP_MODE, CaptureRequest.TONEMAP_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.SHADING_MODE, CaptureRequest.SHADING_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE, CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE_ON);
I assume that this jump can not be controlled, but is there any way to reduce the jump?

The solution turned out to be the wrong camera.
Sounds trivial but in this code
CameraManager manager = (CameraManager) mContext.getSystemService(Context.CAMERA_SERVICE);
manager.getCameraIdList()
The manager returns all the accessible camera for a given device.
In LGV30 since there are 2 rear cameras, one regular and another wide angle, I was accessing the wide angle camera.
Wide angle camera was not able to handle the Image Stabilization and was causing this issue. So the solution was to pick the first camera from the list for given camera facing. This is still a workaround because I couldn't figure out the API to check whether given camera is regular or workaround.
Details for above question is in How to check whether given Camera is regular camera or wide angle camera?

Who rotates the frame from a camera when rendering in a SurfaceTexture?

UPDATE : question is about Camera2
I'm trying to figure out who applies the rotation transform when a camera preview is drawn on a SurfaceTexture.
When requesting the preview sizes from the camera you always get pairs where the width is larger than the height (because landscape is the usual orientation when taking a picture).
When using the device on portrait mode and setting the preview size (for ex. 1600 x1200) the frames from the camera are correctly rotated but I can't find out where it's done, is it something CameraDevice does automatically based on Surface type ? or is it the SurfaceTexture who rotates the image?

The answer depends a bit on which camera API you're using; since you mention CameraDevice, I assume you mean camera2.
In camera2, one of the goals was to simplify the rotation handling where possible; so if you draw preview into a SurfaceView, the camera service and the hardware compositor / GPU cooperate and handle all the rotation for you. You always request landscape resolutions from the sensor (which is aligned with the long edge of the sensor matching the long edge of the device), and you just need to make sure your SurfaceView's aspect ratio matches that of the requested resolution (so in portrait, you need 9:16 View to display 1080p correctly; in landscape you need a 16:9 View).
For TextureView, and completely custom GL drawing with SurfaceTexture, the API can't do it all for you. The SurfaceTexture's transform is set by the camera API to rotate the camera output to the device's native orientation (generally portrait for phones, landscape for some tablets), but it can't handle the additional rotation if your app's UI is not in the native orientation.
In that case, you'll need to get the transform matrix from the SurfaceTexure, apply the extra rotation to match your app UI, and set that as the TextureView matrix.

It's been about a year since I worked on Camera stuff, so I apologize for a lack of specifics. But I do remember that the key when displaying the Camera preview was calculating the correct transform matrix to apply to the SurfaceView (or TextureView) to apply any scaling or rotation that is needed. So regarding "who is doing the rotation" I guess you can say it is the view, as instructed by the Transform that you supply. So the matrix will be based on the relationship of the preview resolution you request (from the list of what's capable for your device) compared to the actual laid out dimensions of your view, plus the physical orientation of the device sensor as compared to the current orientation of the device screen.
In the old camera API, Camera.CameraInfo has an orientation field that supplies the physical orientation with which the sensor is mounted. It's typically 90 for a landscape mounted sensor on most phones, but you'll also find some 270s out there (effectively "upside down"). 0 and 180 may exist but I've never seen them.

Change video orientation by 90 degrees anticlockwise after recording is done?

I am recording a video using front camera in a particular orientation and then i am uploading it to server , to do some open cv analysis to it but the video server receiving is in landscape mode (horizontally align) so it can not do it's analysis properly .
What i want is that before uploading the video i some how can change it's orientation by by 90 degrees anticlockwise so that analysis can be done properly ?

Solved it by myself , answering so that if someone find himself in same position hoping it will help
Main point was to understand the difference between
android.hardware.Camera camera.setDisplayOrientation(degrees1); and MediaRecorder mediaRecorder.setOrientationHint(degrees2);
former set the way your camera preview will look when you are recording and latter one will set the way your recorded video will be playback when you will play it after.
So if you want your user to record in one orientation(for eg landscape) and then show it to him in portrait way (and can not do so just by setting android:orientation , happens sometimes in pretty specific cases)then this will be helpful .

Android video recorded using mediaRecorder and surface view doesn't play in Landscape View although recorded in Portrait View

I am working on an app where I have to record a video which is supposed to be in Portrait View specifically and play it in Portrait View as well. I'm using mediaRecorder to record the video.
Have used this site for reference: https://examples.javacodegeeks.com/android/core/android-video-capture-example/
and made slight changes to suit my requirements:
Here is the code:
For the camera I use these settings:
mCamera = Camera.open(findFrontFacingCamera());
mCamera.setDisplayOrientation(90);
Camera.Parameters params= mCamera.getParameters();
params.set("rotation", 90);
params.set("orientation", "portrait");
mCamera.setParameters(params);
mPreview.refreshCamera(mCamera);
The problem I'm facing right now is that video gets recorded (what seems to be) in Portrait View but when I play it using any player for eg. MX Player I get a video that is recorded/played in Landscape View.
Here are the screens for reference:
The video played:
I have used the setDisplayOrientation(90) parameter but its not working.
How do I achieve that I'm trying to achieve?

Unlike iOS camera, we cannot truly choose portrait camera frame orientation on Android.
Camera.setDisplayOrientation() only refers to the orientation of live video preview from camera; the actual frame data is never rotated on Android. You can see it if you use an onPreviewFrame() callback. Camera.setRotation() only effects the onPictureTaken() callback, and on majority of devices only sets an EXIF rotation flag.
Setting "rotation" and "orientation" explicitly by name is dangerous: some devices may recognize these keys, other will silently ignore them, while yet other ROMs may throw a RuntimeException when the Camera.setParameters() params with unexpected key or value is received. But at any rate, I have never seen an Android device that changes the recorded video frames when any of these flags.
You have the MediaRecorder.setOrientationHint() API, but it will not rotate the frames of recorded video, only put a flag in the video header.
So, unfortunately you must rotate the video on the receiving side. This depends on the way you render your video, and yes, you can check the orientation hint if you want.

How to get right camera preview independent from the screen orientation?

Why in the portrait screen orientation the camera surface is rotated by -90 degrees "90 degrees anticlockwise". I want the camera preview to be normal regardless of the screen orientation. How can I achieve that.

Did you tried this:
mCam.setDisplayOrientation(90); use it inside surfaceChanged(...)
setDisplayOrientation
Set the clockwise rotation of preview display in degrees. This affects the preview frames and the picture displayed after snapshot. This method is useful for portrait mode applications. Note that preview display of front-facing cameras is flipped horizontally before the rotation, that is, the image is reflected along the central vertical axis of the camera sensor. So the users can see themselves as looking into a mirror.
This does not affect the order of byte array passed in onPreviewFrame(byte[], Camera), JPEG pictures, or recorded videos. This method is not allowed to be called during preview.