Develop Reference

Rotating the preview with CameraX

I have a proprietary device with a rotated camera, that doesn't report its rotation to the OS.
We use CameraX with PreviewView, and I get the image 90 deg rotated, and mirrored.
Is there an efficient way to rotate the preview?
We tried:
Setting the setTargetOrientation of camera, of analyzer, of Preview. None works.
Rotating the PreviewView of course doesn't work, it just modifies the ViewPort because it won't rotate the surface.
ViewFinder library - outputs bitmaps but doesn't connect to Preview (or does anybody have a solution for that?). Without setSurfaceProvider, nothing will empty the frames pipeline, and it will display nothing.
Ugly "Solution":
We currently use bitmap (via Analyzer), then we display the bitmap. That's ugly, and CPU expensive.
Is Camera2 the only way to rotate previews nicely? Camera2 does bitmaps anyway, but it may do the YUV conversion on hardware.
NOTE
Solutions like here don't work. Setting target orientation of the Preview only modifies the scale to compensate for aspect ratio. It doesn't actually rotate. Also that doesn't work because it's obsolete and not valid anymore.

The Preview#setTargetRotation API should work: https://developer.android.com/reference/androidx/camera/core/Preview#setTargetRotation(int)
For this API to work, your PreviewView has to be in the COMPATIBLE mode:
PreviewView#setImplementationMode(ImplementationMode.COMPATIBLE);
You can find a complete sample on GitHub: https://github.com/androidx/androidx/blob/androidx-main/camera/integration-tests/viewtestapp/src/main/java/androidx/camera/integration/view/PreviewViewFragment.java
I uploaded a screen record of the preview rotating. Please take look and let me know if this is not your desired result: https://github.com/xizhang/public-files/blob/main/stackoverflow74798791/preview_rotation.mp4
You can also download the APK and test it yourself: https://github.com/xizhang/public-files/blob/main/stackoverflow74798791/camera-testapp-view-debug.apk

Simulating FOV changes in VR for Google-Cardboard with Unity

I am working on my first VR project, in which I am displaying satellite data inside of a sphere. The camera/observer is placed in the middle of the sphere and "looks up" at the satellite data, which is rendered in all directions. I am doing this under Unity 2021 using the latest Cardboard SDK and running it on a Pixel-3 on Android 12. After some tinkering, I managed to get the scene to render, but the observer is MUCH too close to the scene. I am aware that the FOV is fixed by the device, but is seems to me that I should be able to scale the scene to "zoom out". However, nothing I have tried works, including the following;
Simply changing the size of the sphere (which is just a single "flip-normalled" object)
Changing the camera parameters (Note: I now understand that these have zero effect in VR, as the device sets the FOV).
Placing the camera object, embedded in an XRRig prefab in my case, inside an arbitrary "GameObject" and re-scaling the object (as specified here)
As in 3, but placing every object inside the GameObject
None of these have any effect on the eventual scene as built on the device. I am at a loss. Surely what I am attempting is possible? I really just want a tiny observer, i.e. to make the "sky" seem much farther away. Any/all help appreciated.
Cheers.

Perhaps you should make the satellites, or the images, smaller when rendering them to the sphere. Just scaling the sphere itself will probably just make everything larger, or smaller, to match the size of the sphere.

Does mirroring the (front) camera affect MLKit with CameraX?

You might consider this question a sequel to this other one I've recently asked. I have a custom device with a front camera that is mirrored by default and I would like it to always behave like it was flipped horizontally. Note that the end goal of my app is to perform face, barcode and facemask detection (the last one with a custom .tflite model), and I'm trying to understand if it's possible with the current state of CameraX and the quirks of the device I'm using.
For the preview, I can force PreviewView to use a TextureView by calling PreviewView#implementationMode = PreviewView.ImplementationMode.COMPATIBLE so I can just call PreviewView#setScaleX(-1) and it gets displayed like I want.
For taking pictures, I haven't tried yet but I know I can pass setReversedHorizontal(true) to the ImageCapture.OutputFileOptions used in the ImageCapture.Builder, so the image should be saved in reverse.
For image analysis I really don't know. If the input image is taken directly from the preview stream, then it should still be in its default state because PreviewView#setScaleX(-1) only flips the View on screen, right? So in my Analyzer's analyze() I would need to convert the ImageProxy#image to bitmap, then flip it and pass the result to InputImage.fromBitmap().
Is this everything I need? Is there a better way to do this?

Image Stabilization Issue in LGV30

Video suddenly zooming into the frame when recording starts in preview mode is known behavior, but the issue I am facing currently is the extreme jump in preview when using LGV30.
In pixel 1 XL and pixel 2 XL the jump is negligible
In Samsung S8+ the jump is visible but can be considered for the final smoother videos
But in LGV30 the jump is terrible, the video moves to the extreme top right corner of the preview and the final output quality of the video is worse than VGA.
I know that Image Stabilization is dependent on the combination of Sensor Orientation, Gyroscope and Accelerometer and the jump in video is decided by the quality of these sensors, but is there any way I can control the Jump in LGV30 or is it the hardware that has flaw.
I am writing down my configuration code for reference
mPreviewBuilder.set(CaptureRequest.COLOR_CORRECTION_MODE, CaptureRequest.COLOR_CORRECTION_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.CONTROL_CAPTURE_INTENT, CaptureRequest.CONTROL_CAPTURE_INTENT_VIDEO_RECORD);
mPreviewBuilder.set(CaptureRequest.HOT_PIXEL_MODE, CaptureRequest.HOT_PIXEL_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.EDGE_MODE, CaptureRequest.EDGE_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.NOISE_REDUCTION_MODE, CaptureRequest.NOISE_REDUCTION_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.TONEMAP_MODE, CaptureRequest.TONEMAP_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.SHADING_MODE, CaptureRequest.SHADING_MODE_FAST);
mPreviewBuilder.set(CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE, CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE_ON);
I assume that this jump can not be controlled, but is there any way to reduce the jump?

The solution turned out to be the wrong camera.
Sounds trivial but in this code
CameraManager manager = (CameraManager) mContext.getSystemService(Context.CAMERA_SERVICE);
manager.getCameraIdList()
The manager returns all the accessible camera for a given device.
In LGV30 since there are 2 rear cameras, one regular and another wide angle, I was accessing the wide angle camera.
Wide angle camera was not able to handle the Image Stabilization and was causing this issue. So the solution was to pick the first camera from the list for given camera facing. This is still a workaround because I couldn't figure out the API to check whether given camera is regular or workaround.
Details for above question is in How to check whether given Camera is regular camera or wide angle camera?

Who rotates the frame from a camera when rendering in a SurfaceTexture?

UPDATE : question is about Camera2
I'm trying to figure out who applies the rotation transform when a camera preview is drawn on a SurfaceTexture.
When requesting the preview sizes from the camera you always get pairs where the width is larger than the height (because landscape is the usual orientation when taking a picture).
When using the device on portrait mode and setting the preview size (for ex. 1600 x1200) the frames from the camera are correctly rotated but I can't find out where it's done, is it something CameraDevice does automatically based on Surface type ? or is it the SurfaceTexture who rotates the image?

The answer depends a bit on which camera API you're using; since you mention CameraDevice, I assume you mean camera2.
In camera2, one of the goals was to simplify the rotation handling where possible; so if you draw preview into a SurfaceView, the camera service and the hardware compositor / GPU cooperate and handle all the rotation for you. You always request landscape resolutions from the sensor (which is aligned with the long edge of the sensor matching the long edge of the device), and you just need to make sure your SurfaceView's aspect ratio matches that of the requested resolution (so in portrait, you need 9:16 View to display 1080p correctly; in landscape you need a 16:9 View).
For TextureView, and completely custom GL drawing with SurfaceTexture, the API can't do it all for you. The SurfaceTexture's transform is set by the camera API to rotate the camera output to the device's native orientation (generally portrait for phones, landscape for some tablets), but it can't handle the additional rotation if your app's UI is not in the native orientation.
In that case, you'll need to get the transform matrix from the SurfaceTexure, apply the extra rotation to match your app UI, and set that as the TextureView matrix.

It's been about a year since I worked on Camera stuff, so I apologize for a lack of specifics. But I do remember that the key when displaying the Camera preview was calculating the correct transform matrix to apply to the SurfaceView (or TextureView) to apply any scaling or rotation that is needed. So regarding "who is doing the rotation" I guess you can say it is the view, as instructed by the Transform that you supply. So the matrix will be based on the relationship of the preview resolution you request (from the list of what's capable for your device) compared to the actual laid out dimensions of your view, plus the physical orientation of the device sensor as compared to the current orientation of the device screen.
In the old camera API, Camera.CameraInfo has an orientation field that supplies the physical orientation with which the sensor is mounted. It's typically 90 for a landscape mounted sensor on most phones, but you'll also find some 270s out there (effectively "upside down"). 0 and 180 may exist but I've never seen them.