So I have an issue with a device (Alcatel 5033D 1) that's always giving me a resolution of 480x640 for CameraX analysis. This is the last version of the code in which I init it:
private void initCameraAnalysis(Size resolution) {
Log.d(TAG, "initCameraAnalysis: ");
cameraAnalysis = new ImageAnalysis.Builder()
.setTargetResolution(resolution)
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.setImageQueueDepth(0)
.build();
cameraAnalysis.setAnalyzer(
cameraExecutor,
new QualityAnalyzer()
);
}
I just added this part to see if it would help, initially these calls weren't there and they actually don't help with this issue:
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.setImageQueueDepth(0)
The resolution is passed when calling that method as follows, and so I'm requesting 720x1280 that's what I need (using portrait orientation):
initCameraAnalysis(new Size(720, 1280));
Now, this is a low end device and always fallback to 480x640 but I have another app made by a colleague that still uses Camera2 and that one can make 720x1280 with this same device without issue, so I know for sure the device is actually capable of doing 720x1280.
Moreover, I have added an ImageCapture use case and that one can do 720x1280 without issue too, but I need it to do 720x1280 during the analysis.
So I wonder if there'd be any way to force 720x1280 in the analysis use case despite the stubbornness of CameraX not taking it, even if it means using some Camera2 extension or whatever.
Otherwise is likely I'd have to rewrite the whole app using Camera2 instead, which seems like going backwards in time...
Thanks a lot in advance!
I don't know If you have seen it but just in case you did not I would like to point what stated here:
Express the resolution Size in the coordinate frame after rotating the supported sizes by the target rotation. For example, a device with portrait natural orientation in natural target rotation requesting a portrait image may specify 480x640, and the same device, rotated 90 degrees and targeting landscape orientation may specify 640x480.
Maybe also give it a shot with:
val imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(1280, 720))
.build()
Related
So I have an application that uses CameraX ImageCapture use case to take a selfie picture than then is passed to an AI algorithm to do some stuff with it.
Now, I have an user with a Samsung Galaxy S21 who when taking pictures in one specific place with some specific light conditions is producing an image that doesn't work as expected with the AI algorithm. I have examined these images myself and noticed that the problem seems to be that ImageCapture applies strong noise reduction, so strong that even for the human eye it looks wrong, like if it was a painting instead of a photograph.
I sent a modified version of such app to this user for trying which captures the image from the Analysis use case instead and the produced image does not have that problem, so it seems whatever it is, it's some post-processing done by the ImageCapture use case that's not done in the Analysis use case.
Now, I don't seem to find a way to tweak this post-processing in CameraX, in fact I haven't even found how to do it with Camera2 neither. At first I thought it may be HDR, and I found there are some extensions to enable HDR, Night Mode and such in CameraX, but all these are disabled by default according to the documentation, and as far as you use the DEFAULT_FRONT_CAMERA none should be applied, and that's what I'm using.
CameraSelector.DEFAULT_FRONT_CAMERA
In any case it's clear that some heavy post-processing is being done to these images in the ImageCapture use case, so I'm wondering how could I disable these.
BTW, I tried initialising the ImageCapture use case with CAPTURE_MODE_MINIMIZE_LATENCY in the hope that such flag would reduce the post-processing and hopefully remove noise reduction, but that didn't work.
imageCapture = new ImageCapture.Builder()
.setTargetResolution(resolution)
.setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
.build();
Any ideas on how to go beyond this to get that noise reduction filter disabled?
Thanks,
Fran
I found a way using Camera2Interop.Extender:
private void initImageCapture(Size resolution) {
Log.d(TAG, "initCameraCapture: ");
ImageCapture.Builder imageCaptureBuilder = new ImageCapture.Builder();
imageCaptureBuilder
.setTargetResolution(resolution)
.setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY);
Camera2Interop.Extender extender = new Camera2Interop.Extender(imageCaptureBuilder);
extender.setCaptureRequestOption(CaptureRequest.NOISE_REDUCTION_MODE, CaptureRequest.NOISE_REDUCTION_MODE_OFF);
imageCapture = imageCaptureBuilder.build();
}
I'm trying to use the maximum resolution when taking a picture using the new camera2 API. First of all, I try to get the available resolutions using the following code:
Charac:
CameraCharacteristics characteristics = mManager.getCameraCharacteristics(mCameraId);
StreamConfigurationMap streamMap =
characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
if (streamMap == null) {
throw new RuntimeException("StreamConfigurationMap is null. Should not happen.");
}
streamMap.getOutputSizes(ImageFormat.*); // I replaced the star with all possible values
The issue is this returns the following list of resolutions : [1280x960, 1088x1088, 640x480, 352x288, 176x144, 3264x1836, 3264x2448, 1280x720, 960x720, 720x480, 320x240, 1920x1080, 2400x1080, 1440x1080, 2576x1932, 3264x1468] (biggest is 3264x2448)
However when using the stock camera app (Samsung A70) the picture resolution is surprisingly bigger, ie: 4032x3024.
During investigation I tried to download many camera apps from the store to see what is the max resolution in there case, and all of them returned the same resolution as mine, ie: 3264x2448.
My question is how can the Stock Camera App get a bigger resolution than the rest of us?
Did you check the output of StreamConfigurationMap#getHighResolutionOutputSizes() as well?
Some devices list their highest resolutions there, because they can't maintain >= 20-fps output rate guaranteed by the BURST_CAPTURE capability, which is required for all sizes on the normal size list if you want a FULL-level device. As long as at least 8 MP is listed in normal sizes, the device is compliant with the BURST_CAPTURE rules and higher resolutions can be listed under getHighResolutionOutputSizes.
I want to capture photos with fixed resolution (i.e. 1200x1600) using CameraX library.
So according to documentation I can set target resolution for ImageCaptureConfig:
val imageCaptureConfig = ImageCaptureConfig.Builder()
.setLensFacing(CameraX.LensFacing.BACK)
.setCaptureMode(ImageCapture.CaptureMode.MAX_QUALITY)
.setTargetResolution(Size(1200, 1600))
.setTargetAspectRatio(Rational(3,4))
.build()
Documentation describes setTargetResolution method as follows:
Sets the intended output target resolution.
The target resolution attempts to establish a minimum bound for the
image resolution. The actual image resolution will be the closest
available resolution in size that is not smaller than the target
resolution, as determined by the Camera implementation. However, if no
resolution exists that is equal to or larger than the target
resolution, the nearest available resolution smaller than the target
resolution will be chosen.
Correct me if I am wrong, but if device is able to take photos larger than 1200x1600 (e.g. 3024x4032), the output photo will be at least 1200x1600.
Unfortunately in many devices (e.g. Huawei P20 Pro, Samsung A5) the output photo is significantly smaller than 1200x1600, e.g. 480x640. Notice that these devices are able to take really large photos.
Is my ImageCaptureConfig badly configured or this is a bug?
The android documentation say this:
You cannot set both target aspect ratio and target resolution on the same use case. Doing so will throw an IllegalArgumentException when building the config object.
So decide what is mor important for you and choose only one.
It could be a bug that have been recently fixed at AOSP.
refer https://android.googlesource.com/platform/frameworks/support/+/5c1aed8c4c502a74eb4ee6d30fe2089f4afcaf11
When you use .setTargetResolution(Size), the resolution Size should be expressed in the coordinate frame after rotating the supported sizes by the target rotation.
For example, a device with portrait natural orientation in natural target rotation requesting a portrait image may specify 480x640, and the same device, rotated 90 degrees and targeting landscape orientation may specify 640x480.
So this in Portait:
val imageCaptureConfig = ImageCaptureConfig.Builder()
.setTargetResolution(Size(1200, 1600))
Became that in Landscape:
val imageCaptureConfig = ImageCaptureConfig.Builder()
.setTargetResolution(Size(1600, 1200))
Be careful of the orientation of your device:
fun getTargetResolution(): Size {
return when (resources.configuration.orientation) {
Configuration.ORIENTATION_PORTRAIT -> Size(1200, 1600)
Configuration.ORIENTATION_LANDSCAPE -> Size(1600, 1200)
else -> Size(1600, 1200)
}
}
fun setupCamera() {
...
val resolution = getTargetResolution()
val imageCaptureConfig = ImageCaptureConfig.Builder()
.setTargetResolution(resolution)
...
}
It is the same thing with ImageAnalysis and ImagePreview
If you want some documentation, go here
In my Camera2 API project for Android, I want to set a region for my Exposure Calculation. Unfortunately it doesn't work. On the other side the Focus region works without any problems.
Device: Samsung S7 / Nexus 5
1.) Initial values for CONTROL_AF_MODE & CONTROL_AE_MODE
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_AUTO);
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON);
2.) Create the MeteringRectangle List
meteringFocusRectangleList = new MeteringRectangle[]{new MeteringRectangle(0,0,500,500,1000)};
3.) Check if it is supported by the device and set the CONTROL_AE_REGIONS (same for CONTROL_AF_REGIONS)
if (camera2SupportHandler.cameraCharacteristics.get(CameraCharacteristics.CONTROL_MAX_REGIONS_AE) > 0) {
camera2SupportHandler.mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_REGIONS, meteringFocusRectangleList);
}
4.) Tell the camera to start Exposure control
camera2SupportHandler.mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER, CameraMetadata.CONTROL_AE_PRECAPTURE_TRIGGER_START);
The CONTROL_AE_STATE is always in CONTROL_AE_STATE_SEARCHING, but doesn't use the configured regions...
After long testing & development I've found an answer.
The coordinate system - Camera 1 API VS Camera 2 API
RED = CAM1; GREEN = CAM2; As shown in the image below, the blue rect are the coordinates for a possible focus/exposure area for the Cam1. By using the Cam2 API, there must be firstly queried the max of the height and the width. Please find more info here.
Initial values for CONTROL_AF_MODE & CONTROL_AE_MODE: See in the question above.
Set the CONTROL_AE_REGIONS: See in the question above.
Set the CONTROL_AE_PRECAPTURE_TRIGGER.
// This is how to tell the camera to start AE control
CaptureRequest captureRequest = camera2SupportHandler.mPreviewRequestBuilder.build();
camera2SupportHandler.mCaptureSession.setRepeatingRequest(captureRequest, captureCallbackListener, camera2SupportHandler.mBackgroundHandler);
camera2SupportHandler.mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER, CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER_START);
camera2SupportHandler.mCaptureSession.capture(captureRequest, captureCallbackListener, camera2SupportHandler.mBackgroundHandler);
The ''captureCallbackListener'' gives feedback of the AE control (of course also for AF control)
So this configuration works for the most Android phones. Unfortunately it doesn't work for the Samsung S6/7. For this reason I've tested their Camera SDK, which can be found here.
After deep investigations I've found the config field ''SCaptureRequest.METERING_MODE''. By setting this to the value of ''SCaptureRequest.METERING_MODE_MANUAL'', the AE area works also the Samsung phones.
I'll add an example to github asap.
Recently I had the same problem and finally found a solution that helped me.
All I needed to do was to step 1 pixel from the edges of the active sensor rectangle. In your example instead of this rectangle:
meteringRectangleList = new MeteringRectangle[]{new MeteringRectangle(0,0,500,500,1000)};
I would use this:
meteringRectangleList = new MeteringRectangle[]{new MeteringRectangle(1,1,500,500,1000)};
and it started working as magic on both Samsung and Nexus 5!
(note that you should also step 1 pixel from right/bottom edges if you use maximum values there)
It seems that many vendors have poorly implemented this part of documentation
If the metering region is outside the used android.scaler.cropRegion returned in capture result metadata, the camera device will ignore the sections outside the crop region and output only the intersection rectangle as the metering region in the result metadata. If the region is entirely outside the crop region, it will be ignored and not reported in the result metadata.
After implementing the camera2 API for the inApp camera I noticed that on Samsung devices the images appear blurry. After searching about that I found the Sasmung Camera SDK (http://developer.samsung.com/galaxy#camera). So after implementing the SDK on Samsung Galaxy S7 the images are fine now, but on Galaxy S6 they are still blurry. Someone experienced those kind of issues with Samsung devices?
EDIT:
To complement #rcsumners comment. I am setting autofocus by using
mPreviewBuilder.set(SCaptureRequest.CONTROL_AF_TRIGGER, SCaptureRequest.CONTROL_AF_TRIGGER_START);
mSCameraSession.capture(mPreviewBuilder.build(), new SCameraCaptureSession.CaptureCallback() {
#Override
public void onCaptureCompleted(SCameraCaptureSession session, SCaptureRequest request, STotalCaptureResult result) {
isAFTriggered = true;
}
}, mBackgroundHandler);
It is a long exposure image where the use has to take an image of a static non moving object. For this I am using the CONTROL_AF_MODE_MACRO
mCaptureBuilder.set(SCaptureRequest.CONTROL_AF_MODE, SCaptureRequest.CONTROL_AF_MODE_MACRO);
and also I am enabling auto flash if it is available
requestBuilder.set(SCaptureRequest.CONTROL_AE_MODE,
SCaptureRequest.CONTROL_AE_MODE_ON_AUTO_FLASH);
I am not really an expert in this API, I mostly followed the SDK example app.
There could be a number of issues causing this problem. One prominent one is the dimensions of your output image
I ran Camera2 API and the preview is clear, but the output was quite blurry
val characteristics: CameraCharacteristics? = cameraManager.getCameraCharacteristics(cameraId)
val size = characteristics?.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)?.getOutputSizes(ImageFormat.JPEG) // The issue
val width = imageDimension.width
val height = imageDimension.height
if (size != null) {
width = size[0].width; height = size[0].height
}
val imageReader = ImageReader.newInstance(width, height, ImageFormat.JPEG, 5)
The code below was returning a dimension about 245*144 which was way to small to be sent to the image reader. Some how the output was stretching the image making it end up been blurry. Therefore I removed this line below.
val size = characteristics?.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)?.getOutputSizes(ImageFormat.JPEG) // this was returning a small
Setting the width and height manually resolved the issue.
You're setting the AF trigger for one frame, but then are you waiting for AF to complete? For AF_MODE_MACRO (are you verifying the device lists support for this AF mode?) you need to wait for AF_STATE_FOCUSED_LOCKED before the image is guaranteed to be stable and sharp. (You may also receive NOT_FOCUSED_LOCKED if the AF algorithm can't reach sharp focus, which could be because the object is just too close for the lens, or the scene is too confusing)
On most modern devices, it's recommended to use CONTINUOUS_PICTURE and not worry about AF triggering unless you really want to lock focus for some time period. In that mode, the device will continuously try to focus to the best of its ability. I'm not sure all that many devices support MACRO, to begin with.