Develop Reference

preview stretches in camera2 apis

Following are the screenshots when using texture view in camera2 apis.In full screen the preview stretches,but it works when using lower resolution(second image).
How to use this preview in full screen without stretching it.

Below answer assumes you are in portrait mode only.
Your question is
How to use the preview in full-screen without stretching it
Let's break it down to 2 things:
You want the preview to fill the screen
The preview cannot be distorted
First you need to know that this is logically impossible without crop, if your device's viewport has a different aspect ratio with any available resolution the camera provides.
So I would assume you accept cropping the preview.
Step 1: Get a list of available resolutions
StreamConfigurationMap map = mCameraCharacteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
if (map == null) {
throw new IllegalStateException("Failed to get configuration map: " + mCameraId);
}
Size[] sizes = map.getOutputSizes(SurfaceTexture.class);
Now you get a list of available resolutions (Sizes) of your device's camera.
Step 2: Find the best aspect ratio
The idea is to loop the sizes and see which one best fits. You probably need to write your own implementation of "best fits".
I am not going to provide any code here since what I have is quite different from your use case. But ideally, it should be something like this:
Size findBestSize (Size[] sizes) {
//Logic goes here
}
Step 3: Tell the Camera API that you want to use this size
//...
textureView.setBufferSize(bestSize.getWidth(), bestSize.getHeight());
Surface surface = textureView.getSurface();
try {
mPreviewRequestBuilder = mCamera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
mPreviewRequestBuilder.addTarget(surface);
mCamera.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
mSessionCallback, null);
} catch (final Exception e) {
//...
}
Step 4: Make your preview extends beyond your viewport
This is then nothing related to the Camera2 API. We "crop" the preview by letting the SurfaceView / TextureView extends beyond device's viewport.
First place your SurfaceView or TextureView in a RelativeLayout.
Use the below to extend it beyond the screen, after you get the aspect ratio from step 2.
Note that in this case you probably need to know this aspect ratio before you even start the camera.
//Suppose this value is obtained from Step 2.
//I simply test here by hardcoding a 3:4 aspect ratio, where my phone has a thinner aspect ratio.
float cameraAspectRatio = (float) 0.75;
//Preparation
DisplayMetrics metrics = new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(metrics);
int screenWidth = metrics.widthPixels;
int screenHeight = metrics.heightPixels;
int finalWidth = screenWidth;
int finalHeight = screenHeight;
int widthDifference = 0;
int heightDifference = 0;
float screenAspectRatio = (float) screenWidth / screenHeight;
//Determines whether we crop width or crop height
if (screenAspectRatio > cameraAspectRatio) { //Keep width crop height
finalHeight = (int) (screenWidth / cameraAspectRatio);
heightDifference = finalHeight - screenHeight;
} else { //Keep height crop width
finalWidth = (int) (screenHeight * cameraAspectRatio);
widthDifference = finalWidth - screenWidth;
}
//Apply the result to the Preview
RelativeLayout.LayoutParams lp = (RelativeLayout.LayoutParams) cameraView.getLayoutParams();
lp.width = finalWidth;
lp.height = finalHeight;
//Below 2 lines are to center the preview, since cropping default occurs at the right and bottom
lp.leftMargin = - (widthDifference / 2);
lp.topMargin = - (heightDifference / 2);
cameraView.setLayoutParams(lp);
If you don't care about the result of Step 2, you can actually ignore Step 1 to Step 3 and simply use a library out there, as long as you can configure its aspect ratio. (It looks like this one is the best, but I haven't tried yet)
I have tested using my forked library. Without modifying any code of my library, I managed to make the preview fullscreen just by using Step 4:
Before using Step 4:
After using Step 4:
And the preview just after taking a photo will not distort as well, because the preview is also extending beyond your screen.
But the output image will include area that you cannot see in the preview, which makes perfect sense.
The code of Step 1 to Step 3 are generally referenced from Google's CameraView.

That's a common problem on some devices. I've noticed it mostly on samsung. You may use a trick with setting transformation on your TextureView to make it centerCrop like ImageView behaviour

I also faced similar situation, but this one line solved my problem
view_finder.preferredImplementationMode = PreviewView.ImplementationMode.TEXTURE_VIEW
in your xml:
<androidx.camera.view.PreviewView
android:id="#+id/view_finder"
android:layout_width="match_parent"
android:layout_height="match_parent" />
For camera implementation using cameraX you can refer
https://github.com/android/camera-samples/tree/master/CameraXBasic

I figured out what was your poroblem. You were probably trying something like this:
textureView.setSurfaceTextureListener(new TextureView.SurfaceTextureListener() {
#Override
public void onSurfaceTextureAvailable(SurfaceTexture surfaceTexture, int i, int j) {
cam.startPreview(surfaceTexture, i, j);
cam.takePicture();
}
public void onSurfaceTextureSizeChanged(SurfaceTexture surfaceTexture, int i, int i1) { }
public boolean onSurfaceTextureDestroyed(SurfaceTexture surfaceTexture) { return false; }
public void onSurfaceTextureUpdated(SurfaceTexture surfaceTexture) { }
});

Android Camera.Parameters setPictureSize not working

I am trying to set the best possible output picture size in my camera object. So that, i can get a perfect downscaled sample image and display it.
During debugging i observed i am setting output picture size exactly the size of my screen dimensions. But when i DecodeBounds of the returned image by camera. I get some larger number!
Also i am not setting my display dimensions as expected output picture size. Code used to calculate and set the output picture size is given below.
I am using this code for devices having API level < 21, so using camera shouldn't be a problem.
I don't have any idea of why i am getting this behavior. Thanks in advance for help!
Defining Camera parameter
Camera.Parameters parameters = mCamera.getParameters();
setOutputPictureSize(parameters.getSupportedPictureSizes(), parameters); //update paramters in this function.
//set the modified parameters back to mCamera
mCamera.setParameters(parameters);
Optimal picture size calculation
private void setOutputPictureSize(List<Camera.Size> availablePicSize, Camera.Parameters parameters)
{
if (availablePicSize != null) {
int bestScore = (1<<30); //set an impossible value.
Camera.Size bestPictureSize = null;
for (Camera.Size pictureSize : availablePicSize) {
int curScore = calcOutputScore(pictureSize); //calculate sore of the current picture size
if (curScore < bestScore) { //update best picture size
bestScore = curScore;
bestPictureSize = pictureSize;
}
}
if (bestPictureSize != null) {
parameters.setPictureSize(bestPictureSize.width, bestPictureSize.height);
}
}
}
//calculates score of a target picture size compared to screen dimensions.
//scores are non-negative where 0 is the best score.
private int calcOutputScore(Camera.Size pictureSize)
{
Point displaySize = AppData.getDiaplaySize();
int score = (1<<30);//set an impossible value.
if (pictureSize.height < displaySize.x || pictureSize.width < displaySize.y) {
return score; //return the worst possible score.
}
for (int i = 1; ; ++i) {
if (displaySize.x * i > pictureSize.height || displaySize.y * i > pictureSize.width) {
break;
}
score = Math.min(score, Math.max(pictureSize.height-displaySize.x*i, pictureSize.width-displaySize.y*i));
}
return score;
}

Finally i resolved the issue after many attempts! Below are my findings:
Step 1. If we are already previewing, call mCamera.stopPreview()
Step 2. Set modified parameters by calling mCamera.setParameters(...)
Step 3. Start previewing again, call mCamera.startPreview()
If i call mCamera.setParameters without stopping preview (Assuming camera is previewing). Camera seems to ignore the updated parameters.
I came up with this solution after several trail and errors. Anyone know better way to update parameters during preview please share.

Low quality picture and what's the usage of overriding surfaceChanged when using Android Camera API

I have a problem for my application, specifically, when I want to take a picture (from android Camera API) and send it to my server, I got some strange pictures like following example:
which contain many noise points and actually the size/resolution are very small (176*144 pixels). And this is my original code for surfaceChanged:
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
// start preview with new settings
try {
Camera.Parameters parameters = mCamera.getParameters();
parameters.set("orientation", "portrait");
mCamera.setDisplayOrientation(90);
mCamera.setPreviewDisplay(mHolder);
mCamera.startPreview();
} catch (Exception e){
Log.e(LOG_TAG, "Error starting camera preview: " + e.getMessage());
}
mCamera.startPreview();
}
And I tried to ask some classmates for this issue, they don't know but one of them give the following code:
#Override
public void surfaceChanged(SurfaceHolder arg0, int arg1, int arg2, int arg3) {
try {
// camera.stopPreview();
// Camera.Parameters mParameters = camera.getParameters();
Camera.Parameters parameters = mCamera.getParameters();
Camera.Size optiSize = getBestPreviewSize(720, 720);
if (optiSize != null) {
parameters.setPreviewSize(optiSize.width, optiSize.height);
parameters.setPictureSize(optiSize.width, optiSize.height);
}
parameters.set("orientation", "portrait");
mCamera.setDisplayOrientation(90);
List<String> focusModes = parameters.getSupportedFocusModes();
if (focusModes.contains(Camera.Parameters.FOCUS_MODE_AUTO)) {
parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
}
mCamera.setParameters(parameters);
// mParameters.setFlashMode(Camera.Parameters.FLASH_MODE_TORCH);
// camera.setParameters(mParameters); // apply the changes
} catch (Exception e) {
// older phone - doesn't support these calls
}
mCamera.startPreview();
}
private Camera.Size getBestPreviewSize(int width, int height) {
List<Camera.Size> sizes = mCamera.getParameters()
.getSupportedPreviewSizes();
if (sizes == null)
return null;
Camera.Size optimalSize = null;
int tmpSize;
int minWidthDiff = 1000;
for (Camera.Size size : sizes) {
if (size.width > size.height)
tmpSize = size.height;
else
tmpSize = size.width;
if (Math.abs(tmpSize - width) < minWidthDiff) {
minWidthDiff = Math.abs(tmpSize - width);
optimalSize = size;
}
}
return optimalSize;
}
And this one works pretty well, it can store the original picture with the full resolution. Although I modify a little bit of my other codes to make the new codes compatible with my system, (add AutoFocus, for example). But I think the problem occurs because the method surfaceChanged because if I take image only with Autofocus, it still not work.
Therefore my question is: why this method will influence the quality of my image. I thought this method is only called when we "change" the surface, something like rotate the screen. But apparently it do something more than that?
Can anybody give me some help? Or some posts to explain this fact? Thank you very much in advance.

This method is called when the surface is changed, e.g. it is resized. It is common to adjust some of the camera's parameters to better fit the new configurations (such as setting an appropriate preview size).
What you are doing in the first code is you're getting the camera's Camera.Parameters, set a key-value pair (by the way, the documentation does not mention any parameter with the key "orientation" - it probably has no meaning). What you forgot to do is to apply the new parameters to the camera using Camera.setParameters(). Then again, without actually setting any valid parameters, this would have no effect.
This leads to the low-quality picture issue. You should use the Camera.Parameters object to set a desired preview and picture size. In the second block of code, this is done inside getBestPreviewSize(). A list of available preview sizes is got and the most fitting one is chosen based on the preview's size. It might be helpful to set a satisfactory picture size, too.
The thing is, different devices have different set of supported values for the parameters. If you want to provide a consistent functionality, you should check the supported values using Camera.Parameters.getSupported* methods and set them accordingly.

Camera2 with a SurfaceView

I'm trying to get the new Camera2 working with a simple SurfaceView and I'm having some problems with the live preview. On some devices the image is stretched out of proportions while looking fine on others.
I've setup a SurfaceView that I programatically adjust to fit the size of the preview stream size.
On Nexus 5 this looks fine but one Samsung devices its way off. Also the Samsung devices have a black border on the right part of the preview.
Is it really not possible to work with the SurfaceView or is this the time to switch to TextureView ?

Yes, it is certainly possible. Note that the SurfaceView and its associated Surface are two different things, and each can/must be assigned a size.
The Surface is the actual memory buffer which will hold the output of the camera, and thus setting its size dictates the size of the actual image you will get from each frame. For each format available from the camera, there is a small set of possible (exact) sizes you can make this buffer.
The SurfaceView is what does the displaying of this image when it is available, and can basically be any size in your layout. It will stretch its underlying associated image data to fit whatever its layout size is, but note this display size is different from the data's size- Android will resize the image data for display automatically. This is what is probably causing your stretching.
For example, you can make a SurfaceView-based autofit View similar to the camera2basic's AutoFitTextureView as follows (this is what I use):
import android.content.Context;
import android.util.AttributeSet;
import android.view.SurfaceView;
public class AutoFitSurfaceView extends SurfaceView {
private int mRatioWidth = 0;
private int mRatioHeight = 0;
public AutoFitSurfaceView(Context context) {
this(context, null);
}
public AutoFitSurfaceView(Context context, AttributeSet attrs) {
this(context, attrs, 0);
}
public AutoFitSurfaceView(Context context, AttributeSet attrs, int defStyle) {
super(context, attrs, defStyle);
}
/**
* Sets the aspect ratio for this view. The size of the view will be measured based on the ratio
* calculated from the parameters. Note that the actual sizes of parameters don't matter, that
* is, calling setAspectRatio(2, 3) and setAspectRatio(4, 6) make the same result.
*
* #param width Relative horizontal size
* #param height Relative vertical size
*/
public void setAspectRatio(int width, int height) {
if (width < 0 || height < 0) {
throw new IllegalArgumentException("Size cannot be negative.");
}
mRatioWidth = width;
mRatioHeight = height;
requestLayout();
}
#Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
super.onMeasure(widthMeasureSpec, heightMeasureSpec);
int width = MeasureSpec.getSize(widthMeasureSpec);
int height = MeasureSpec.getSize(heightMeasureSpec);
if (0 == mRatioWidth || 0 == mRatioHeight) {
setMeasuredDimension(width, height);
} else {
if (width < height * mRatioWidth / mRatioHeight) {
setMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
} else {
setMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
}
}
}
}

In the docs for createCaptureSession it says
For drawing to a SurfaceView: Once the SurfaceView's Surface is
created, set the size of the Surface with setFixedSize(int, int) to be
one of the sizes returned by getOutputSizes(SurfaceHolder.class)

I've never worked with Camera2 (but very interesting in the new technology) and all I can suggest you is to follow the samples. And if you check it carefully, the sample uses a Custom TextureView, you should probalby try to copy it to your project:
http://developer.android.com/samples/Camera2Basic/src/com.example.android.camera2basic/AutoFitTextureView.html
/**
* A {#link TextureView} that can be adjusted to a specified aspect ratio.
*/
public class AutoFitTextureView extends TextureView {
also considering that different between SurfaceView and TextureView (theoretically) is just that you can use as a normal view in layout and the other "punches a whole" through the view hierarchy, and that Camera2 is only available in API21+... there should be no harm in migrating to TextureView

ImageReader in Android needs too long time for one frame to be available

I am developing an Android App in which I'm using ImageReader to get image from a Surface. The surface's data is achieved from the VirtualDisplay when i record screen in Lollipop version. The problem is the image is available with very low rate (1 fps) (OnImageAvailableListener.onImageAvailable() function is invoked). When i tried to use MediaEncoder with this surface as an input surface the output video looks smooth under 30fps.
Is there any suggestion for me to read the image data of surface with high fps?
ImageReader imageReader = ImageReader.newInstance(width, height, PixelFormat.RGBA_8888, 2);
mImageReader.setOnImageAvailableListener(onImageListener, null);
mVirtualDisplay = mMediaProjection.createVirtualDisplay("VideoCap",
mDisplayWidth, mDisplayHeight, mScreenDensity,
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
imageReader.getSurface(), null /*Callbacks*/, null /*Handler*/);
//
//
OnImageAvailableListener onImageListener = new OnImageAvailableListener() {
#Override
public void onImageAvailable(ImageReader reader) {
// TODO Auto-generated method stub
if(reader != mImageReader)
return;
Image image = reader.acquireLatestImage();
if(image == null)
return;
// do some stuff
image.close();
}
};

FPS increased extremely when I switched to another format :
mImageReader = ImageReader.newInstance(mVideoSize.getWidth(), mVideoSize.getHeight(), ImageFormat.YUV_420_888, 2);
Hope this will help you.

I found that in addition to selecting the YUV format, I also had to select the smallest image size available for the device in order to get a significant speed increase.