I'm building an app for Android but The device do have a square screen. The screen is 320x320 and the camera app use the SurfaceView to show the preview as below :
mCameraView = (SurfaceView) findViewById(R.id.surface);
LayoutParams params = mCameraView.getLayoutParams();
int camera_dimension = getResources().getInteger(R.integer.camera_dimension);
params.height = camera_dimension; //320px
params.width = camera_dimension;
mCameraView.setLayoutParams(params);
mCameraViewHolder = mCameraView.getHolder();
mCameraViewHolder.addCallback(this);
mCameraViewHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
On Surface changed, I'm doing this and it works but the supporter preview is w480 x h320
public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
int mWidth = w;
int mHeight = h;
int mFormat = format;
try {
mCameraAccess.mCamera.stopPreview();
} catch (Exception e){
// ignore: tried to stop a non-existent preview
}
Camera.Parameters param = mCameraAccess.mCamera.getParameters();
List<Camera.Size> SupportedPreview = param.getSupportedPreviewSizes();
int ScreenSize = getResources().getInteger(R.integer.camera_dimension);
for (int i = 0; i < SupportedPreview.size(); i++) {
if (SupportedPreview.get(i).height == ScreenSize) {
param.setPreviewSize(SupportedPreview.get(i).width, SupportedPreview.get(i).height);
break;
}
}
mCameraAccess.mCamera.setParameters(param);
mCameraAccess.mCamera.startPreview();
}
How can make sure that my preview inside the viewholder is not compressed but a kind of centercrop. As the preview is a rectangle, I just need the square centered on the image. Usually I'm using scaleType but it's not supported in Surface view
Any idea ?
The solution which i have figured out is:
1)-Keep surface view full screen,so that is doesn't stretch.
2)-Put a view over surfaceview with full opacity.So that is looks like camera is already squared.
3)-After capturing image or video,you will have to crop them.
For video you have to use some video processing library like javacv.Using this library you can extract video frames,convert them to bitmap,crop bitmap in square and then re-encode into video.
To get accurate results you will need to play around with different techniques like zooming camera during capture etc. according to your needs.
Original Image:
Squared Image:
Related
Following are the screenshots when using texture view in camera2 apis.In full screen the preview stretches,but it works when using lower resolution(second image).
How to use this preview in full screen without stretching it.
Below answer assumes you are in portrait mode only.
Your question is
How to use the preview in full-screen without stretching it
Let's break it down to 2 things:
You want the preview to fill the screen
The preview cannot be distorted
First you need to know that this is logically impossible without crop, if your device's viewport has a different aspect ratio with any available resolution the camera provides.
So I would assume you accept cropping the preview.
Step 1: Get a list of available resolutions
StreamConfigurationMap map = mCameraCharacteristics.get(
CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
if (map == null) {
throw new IllegalStateException("Failed to get configuration map: " + mCameraId);
}
Size[] sizes = map.getOutputSizes(SurfaceTexture.class);
Now you get a list of available resolutions (Sizes) of your device's camera.
Step 2: Find the best aspect ratio
The idea is to loop the sizes and see which one best fits. You probably need to write your own implementation of "best fits".
I am not going to provide any code here since what I have is quite different from your use case. But ideally, it should be something like this:
Size findBestSize (Size[] sizes) {
//Logic goes here
}
Step 3: Tell the Camera API that you want to use this size
//...
textureView.setBufferSize(bestSize.getWidth(), bestSize.getHeight());
Surface surface = textureView.getSurface();
try {
mPreviewRequestBuilder = mCamera.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
mPreviewRequestBuilder.addTarget(surface);
mCamera.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
mSessionCallback, null);
} catch (final Exception e) {
//...
}
Step 4: Make your preview extends beyond your viewport
This is then nothing related to the Camera2 API. We "crop" the preview by letting the SurfaceView / TextureView extends beyond device's viewport.
First place your SurfaceView or TextureView in a RelativeLayout.
Use the below to extend it beyond the screen, after you get the aspect ratio from step 2.
Note that in this case you probably need to know this aspect ratio before you even start the camera.
//Suppose this value is obtained from Step 2.
//I simply test here by hardcoding a 3:4 aspect ratio, where my phone has a thinner aspect ratio.
float cameraAspectRatio = (float) 0.75;
//Preparation
DisplayMetrics metrics = new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(metrics);
int screenWidth = metrics.widthPixels;
int screenHeight = metrics.heightPixels;
int finalWidth = screenWidth;
int finalHeight = screenHeight;
int widthDifference = 0;
int heightDifference = 0;
float screenAspectRatio = (float) screenWidth / screenHeight;
//Determines whether we crop width or crop height
if (screenAspectRatio > cameraAspectRatio) { //Keep width crop height
finalHeight = (int) (screenWidth / cameraAspectRatio);
heightDifference = finalHeight - screenHeight;
} else { //Keep height crop width
finalWidth = (int) (screenHeight * cameraAspectRatio);
widthDifference = finalWidth - screenWidth;
}
//Apply the result to the Preview
RelativeLayout.LayoutParams lp = (RelativeLayout.LayoutParams) cameraView.getLayoutParams();
lp.width = finalWidth;
lp.height = finalHeight;
//Below 2 lines are to center the preview, since cropping default occurs at the right and bottom
lp.leftMargin = - (widthDifference / 2);
lp.topMargin = - (heightDifference / 2);
cameraView.setLayoutParams(lp);
If you don't care about the result of Step 2, you can actually ignore Step 1 to Step 3 and simply use a library out there, as long as you can configure its aspect ratio. (It looks like this one is the best, but I haven't tried yet)
I have tested using my forked library. Without modifying any code of my library, I managed to make the preview fullscreen just by using Step 4:
Before using Step 4:
After using Step 4:
And the preview just after taking a photo will not distort as well, because the preview is also extending beyond your screen.
But the output image will include area that you cannot see in the preview, which makes perfect sense.
The code of Step 1 to Step 3 are generally referenced from Google's CameraView.
That's a common problem on some devices. I've noticed it mostly on samsung. You may use a trick with setting transformation on your TextureView to make it centerCrop like ImageView behaviour
I also faced similar situation, but this one line solved my problem
view_finder.preferredImplementationMode = PreviewView.ImplementationMode.TEXTURE_VIEW
in your xml:
<androidx.camera.view.PreviewView
android:id="#+id/view_finder"
android:layout_width="match_parent"
android:layout_height="match_parent" />
For camera implementation using cameraX you can refer
https://github.com/android/camera-samples/tree/master/CameraXBasic
I figured out what was your poroblem. You were probably trying something like this:
textureView.setSurfaceTextureListener(new TextureView.SurfaceTextureListener() {
#Override
public void onSurfaceTextureAvailable(SurfaceTexture surfaceTexture, int i, int j) {
cam.startPreview(surfaceTexture, i, j);
cam.takePicture();
}
public void onSurfaceTextureSizeChanged(SurfaceTexture surfaceTexture, int i, int i1) { }
public boolean onSurfaceTextureDestroyed(SurfaceTexture surfaceTexture) { return false; }
public void onSurfaceTextureUpdated(SurfaceTexture surfaceTexture) { }
});
I have found out that when using a textureview instead of a surfaceview as a camera preview (both hooked up to the camera via a mediarecorder) then the preview is much more fuzzy.
What I mean by fuzzy is that in a texture view you can see the pixels, especially when zooming. That is not the case when using a surfaceview. Why is that the case?
UPD:
Sorry,but after I re-write my shit code, the key is the preview size too small that caused "fuzziness", so you should set a reasonable preview Size,not the reason strikeout below, but auto-focus is suggested ...
Size size = getBestSupportSize(parameters.getSupportedPreviewSizes(), width, height);
parameters.setPreviewSize(size.width, size.height);
As to the method getBestSupportSize(), how to get the bestSize for your project needs, in this case, it is as large as the screen width andhe ratio is 4/3 your's may be some other, I calculate the ration dividing width/height.
private Size getBestSupportSize(List<Size> sizes, int width, int height) {
Size bestsize = sizes.get(0);
int screenWidth = getResources().getDisplayMetrics().widthPixels;
int dt = Integer.MAX_VALUE;
for (int i = sizes.size() - 1; i >= 0; i--) {
Log.d(TAG, "-index : " + i);
Size s = sizes.get(i);
if (s.width * 3.0f / 4 == s.height) {
int newDT = Math.abs(screenWidth - s.width);
if (newDT < dt && screenWidth < s.width) {
dt = newDT;
bestsize = s;
}
}
}
return bestsize;//note that if no "4/3" size supported,default return size[0]
}
So this "fuzziness" was caused by a small previewSize calcualate a best size for the camera using this getSupportedPreviewSizes() method
And I will keep the autoFocus snippet below, strikeout though, FYR if is needed.
Well i got the solution for this "fuzzy" problem,and my case is just using TextureView andsurfaceTexture to take a pic instead of old surfaceView withsurfaceHolderway.
The key is set this mCamera.autofocus(), why the pic is"fuzzy" is bacause we lack of this autoFocus setting.
like below :
mCamera.setPreviewTexture(surface);
//enable autoFocus if moving
mCamera.setAutoFocusMoveCallback(new AutoFocusMoveCallback() {
#Override
public void onAutoFocusMoving(boolean start, Camera camera) {
if (start) { //true means you are moving the camera
mCamera.autoFocus(myAutoFocus);
}
}
});
mCamera.startPreview();
The autoFocusCallback like this:
AutoFocusCallback myAutoFocus = new AutoFocusCallback() {
#Override
public void onAutoFocus(boolean success, Camera camera) {
}
};
I have built a module similar to vine app video recording. But I am not able to make the video size to 480x480 px . Is there any way to do that. Thanks
The Android camera has a limited list of available sizes for the camera. So we need to select best camera size and select sub image(480x480) from the original camera image.
For example on my HTC one m8 I have this sizes for the camera:
1920x1088
1920x1080
1808x1080
....
720x480
640x360
640x480
576x432
480x320
384x288
352x288
320x240
240x160
176x144
You can retrieve list of available size by use getSupportedPreviewSizes() method.
public Camera mCamera;//Your camera instance
public List<Camera.Size> cameraSizes;
private final int CAMERA_IMAGE_WIDTH = 480;
private final int CAMERA_IMAGE_HEIGHT = 480;
...
cameraSizes = mCamera.getParameters().getSupportedPreviewSizes()
After that, you need to find most suitable camera size and set preview size for the camera.
Camera.Size findBestCameraSize(int width, int height){
Camera.Size bestSize = cameraSizes.get(0);
int minimalArea = bestSize.height * bestSize.width;
for(int i = 1;i < cameraSizes.size();i++){
Camera.Size size = cameraSizes.get(i);
int area = size.height * size.width;
if(size.width < width || size.height < height){
continue;
}
if(area < minimalArea){
bestSize = size;
minimalArea = area;
}
}
return bestSize;
}
...
SurfaceHolder.Callback surfaceCallback = new SurfaceHolder.Callback() {
public void surfaceCreated(SurfaceHolder holder) {
//Do something
}
public void surfaceChanged(SurfaceHolder holder,
int format, int width,
int height) {
Camera.Parameters params = mCamera.getParameters();
Camera.Size size = findBestCameraSize(CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT);
params.setPreviewSize(size.width, size.height);
camera.setParameters(params);
if(mCamera != null){
mCamera.startPreview();
}
}
public void surfaceDestroyed(SurfaceHolder holder) {
// Do something
}
};
After setup camera size, we need to get sub image from result bitmap, from the camera. You need put this code where you receive a bitmap picture(Usually I use an OpenCV library and matrixes for better performance).
Bitmap imageFromCamera = //here ve receive image from camera.
Camera.Size size = mCamera.getParameters().getPreviewSize();
int x = (size.width - CAMERA_IMAGE_WIDTH)/2;
int y = (size.height - CAMERA_IMAGE_HEIGHT)/2;
Bitmap resultBitmap = null;
if(x < 0 || y < 0){
resultBitmap = imageFromCamera;
}else{
resultBitmap = Bitmap.createBitmap(imageFromCamera, x, y, CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT);
}
The video capture resolution for android are limited to the native resolutions supported by the camera.
You can try using a 3rd party library for video post processing. So you can crop or re-scale the video captured by the camera.
I am using this one
android-gpuimage-videorecording
and it works quite well.
I am using the Camera on Android. The SurfaceHolder is used to show the preview screen. I need to have the exact, pixel by pixel picture seen on the SurfaceView. How do I do that ?
I tried getting the dimensions of the Surface view when
public void surfaceChanged(SurfaceHolder holder,
int format, int width,
int height) {}
Is called, then use those measurements to transform the taken picture from
public void onPictureTaken( byte[] data, Camera camera ) { }
This works, but only on some devices. Other devices distort the picture.
EASY:
Create a
byte[] buffer;
camera.addCallbackBuffer(buffer);
You also want to set the size of the buffer:
int height = camera.getParameters().getPreviewSize().height;
int width = camera.getParameters().getPreviewSize().width;
int bitsPerPixel = ImageFormat.getBitsPerPixel(camera.getParameters().getPreviewFormat());
buffer = new byte[width * height * bitsPerPixel];
I'm making a line follower for my robot on Android (to learn Java/Android programming), currently I'm facing the image processing problem: the camera preview returns an image format called YUV which I want to convert to a threshold in order to know where the line is, how would one do that?
As of now I've succeeded getting something, that is I definitely can read data from the camera preview and by some miracle even know if the light intensity is over or under a certain value at a certain area on the screen. My goal is to draw the robot's path on an overlay over the camera's preview, that too works to some extent, but the problem is the YUV management.
As you can see not only the dark area is drawn sideways, but it also repeats itself 4 times and the preview image is stretched, I cannot figure out how to fix these problems.
Here's the relevant part of code:
public void surfaceCreated(SurfaceHolder arg0) {
// TODO Auto-generated method stub
// camera setup
mCamera = Camera.open();
Camera.Parameters parameters = mCamera.getParameters();
List<Camera.Size> sizes = parameters.getSupportedPreviewSizes();
for(int i=0; i<sizes.size(); i++)
{
Log.i("CS", i+" - width: "+sizes.get(i).width+" height: "+sizes.get(i).height+" size: "+(sizes.get(i).width*sizes.get(i).height));
}
// change preview size
final Camera.Size cs = sizes.get(8);
parameters.setPreviewSize(cs.width, cs.height);
// initialize image data array
imgData = new int[cs.width*cs.height];
// make picture gray scale
parameters.setColorEffect(Camera.Parameters.EFFECT_MONO);
parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
mCamera.setParameters(parameters);
// change display size
LayoutParams params = (LayoutParams) mSurfaceView.getLayoutParams();
params.height = (int) (mSurfaceView.getWidth()*cs.height/cs.width);
mSurfaceView.setLayoutParams(params);
LayoutParams overlayParams = (LayoutParams) swOverlay.getLayoutParams();
overlayParams.width = mSurfaceView.getWidth();
overlayParams.height = mSurfaceView.getHeight();
swOverlay.setLayoutParams(overlayParams);
try
{
mCamera.setPreviewDisplay(mSurfaceHolder);
mCamera.setDisplayOrientation(90);
mCamera.startPreview();
}
catch (IOException e)
{
e.printStackTrace();
mCamera.stopPreview();
mCamera.release();
}
// callback every time a new frame is available
mCamera.setPreviewCallback(new PreviewCallback() {
public void onPreviewFrame(byte[] data, Camera camera)
{
// create bitmap from camera preview
int pixel, pixVal, frameSize = cs.width*cs.height;
for(int i=0; i<frameSize; i++)
{
pixel = (0xff & ((int) data[i])) - 16;
if(pixel < threshold)
{
pixVal = 0;
}
else
{
pixVal = 1;
}
imgData[i] = pixVal;
}
int cp = imgData[(int) (cs.width*(0.5+(cs.height/2)))];
//Log.i("CAMERA", "Center pixel RGB: "+cp);
debug.setText("Center pixel: "+cp);
// process preview image data
Paint paint = new Paint();
paint.setColor(Color.YELLOW);
int start, finish, last;
start = finish = last = -1;
float x_ratio = mSurfaceView.getWidth()/cs.width;
float y_ratio = mSurfaceView.getHeight()/cs.height;
// display calculated path on overlay using canvas
Canvas overlayCanvas = overlayHolder.lockCanvas();
overlayCanvas.drawColor(0, Mode.CLEAR);
// start by finding the tape from bottom of the screen
for(int y=cs.height; y>0; y--)
{
for(int x=0; x<cs.width; x++)
{
pixel = imgData[y*cs.height+x];
if(pixel == 1 && last == 0 && start == -1)
{
start = x;
}
else if(pixel == 0 && last == 1 && finish == -1)
{
finish = x;
break;
}
last = pixel;
}
//overlayCanvas.drawLine(start*x_ratio, y*y_ratio, finish*x_ratio, y*y_ratio, paint);
//start = finish = last = -1;
}
overlayHolder.unlockCanvasAndPost(overlayCanvas);
}
});
}
This code generates an error sometimes when quitting the application due to some method being called after release, which is the least of my problems.
UPDATE:
Now that the orientation problem is fixed (CCD sensor orientation) I'm still facing the repetition problem, this is probably related to my YUV data management...
Your surface and camera management looks correct, but I would doublecheck that camera actually accepted preview size settings ( some camera implementations reject some settings silently)
As you are working with portrait mode, you have to keep in mind that camera does not give a fart about prhone orientation - its coordinate origin isdetermined by CCD chip and is always to right corner and scan direction is from top to bottom and right to left - quite different from your overlay canvas. ( But if you are in landscape mode, everything is correct ;) ) - this is certaily source of odd drawing result
Your threshloding is bit naive and not very usefull in real life - I would suggest adaptive threshloding. In our javaocr project ( pure java, also has android demos ) we implemented efficient sauvola binarisation (see demos):
http://sourceforge.net/projects/javaocr/
Performance binarisation can be improved to work only on single image rows (patches welcome)
Issue with UV part of image is easy - default forman is NV21, luminance comes first
and this is just byte stream, and you do not need UV part of image at all (look into demos above)