The goal is to crop a preview on the surface for frames that come from Camera2 api capture session, but not to crop the video that will be created itself.
For example, I have a streaming resolution of 1920x1080 (16:9), and the screen size (just for instance) 2000x3000 (2:3 -> 6:9 or 16:24, lets pick the second variant for example), so I'd like to have my video in the original streaming resolution - 1920x1080, but my preview to fill all the available space without View resizing - so it should be 5333x3000 (just bumping the size up to fill the rectangle area using the same aspect ratio as the streaming resolution's one), and then to "cut" the "frame data that corresponds to this surface's resolution (which is, I suppose, 5333x2000)" to 2000x3000 (just remove (5333 - 2000) / 2 from both top and bottom).
Is it possible?
P.S.: the bad thing is that google sample for camera2 api resizes the view itself, and these "blank areas" are undesired for me. I haven't found anything that even closely matches my problem.
P.S.S.: AFAIU this SO solution crops the frame that comes from the camera itself, but I need my video to be in the original resolution.
If you're using a TextureView, you can probably adjust its transform matrix to scale up the preview (and cut off the edges in the process). Read the existing matrix, fix up the aspect ratio and scale it up, and then save the new matrix.
But please note that saving a different field of view than what you're showing to the user is probably going to get you negative reactions - please use the preview to frame what they want to record, and if what you're saving has extra stuff, the recorded video won't match expectations. Of course, maybe this isn't a concern for your use case.
Related
I want to display a preview in fullscreen using camera2 api. The camera size I selected is 4160x3120 (16:9) and my screen is 1080x1920 (9:16). Therefore if I want the preview to be right scaled I have to crop the camera output :
Rect zoomCrop = new Rect(0, 0, 1755, 3120);
captureRequestBuilder.set(CaptureRequest.SCALER_CROP_REGION, zoomCrop);
The cropped preview should be of size 1755x3120 which has a ratio of 9:16 as well. Still the preview is streched.
Question :
Am I using SCALER_CROP_REGION correctly ? If not, how am I supposed to use it ?
Thanks in advance
I've been struggling with the same sort of problem, as I want to implement a camera preview stream into a square TextureView.
Working with the SCALER_CROP_REGION seemed like the obvious place to resolve issues with incompatible aspect ratios, but I don't think that operation will ever actually change the aspect ratio of the result, hence it doesn't prevent stretching/warping behaviour. (the docs talk about this)
I've made progress by implementing a similar transform to the one in the Camera2Basic example project - see the function configureTransform in Camera2BasicFragment).
Using this sort of view-level scaling matrix, you can manage the scaling/cropping of the camera preview without altering the size of your view.
It sounds like you are getting a landscape image and displaying it in a portrait view - this means that you will have 'pillar box' lines above and below the image if you want to avoid distortion.
If you did want to fill the screen, and not have distortion or the pillar box lines then you would have to accept that part of your image will not be visible, as its outside the shape of the viewing area.
SCALER_CROP_REGION is tricky to understand without diagrams - I found the best explanation to be in the Andoir source documentation here: https://source.android.com/devices/camera/camera3_crop_reprocess
One diagram reproduced from that helps as an example here - the reason for highlight this is to show how the co-ordinates are being used. If you look at your crop region it will be biased towards the top left as you start at (0,0), which I am not sure is what you actually want.
I'm in the process of attempting to use the camera and some motion tracking AS3 classes to detect movement in front of a ViewSonic Smart Display, for the sake of a demo. I've gotten the app and detection to function on other Android devices, but the 'Smart Display' is presenting me with some odd issues.
Taking a long shot that someone might've encountered this, but this is the very simple camera set up code I reduced the issue down to:
var camera = Camera.getCamera();
camera.setMode(stage.stageWidth, stage.stageHeight, 30, true);
var video:Video = new Video(stage.stageWidth, stage.stageHeight);
video.attachCamera(camera);
My problem lies at the point of "video.attachCamera"
For some reason, this device takes this function as "Display the video in a tiny window in the upper right hand corner" and ignores all other code, dominating the screen with blank black, and a tiny (maybe 40x20px square) of video stream.
Image of it occuring...
Any help is much appreciated, thanks
The problem might be the values that you are passing to the camera with the setMode() method. You are trying to set the camera to capture at the width/height of the stage.
The camera likely does not have such a capture resolution, and as the documentation for setMode() states, it will try to find something that is close to what you have specified:
Sets the camera capture mode to the native mode that best meets the specified requirements. If the camera does not have a native mode that matches all the parameters you pass, the runtime selects a capture mode that most closely synthesizes the requested mode. This manipulation may involve cropping the image and dropping frames.
Now, it is granted that you would expect Flash to have picked a resolution that is bigger than what is shown in your screenshot. But given the myriad of camera devices/drivers, it's possible this is not working too well in your case.
You might start off by experimenting w/more typical resolutions to capture the video: 480x320, 640x480, 800x600, or at the most 1024x768. Most applications on the web probably use the first or second capture resolutions.
So change:
camera.setMode(stage.stageWidth, stage.stageHeight, 30, true);
To:
camera.setMode(640, 480, 30, true);
Note you can display the video in any size you want, but the capture resolutions you can use depend on your camera hardware/drivers/OS/etc. Typical resolutions have a 4:3 aspect ratio and are relatively small (not the full dimensions of the screen/stage). The capture resolution you use affects the quality of video and the amount of network bandwidth you need to stream the video. Generally (for streaming), you don't want to use a big capture resolution, but maybe it's not so important in your motion capture use case.
I am attempting to allow users to record video that is a different size than the actual on-screen preview that they can see while recording. This seems to be possible from this documentation concerning the getSupportedVideoSizes function which states:
If the returned list is not null, the returned list will contain at
least one Size and one of the sizes in the returned list must be
passed to MediaRecorder.setVideoSize() for camcorder application if
camera is used as the video source. In this case, the size of the
preview can be different from the resolution of the recorded video
during video recording.
This suggests that some phones will return null from this fn (in my experience the Galaxy SIII does) but for those who do not, it is possible to provide a preview with a different resolution than the actual video. Is this understanding correct? Do some phones allow the behavior and others not?
Attempting a Solution:
In the official description of the setPreviewDisplay function, which is used in the lengthy process of setting up for video recording, it is mentioned that:
If this method is called with null surface or not called at all, media
recorder will not change the preview surface of the camera.
This seems to be what I want, but unfortunately if I do this, the whole video recording process is completely messed up. I am assuming that this function can not be passed null or not called at all in the process of recording video. Perhaps in other contexts this is okay. Unfortunately though, this does not seem to help me.
My only next steps are to look into TextureViews and to use a preview Texture as opposed to a typical SurfaceView implementation in order to use openGL to stretch the texture to my desired size that differs from the actual resolution (and crop any excess off the screen), and then to Construct a Surface for the setPreviewDisplay function with the Surface(SurfaceTexture surfaceTexture) constructor for a Surface. I would like to avoid using a TextureView due to incompatibility below ICS, and also because this adds significant complexity.
This seems like a delicate process, but I am hoping someone can offer some advice in this area.
Thank you.
a.Assume the user sets the size of x,y as video size
b.Now with getSupportedVideoSizes function get the entire list and see if x,y falls in one of them and set the MediaRecorder.setVideoSize().If x,y does not fall in the getSupportedVideoSizes list,then set the default profile for the video record.
This is about the video size
Now coming to the preview size,Not much workaround options.
Take a RelativeLayout which holds the SurfaceView.
<android.view.SurfaceView xmlns:android="http://schemas.android.com/apk/res/android"
android:id="#+id/preview"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
/>
preview is the name of the SurfaceView.
Here i have given a sample of re-sizing it to half of the width and height.
resetCamera(); //reset the camera
ViewGroup.LayoutParams params = preview.getLayoutParams();
RelativeLayout myRelLayout = (RelativeLayout) findViewById(R.id.myRelLayout);
params.width = (int) (myRelLayout.getWidth()/2);
params.height = (int)(myRelLayout.getHeight()/2);
preview.setLayoutParams(params);
initCamera(); //initiate the camera(open camera, set parameter, setPreviewDisplay,startPreview)
please look at the resolution of the preview and then scale down the height or width accordingly based on the video size.
Hope it helps.
As you mention, this is only possible when getSupportedVideoSizes() returns a non-null list.
But if you do see a non-null list, then this simple approach should work:
Set the desired preview resolution with setPreviewSize; the size you select has to be one of the sizes given from getSupportedPreviewSizes.
Set the preview display to your SurfaceView or SurfaceTexture with setPreviewDisplay or setPreviewTexture, respectively.
Start preview.
Create the media recorder, and set its video size either directly with setVideoSize using one of the sizes from getSupportedVideoSizes, or use one of the predefined Camcorder profiles to configure all the media recorder settings for a given quality/size.
Pass the camera object to MediaRecorder's setCamera call, configure the rest of the media recorder, and start recording.
On devices with a non-null getSupportedVideoSizes list, this should result in preview staying at the resolution set by your setPreviewSize call, with recording operating at the set video size/camcorder profile resolution. On devices with no supported video sizes, the preview size will be reset by the MediaRecorder to match the recording size. You should be able to test this by setting a very low preview resolution and a high recording resolution (say, 160x120 for preview, 720p for recording). It should be obvious if the MediaRecorder switches the preview resolution to 720p when recording starts, as the preview quality will jump substantially.
Note that the preview size is not directly linked to the dimensions of the display SurfaceView; the output of the camera preview will be scaled to fit into the SurfaceView, so if your SurfaceView's dimensions are, say 100x100 pixels due to your layout and device, whatever the preview resolution you use will be scaled to 100x100 for display. So you still need to make sure to keep the SurfaceView's aspect ratio correct so that the preview is not distorted.
And for power efficiency, you should not use a preview resolution much higher than the actual number of pixels in your SurfaceView, since the additional resolution will be lost in fitting the preview in the surfaceview. This is of course only possible for recording when getSupportedVideoSizes() returns a non-null value.
First, I will try to answer your specific questions.
it is possible to provide a preview with a different resolution than the actual video. Is this understanding correct?
Yes, preview size is more often than not different from recording size. Preview size is more often than not linked to your display size. So if a phone has display of CIF (352 x 288), but is capable of recording D1 (720 x 480), then preview size and recording size will be different. I feel that other experts have answered sufficiently on this point.
Do some phones allow the behavior and others not?
Most of the latest phones support this feature except maybe a few low-end ones.
Along with setPreviewDisplay, we have to consider this point also:
The one exception is that if the preview surface is not set (or set to null) before startPreview() is called, then this method may be called once with a non-null parameter to set the preview surface. (This allows camera setup and surface creation to happen in parallel, saving time.) The preview surface may not otherwise change while preview is running.
Could you please share the issue faced by you when setPreviewDisplay is invoked with a NULL surface?
I've always been under the impression that the preview and the final output are not connected in any way; meaning that I can set the preview to be some arbitrary dimension and that the final JPG will be whatever specific resolution I set in to be in the params, but I just ran into a very odd situation where the image data coming back in the byte[] that's in the jpg callback is different, depending on what dimensions I set my preview to.
Can someone enlighten me on what actual relationship the preview has on the final JPG? (or point me to documentation on said relationship).
TIA
[Edit]
As per ravi's answer, this was my assumption as well, however, I see no alternative but to surmise that they are, in fact, directly connected based on the evidence. I'll post code if necessary (though there's a lot of it) but here's what I'm doing.
I have a preview screen where the user takes a photo of themselves. I then display the picture captured (from the jpg callback bitmap data) in a subsequent draw view and allow them to trace a shape over their photo. I then pass the points of their polygon into a class that cuts that shape out of the original image, and gives back the cut image.
All of this works, BUT depending on how I present the PREVIEW, the polygon cutting class crashes on an array out of bounds index as it tries to access pixels on the final image that simply don't exist. This effect is produced EXCLUSIVELY by altering the shape of the preview View's dimensions. I'm not altering ANYTHING else in the code, and yet, just by mis-shaping my preview view, I can reproduce this error 100% of the time.
I can't see an explanation other than that the preview and the final are directly connected somehow, since I'm never operating on the preview's data, I only display it in a SurfaceView and then move on to deal exclusively with the data from the JPG callback following the user having taken their photo.
There is no relation between the preview resolution and the final image that is captured.
They are completely independent (at least for the still image capture). The preview resolution and the aspect ratio are not interrelated with the final image resolution and the aspect ratio in anyway.
In the camera application that I have written, the preview is always VGA but the image I capture varies from 5M to VGA (depending on the device capability)
Perhaps if you can explain the situation it would be more helpful.
We are currently developing a camera application and face very similiar problems. In our case, we want to display a 16:9 preview, while capturing a 4:3 picture. On most devices this works without any problems, but on some (e.g. Galaxy Nexus, LG Optimus 3D), the output camera picture depends on the preview you've chosen. In our case the outcoming pictures on that devices are distorted when the preview ratio is different from the picture ratio.
We tried to fix this, by changing the preview resolution to a better one just before capturing the image. But this does not work on some devices and occure error while starting the preview again after capturing is finished.
We also tried to fix this, by enlarging the SurfaceView to fullscreen-width and "over fullscreen"-height to make a 16:9 preview out of a 4:3 preview. But this does not work, because SurfaceViews can not be higher then screenheight.
So there IS any connection on SOME devices, and we really want to know, how to fix/workaround this.
I need a little help with getting my camera to work right.
What I'm trying to do is have the user take a picture that will then be used in another activity as the view's background. It is important not to have any skewing, and ideally the image would fill the entire background with the highest resolution possible.
I've been having a heck of a time trying to get the outputted picture of my camera to be oriented properly and be the same aspect of the display. So I took some time to think of exactly what I needed to do, and I don't think I need the normal saved image at all.
What I came up with is that I need a surface view to display the preview, and an overlay for some text and a capture button. When the user "takes the picture" it should autofocus, and then I need to capture the preview (under the screen overlays) to a bitmap to use in the other activity.
*Should I extend a SurfaceView for my preview and add it to a XML layout that contains the overlays?
*How do I save the SurfaceView's image to a bitmap?
Thanks.
Matt,
One basic question ,and excuse my naivety, wouldnt it just be easier to use the built in camera to the Android through an Intent? It is doable, I've done it before.
Apparently, there is no good way to convert the image format of the preview frames to a jpeg, so I ended up selecting the size for the camera to take by going through each of the camera's supported resolutions and getting the closest match the the screen aspect with the highest resolution.
Because the camera.setRotation method doesn't seem to do anything, I just rotate the image 90 with a matrix before saving it to the card if I am in portrait mode.