Android camera: very low preview FPS - android

A week ago I've started researching the Android camera API. I have successfully inited the camera and started preview, and it worked fine. Then I've found out I wasn't initializing and releasing the camera properly, so I overhauled the code somewhat, and now I have a problem that didn't occur initially: extremely low FPS. About 0.5, that's 2 seconds per frame. Interestingly enough, I get 1 frame with a delay, and then a second frame immediately after (1-15 ms), followed again by 2 seconds delay before the next frame.
This is my camera initialization code:
m_openedCamera = Camera.open(id);
m_surfaceHolder = new SurfaceView(MyApplication.instance().getApplicationContext()).getHolder();
Assert.assertNotNull(m_openedCamera);
// This is required on A500 for some reason
Camera.Parameters params = m_openedCamera.getParameters();
params.setPreviewFormat(ImageFormat.NV21);
params.setPreviewSize(320, 240);
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.ICE_CREAM_SANDWICH)
{
params.setRecordingHint(true);
params.setAutoExposureLock(true);
params.setAutoWhiteBalanceLock(true);
}
m_openedCamera.setParameters(params);
int bitsPerPx = ImageFormat.getBitsPerPixel(ImageFormat.NV21);
int width = params.getPreviewSize().width;
int height = params.getPreviewSize().height;
int size = (int)(width * height * bitsPerPx / 8.0);
m_openedCamera.addCallbackBuffer( new byte[size] );
m_openedCamera.addCallbackBuffer( new byte[size] );
m_openedCamera.addCallbackBuffer( new byte[size] );
m_openedCamera.addCallbackBuffer( new byte[size] );
m_openedCamera.setErrorCallback(this);
m_openedCamera.setPreviewDisplay(m_surfaceHolder);
m_openedCameraFacing = facing;
m_openedCamera.setPreviewCallback(this);
m_openedCamera.startPreview();
I have just added callback buffers - hasn't changed anything. In my initial code from a week ago I had no surface view, but removing it now has no effect either.
This occurs on my second, much newer tablet as well, even though FPS is higher there (8-10), and there's no double frame there, the frames are spaced evenly. The FPS used to be at least 20. Light conditions haven't changed between now and then, btw.
Update: tried opening the camera in a separate thread as described here - no change.

params.setRecordingHint(true); is used to start video record starts.
Please check it again with "params.setRecordingHint(false)"

Please make your own previewcallbak function and then
private PreviewCallback mPreviewCallback = new PreviewCallback();
....
setPreviewCallback(mPreviewCallback);
Regarding PreviewCallback(), you can refer to CameraTest.Java in cts directory : cts/tests/tests/hawdware/src/android/hardware/cts

From your comment below
"I have successfully inited the camera and started preview, and it worked fine. Then I've found out I wasn't initializing and releasing the camera properly"
I guess you was using the default camera parameters defined in camera HAL at first. Although it is not clear what "initializing" means, I think they is likely camera parameters setting.
So, I'd like to suggest you remove the code for params and then test it again.
Or, you can test it one by one
Remove only "params.setPreviewSize(320, 240);"
Remove only "params.setPreviewFormat(ImageFormat.NV21);"
Remove both 1 and 2.

Related

Recording darkens preview and ratios

This is coded in NativeScript, so I'll try my best to adapt the scenario to Java. I have created an in-app video view with support to record the video.
This is done as follows:
First I create a SurfaceView that will hold the preview of the camera:
this.mSurfaceView = new android.view.SurfaceView(this._context);
this.mHolder = this.mSurfaceView.getHolder();
this.mHolder.setType(android.view.SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
Then I create an instance of the Camera, and sets the video surface:
var mCamera = android.hardware.Camera;
var camera = mCamera.open(1);
this.camera = camera;
this.camera.setDisplayOrientation(90);
var parameters = camera.getParameters();
parameters.setRecordingHint(true);
if( parameters.isVideoStabilizationSupported() ){
parameters.setVideoStabilization(true);
}
camera.setParameters(parameters);
this.camera.setPreviewDisplay(_this.mHolder);
this.camera.startPreview();
this.camera.startFaceDetection();
Now, all is good. I have the camera preview in the view that I want it to be. The color is good and I think the image aspect ratio is good too.
However, when I initiate the recording, as I do with the following code:
this.mediarecorder = new android.media.MediaRecorder();
// Step 1: Unlock and set camera to MediaRecorder
this.camera.unlock();
this.mediarecorder.setCamera(this.camera);
// Step 2: Set sources
this.mediarecorder.setAudioSource(android.media.MediaRecorder.AudioSource.CAMCORDER);
this.mediarecorder.setVideoSource(android.media.MediaRecorder.VideoSource.CAMERA);
//this.mediarecorder.setOutputFormat(android.media.MediaRecorder.OutputFormat.MPEG_4);
// Step 3: Set a CamcorderProfile (requires API Level 8 or higher)
this.mediarecorder.setProfile(android.media.CamcorderProfile.get(android.media.CamcorderProfile.QUALITY_HIGH));
// platform.screen.mainScreen.widthDIPs
// platform.screen.mainScreen.heightDIPs
// Step 4: Set output file
var fileName = "videoCapture_" + new Date() + ".mp4";
var path = android.os.Environment.getExternalStoragePublicDirectory(android.os.Environment.DIRECTORY_DCIM).getAbsolutePath() + "/Camera/" + fileName;
this.file = new java.io.File(path);
this.mediarecorder.setOutputFile(this.file.toString());
this.mediarecorder.setOrientationHint(270);
try {
this.mediarecorder.prepare();
this.mediarecorder.start();
} catch( ex ) {
console.log(ex);
}
Then, the image suddenly becomes darker, and my face (its what's in focus when I'm trying it out) gets wider. So the aspect ratio changes, and so does the lighting somehow.
I have tried setting setPictureSize on the camera parameters, and setVideoSize on the MediaRecorder with no luck. And for the lighting change, I have simply no clue as to whats going on. Now I've been googling myself half way to heaven, and still found nothing, so I hope someone here has got any tip on what to pursue next?
Video recording generally tries to run at a steady frame rate, such as 30fps. Camera preview often slows itself down to 10-15fps to maintain brightness, so if you're in a darker location, video recording will be darker (since it can't expose for longer than 1/30s instead of 1/10s that camera preview can).
Did you call setVideoSize before or after calling setProfile? The setProfile call changes many parameters, including preview size; most video recording sizes are 16:9, and the default camera preview resolution is likely a 4:3 size. So when you start the recording, the aspect ratio switches.
Most video recording apps use 16:9 preview sizes even before starting recording so that they're consistent. You can also record 4:3 video, but that's generally not what people want to see.

Android MediaRecorder saving empty file and weird camera behaviours

I am having a headache over the Camera API 1 for android. After reading all of the Internet content, I made some sample app that works OK. It creates a service, which then is used to operate with the camera in the background, so there is no preview or activity enabled. To achieve this I use a dummy SurfaceHolder, like this:
protected class MySurfaceHolder implements SurfaceHolder {
private final Surface surface;
private final SurfaceTexture surfaceTexture;
public MySurfaceHolder () {
int[] textures = new int[1];
GLES20.glGenTextures(1, textures, 0);
if (textures.length > 0) {
this.surfaceTexture = new SurfaceTexture(textures[0]);
this.surface = new Surface(this.surfaceTexture);
} else {
this.surface = null;
this.surfaceTexture = null;
}
}
[...]
}
and then I use it like this
// simplified version of my code
try {
initializeCamera(); // open camera and set Camera.Parameters
camera.setPreviewDisplay(new MySurfaceHolder());
camera.startPreview();
camera.unlock();
initializeMediaRecorder(); // create MediaRecorder, set video/audio parameters
mediaRecorder.prepare();
mediaRecorder.start();
// wait until recording finish and exit
} finally {
stopRecording();
}
the Camera and MediaRecorder initialization methods are just like the documentation states they should be (and they work).
Everything works and operates as it should. Almost everything - sometimes, under unknown circumstances the MediaRecorder creates empty files, like 32kB containing only headers and info about the video - no frames. The longer I record like this, the bigger is the file (few kB every few seconds). After 1 minute, the file weights about 80kB. Funny thing is I know that the camera is working and capturing frames (I debugged it a little showing preview frames), but the frames are not written into the output file.
Also when it happens I am not able to record in FHD (1920x1080) - I get the "start failed" message - at this time camera is not capturing frames. The same thing could happen when I use wrong (not supported) video size. I suppose in this case the message is thrown at the mediaRecorder.start(); line, and stopRecording(); is invoked but I am not sure.
After some time or after unknown action the problem is suddenly gone (I don't know when, I don't know how). It happens for sure on Android 5.1, but may happen on other versions as well.
Could this bug be related to my custom surface code?
What could cause the MediaRecorder to not write frames into a file?
Why I am not able to record in FHD, but in the same time I am able to record in HD (1280x720)?
Is there any alternative for MediaRecorder, so I can avoid these bugs?
May it happen when another app is trying to get Camera object, thus distrupting current recording? If so, how to regain access to the Camera object (I apparently am not able to do this now on some devices).
EDIT:
I think I might have a clue. I am calling
camera.setOneShotPreviewCallback(new Camera.PreviewCallback() {
// ... get current frame
}
camera.startPreview();
to get preview frame of current recording. It appears that the bug occurs when I am using this method to get preview frame (at random times). It seems flawed, because not all devices react to this thing properly (sometimes there is no preview frame...). Is there any other, better method of handling current preview frame without the real surface?

Android camera API blurry image on Samsung devices

After implementing the camera2 API for the inApp camera I noticed that on Samsung devices the images appear blurry. After searching about that I found the Sasmung Camera SDK (http://developer.samsung.com/galaxy#camera). So after implementing the SDK on Samsung Galaxy S7 the images are fine now, but on Galaxy S6 they are still blurry. Someone experienced those kind of issues with Samsung devices?
EDIT:
To complement #rcsumners comment. I am setting autofocus by using
mPreviewBuilder.set(SCaptureRequest.CONTROL_AF_TRIGGER, SCaptureRequest.CONTROL_AF_TRIGGER_START);
mSCameraSession.capture(mPreviewBuilder.build(), new SCameraCaptureSession.CaptureCallback() {
#Override
public void onCaptureCompleted(SCameraCaptureSession session, SCaptureRequest request, STotalCaptureResult result) {
isAFTriggered = true;
}
}, mBackgroundHandler);
It is a long exposure image where the use has to take an image of a static non moving object. For this I am using the CONTROL_AF_MODE_MACRO
mCaptureBuilder.set(SCaptureRequest.CONTROL_AF_MODE, SCaptureRequest.CONTROL_AF_MODE_MACRO);
and also I am enabling auto flash if it is available
requestBuilder.set(SCaptureRequest.CONTROL_AE_MODE,
SCaptureRequest.CONTROL_AE_MODE_ON_AUTO_FLASH);
I am not really an expert in this API, I mostly followed the SDK example app.
There could be a number of issues causing this problem. One prominent one is the dimensions of your output image
I ran Camera2 API and the preview is clear, but the output was quite blurry
val characteristics: CameraCharacteristics? = cameraManager.getCameraCharacteristics(cameraId)
val size = characteristics?.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)?.getOutputSizes(ImageFormat.JPEG) // The issue
val width = imageDimension.width
val height = imageDimension.height
if (size != null) {
width = size[0].width; height = size[0].height
}
val imageReader = ImageReader.newInstance(width, height, ImageFormat.JPEG, 5)
The code below was returning a dimension about 245*144 which was way to small to be sent to the image reader. Some how the output was stretching the image making it end up been blurry. Therefore I removed this line below.
val size = characteristics?.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)?.getOutputSizes(ImageFormat.JPEG) // this was returning a small
Setting the width and height manually resolved the issue.
You're setting the AF trigger for one frame, but then are you waiting for AF to complete? For AF_MODE_MACRO (are you verifying the device lists support for this AF mode?) you need to wait for AF_STATE_FOCUSED_LOCKED before the image is guaranteed to be stable and sharp. (You may also receive NOT_FOCUSED_LOCKED if the AF algorithm can't reach sharp focus, which could be because the object is just too close for the lens, or the scene is too confusing)
On most modern devices, it's recommended to use CONTINUOUS_PICTURE and not worry about AF triggering unless you really want to lock focus for some time period. In that mode, the device will continuously try to focus to the best of its ability. I'm not sure all that many devices support MACRO, to begin with.

Android camera preview is dark

I am trying to create a custom camera app. I followed the Android Developer example from here with minor tweaks. However, my camera preview turns out to be rather dark. On the other hand, the stock camera gives a much brighter preview.
I have tried several settings to make it work better but it seems none of them are having any impact. Relevant code is posted here.
CameraActivity (Main)
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_camera);
if(CameraHelper.checkCameraHardware(this)) {
mHelper = new CameraHelper(this, getWindowManager().getDefaultDisplay());
}
FrameLayout preview = (FrameLayout) findViewById(R.id.camera_preview);
mPreview = new CameraPreview(this, CameraHelper.camera);
mPreview.setLayoutParams(new LayoutParams(CameraHelper.mSize.width, CameraHelper.mSize.height, Gravity.CENTER));
preview.addView(mPreview);
}
CameraHelper class (initialize the camera and set the default parameters)
public CameraHelper(CameraListener listener, Display display){
mListener = listener;
camera = getCameraInstance();
mParameters = camera.getParameters();
initCameraParameters();
mSize = getPreviewSize(display);
mParameters.setFocusMode(Parameters.FOCUS_MODE_AUTO);
mParameters.setPictureSize(2560, 1920);
mParameters.setAutoExposureLock(false);
mParameters.setAutoWhiteBalanceLock(false);
mParameters.set("iso", "ISO800"); //Tried with 400, 800, 600 (values obtained from flatten())
mParameters.setColorEffect("none");
mParameters.setPictureSize(2560, 1920);
mParameters.setPreviewFrameRate(20);
mParameters.set("scene-mode", "auto");
mParameters.setFocusMode("auto");
mParameters.setExposureCompensation(4);
camera.setParameters(mParameters);
}
The Camera sends the frames to SurfaceHolder.Surface from the example linked from developer pages above.
See the difference here:
Stock Camera App
My Camera App
Tried setting the ISO, etc based on upack parameters from the camera as posted here. It still didn't work.
Parameters(16369):
effect-values=none,mono,negative,sepia,aqua,sharpen,purple,green-tint,blue-tint,pink,yellow,red-tint,mono,antique;exposure-compensation-step=0.5;focal-length=3.43;focus-areas=(0,0,0,0,0);focus-distances=0.10,1.20,Infinity;focus-mode-values=auto,macro,facedetect;gps-altitude=0;gps-latitude=0;gps-longitude=0;gps-processing-method=GPS;gps-timestamp=0;horizontal-view-angle=51.2;iso=auto;iso-values=auto,ISO50,ISO100,ISO200,ISO400,ISO800,ISO1600;jpeg-quality=1;jpeg-thumbnail-height=480;jpeg-thumbnail-size-values=640x480,0x0;jpeg-thumbnail-width=640;max-exposure-compensation=4;max-num-focus-areas=1;max-zoom=12;min-exposure-compensation=-4;picture-format=jpeg;picture-format-values=jpeg;picture-size-values=2560x1920,2560x1536,2048x1536,2048x1232,1600x1200,1600x960,800x480,640x480;preview-format=yuv420sp;preview-format-values=yuv420sp;preview-fps-range=15000,30000;preview-fps-range-values=(15000,30000);preview-frame-rate=30;preview-frame-rate-values=30;preview-size=640x480;preview-size-values=1280x720,800x480,720x480,640x480,352x288;rotation=0;scene-mode=auto;scene-mode-values=auto,portrait,landscape,night,beach,snow,sunset,fireworks,sports,party,candlelight,asd,backlight,dusk-dawn,text,fall-color;vertical-view-angle=39.4;video-frame-format=yuv422i-yuyv;whitebalance-values=auto,incandescent,fluorescent,daylight,cloudy-daylight;zoom=0;zoom-ratios=100,125,150,175,200,225,250,275,300,325,350,375,400;zoom-supported=true;focus-mode=auto;picture-size=2560x1920;exposure-compensation=4;
Edit: Upon further testing based on comments below, it appears that its just the preview that is turning out darker than it should be. The actual captured image is well lit and exposure compensatiion seems to be working fine. Its just the preview that is giving me a headache. Tested on i9003 running CM11 and Nexus 10 running stock android.
There appears to be a bug with certain cameras reporting the supported preview FPS range incorrectly. You can identify the offending devices by those that return the same value for min and max when calling
getPreviewFpsRange (int[] range)
In my case I saw this issue with devices that reported (15000, 15000) and (30000, 30000), but not with devices where the values were different, like (7000, 30000).
The best solution I could find was to identify the supported FPS range that had different values for min and max, and set that:
Camera.Parameters params = camera.getParameters();
final int[] previewFpsRange = new int[2];
params.getPreviewFpsRange(previewFpsRange);
if (previewFpsRange[0] == previewFpsRange[1]) {
final List<int[]> supportedFpsRanges = params.getSupportedPreviewFpsRange();
for (int[] range : supportedFpsRanges) {
if (range[0] != range[1]) {
params.setPreviewFpsRange(range[0], range[1]);
break;
}
}
}
camera.setParameters(params);
This works because the ranges reported seem to only have 1 item with the actual range. Eg:
BLU Vivo XL:
preview-fps-range=30000,30000
preview-fps-range-values=(15000,15000),(20000,20000),(24000,24000),(5000,30000),(30000,30000)
Pixel:
preview-fps-range=7000,30000
preview-fps-range-values=(15000,15000),(24000,24000),(7000,30000),(30000,30000)
A more robust approach would be to set the min and max by comparing all those available.
In addition to the previous answers, this can happen with Camera2 if you are doing
createCaptureRequest(CameraDevice.TEMPLATE_RECORD)
change to
createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
UPDATE: I have also begun to see a dark preview on some newer Pixel devices and this happens if you don't set the fps in the capture request or if you set the fps to something that the device can't handle BUT not on Samsung devices like the Note 10 and S10
From my experiments, scene-mode setting can change the preview (unlike ISO or exposure-compensation, which both work for captured pictures). Don't use auto. Try scene-mode-values=night or scene-mode=dusk-dawn.
The problem with scenes is that the supported values are not standardized across devices. But some kind of night is usually present.

Android: recorded video seems *distorted*

I'm trying to record video from the Camera using the MediaRecorder. Here's a code snippet
snip..
mr.setAudioSource( MediaRecorder.AudioSource.MIC );
mr.setVideoSource( MediaRecorder.VideoSource.CAMERA);
mr.setOutputFormat( MediaRecorder.OutputFormat.THREE_GPP );
mr.setAudioEncoder( MediaRecorder.AudioEncoder.AMR_NB );
mr.setVideoEncoder( MediaRecorder.VideoEncoder.MPEG_4_SP );
mr.setVideoSize( 200, 200 );
mr.setVideoFrameRate( 15 );
..snap
Code executes on a MileStone/Droid, non-empty output file will be created. But when I try to view the video, it looks like this:
My first thoughts were about some sort of encoding error, so I tried every possible OutputFormat/VideoEncoder combination, with no effetcs on the result.
LogCat shows the following error
CameraInput: Unsupported parameter(x-pvmf/media-input-node/cap-config-interface;valtype=key_specific_value)
But I can't figure out, what I may have set wrong. I used camera.getParameters(), set the preview size with the returned params and then pushed them back using camera.setParameters()...
Worked thru every piece of sample code I could find, but still found no solution.
Does anyone have any ideas ?
you must set the correct setVideoSize( x, y) function.
you must call the function which give you the size options , and choose from that list
when camera open,
Camera.Parameters p = mCamera.getParameters();
p.setPreviewSize(mSur.getWidth(), mSur.getHeight());
mCamera.setParameters(p);
and when you prepareRecord
mCamera.unlock();
if (mRecorder == null) {
mRecorder = new MediaRecorder();
} else {
mRecorder.reset();
}
mNextRecordFileName = getOneFileName();
mRecorder.setCamera(mCamera);
mRecorder.setVideoSource(mVideoSource);
mRecorder.setAudioSource(mAudioSource);
mRecorder.setPreviewDisplay(mSur.getHolder().getSurface());
mRecorder.setOutputFormat(mVideoFormat);
//设置在setEncoder之前才有效,如果不设置,htc会崩溃掉
mRecorder.setVideoSize(this.mSur.getWidth(), mSur.getHeight());
Log.i(TAG, this.mSur.getHolder().getSurfaceFrame().height()+"gao");
Log.i(TAG, this.mSur.getHolder().getSurfaceFrame().width()+"kuan");
Log.i(TAG, "宽"+this.mSur.getWidth()+"高"+mSur.getHeight());
mRecorder.setVideoEncoder(mVideoEncoder);
mRecorder.setAudioEncoder(mAudioEncoder);
mRecorder.setOutputFile(mNextRecordFileName);
hope it can help you
To avoid distortion in recorded video ( I have seen this on Galaxy S3) make sure you set camera parameter preview size and mediarecorder videosize to same height and width.
To get supported camera preview size:
Camera.Parameters parameters = camera.getParameters();
List<Camera.Size> list = parameters.getSupportedPreviewSizes();
parameters.setPreviewSize(list.get(X).width, list.get(X).height);
// If you set Width and Height not supported by device you will get exception
// MediaRecorder object mMediaRecoder
mMediaRecoder.setVideoSize(list.get(X).width, list.get(X).height);
Hey,
I know you posted this a while ago and I doubt your still looking for an answer but I thought this might help someone else out.
I think your problem is mr.setVideoSize( 200, 200 );
I doubt the phones camera supports a 1X1 capture resolution. It is better to use something like mr.setVideoSize(Camcorder.get(Camcorder.QUALITY_LOW).videoFrameWidth,Camcorder.get(Camcorder.QUALITY_LOW).videoFrameHeight);
That will ensure that the resolution is supported by the camera. Also make sure your preview resolution matches your camera resolution, or that can cause the same problem. I know it happens to me if I have my preview set to QUALITY_HIGH and my camera to QUALITY_LOW

Categories

Resources