Develop Reference

Android Camera2 take picture while processing frames

I am using Camera2 API to create a Camera component that can scan barcodes and has ability to take pictures during scanning. It is kinda working but the preview is flickering - it seems like previous frames and sometimes green frames are interrupting realtime preview.
My code is based on Google's Camera2Basic. I'm just adding one more ImageReader and its surface as a new output and target for CaptureRequest.Builder. One of the readers uses JPEG and the other YUV. Flickering disappears when I remove the JPEG reader's surface from outputs (not passing this into createCaptureSession).
There's quite a lot of code so I created a gist: click - Tried to get rid of completely irrelevant code.

Is the device you're testing on a LEGACY-level device?
If so, any captures targeting a JPEG output may be much slower since they can run a precapture sequence, and may briefly pause preview as well.
But it should not cause green frames, unless there's a device-level bug.

If anyone ever struggles with this. There is table in the docs showing that if there are 3 targets specified, the YUV ImageReader can use images with maximum size equal to the preview size (maximum 1920x1080). Reducing this helped!

Yes you can. Assuming that you configure your preview to feed the ImageReader with YUV frames (because you could also put JPEG there, check it out), like so:
mImageReaderPreview = ImageReader.newInstance(mPreviewSize.getWidth(), mPreviewSize.getHeight(), ImageFormat.YUV_420_888, 1);
You can process those frames inside your OnImageAvailable listener:
#Override
public void onImageAvailable(ImageReader reader) {
Image mImage = reader.acquireNextImage();
if (mImage == null) {
return;
}
try {
// Do some custom processing like YUV to RGB conversion, cropping, etc.
mFrameProcessor.setNextFrame(mImage));
mImage.close();
} catch (IllegalStateException e) {
Log.e("TAG", e.getMessage());
}

android camera2 api - How can i add a company logo to the recorded video file

Hy everyone.
I am trying to add a small logo in the corner to the video i've recorded . I've tried to add the imageView directly to the recording surface but it is not the solution.
I guess i'll have to create another surface and merge them together but i couldn't find any tutorial or code sample for such a thing.
I've found the option to add a foreground drawable but it doesn't show the logo on the preview surface.
This is the code:
private void startRecording(){
try {
setupMediaRecorder();
mTextureView.setForeground(getDrawable(R.drawable.toolbarlogo));
SurfaceTexture surfaceTexture = mTextureView.getSurfaceTexture();
surfaceTexture.setDefaultBufferSize(mPreviewSize.getWidth(),mPreviewSize.getHeight());
Surface previewSurfice = new Surface(surfaceTexture);
Surface recordSurface = mMediaRecorder.getSurface();
mPreviewCaptureRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
mPreviewCaptureRequestBuilder.addTarget(previewSurfice);
mPreviewCaptureRequestBuilder.addTarget(recordSurface);
mCameraDevice.createCaptureSession(Arrays.asList(previewSurfice, recordSurface), new CameraCaptureSession.StateCallback() {
#Override
public void onConfigured(#NonNull CameraCaptureSession session) {
try {
session.setRepeatingRequest(
mPreviewCaptureRequestBuilder.build(),null,null
);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
#Override
public void onConfigureFailed(#NonNull CameraCaptureSession session) {
}
},null);
} catch (Exception e) {
e.printStackTrace();
}
}
please help.
Thanks

Adding a separate surface will just show the logo while you are viewing it in your app - if you want to logo to be added to the file so it can be viewed if you share or load the video, then you need to add it to the video itself.
Assuming you capture the video to your local device, then one way you can do this is to use ffmpeg to add an image to the video - see this answer which includes notes on how to position the image:
https://stackoverflow.com/a/10920872/334402
There are several ways to include ffmpoeg in an Android project, but maybe the easiest is to use a well supported ffmpeg wrapper project like this one:
https://github.com/WritingMinds/ffmpeg-android-java
The wrappers basically wrap an interface around there command line ffmpeg tool, which has the advantage that you can use the same syntax - e.g. the syntax in the answer noted above, and leverage the support and Q&A on the web around it.
The disadvantage is that the command line tool was not originally designed to be used this way, but if you use a well supported wrapper you will likely find a lot of the problems have been ironed out.
One thing to note - video processing has quite high CPU and hence battery requirements on a mobile device. If you are going to be uploading the video to a server to share it, it may make more sense to add the image here where you have more CPU horsepower.

How to disable/modify AutoFocus and AutoWhiteBalance on Android Camera using OpenCV

I'm using Android + Opencv(new to opencv) and I'm currently working with real time object detection (the object stays really close to the android device Camera) , and I noticed that the Android camera's autoFocus keeps modifying my frames (kind of 'zoom in' and 'zoom out' effect) which make it harder for me to keep tracking the object.
I need to turn the "AUTO FOCUS" off because in my case the more blurred image input I have, the better, and I also need to turn the AutoWhiteBalance off as well, or maybe set to a different value.
I would like to know how to do it through my OpenCV CameraBridgeViewBase so I could modify the camera's Focus/WhiteBalance settings.
I've trying to find a way to solve it, and I noticed that many people face the same problems.
Here, at Stack Overflow, would be a great place to find someone who have worked with that and found a good way to overcome these problems.

create your own subclass of javacameraview
public class MyJavaCameraView extends JavaCameraView {
where you can have access to mCamera;
add whatever camera access using method you are interested in
for example
// Setup the camera
public void setFlashMode(boolean flashLightON) {
Camera camera = mCamera;
if (camera != null) {
Camera.Parameters params = camera.getParameters();
params.setFlashMode(Camera.Parameters.FLASH_MODE_TORCH);
camera.setParameters(params);
and use this new class as part of the main activity
//force java camera
mOpenCvCameraView = (MyJavaCameraView) findViewById(R.id.activity_surface_view);
mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
mOpenCvCameraView.setCvCameraViewListener(this);
mOpenCvCameraView.enableView();

Does Face Detection require a SurfaceView to work?

Using API 14 I have created an Activity which uses face detection successfully (I'm a bit of a newb at this Face Detection stuff).
I don't want to show the preview of the Camera however; I just want to know when the user's face is in front of the camera. I added a few buttons to make the SurfaceView Visible/Invisible and I found that the face detection stops working when it is INVISIBLE or GONE.
Is there a way to enable the face detection without requiring the SurfaceView in the layout?
Here is how I have coded it:
mCamera.setPreviewDisplay(mSurfaceHolder);
mCamera.startPreview();
if(mCamera.getParameters().getMaxNumDetectedFaces() >0) {
mCamera.setFaceDetectionListener(new Camera.FaceDetectionListener() {
#Override public void onFaceDetection(Face[] faces, Camera camera) {
if(faces.length > 0) {
System.out.println("Found someone");
}
}
});
mCamera.startFaceDetection();
}
To hide the surfaceview I have added a black View. :-)
<View android:layout_width="fill_parent"
android:layout_height="fill_parent"
android:background="#000"/>
I've had a reasonable look around and I haven't found much code using startFaceDetection() yet.
Thanks for any ideas / help.

You should rather use a dummy SurfaceTexture for your purposes.
Just create a SurfaceTexture object by passing any integer such as
mSurfaceTexture = new SurfaceTexture(1);
Now, open your camera and do the following:
mCamera = Camera.open(Camera.CameraInfo.CAMERA_FACING_FRONT);
try{
mCamera.setPreviewTexture(mSurfaceTexture);
}
catch (IOException t) {
//Do Something here
}
3) You can do everything else just the same way i.e. using face detection.

The Camera Preview doesn't get displayed if you omit
setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
I have not yet tested this with Face Detection (no 4.0 device and the emulator doesn't yet support it). But it should work.
Reference: https://groups.google.com/forum/?fromgroups#!topic/android-developers/EzBgJRetaCo
You could also attempt to use setPreviewTexture(SurfaceTexture st) instead of setPreviewDisplay and use a SurfaceTexture that you can control.

ColorEffect not visible on camera preview

I am trying to get a camera preview with a color effect applied to it, such as for example the NEGATIVE effect. There are no errors, and the preview is visible without problems, but independent of the ColorEffect I set - the camera preview remains unchanged. I tested if the effects I am trying to use are available to my phone by running params.getSupportedColorEffects() (also these effects also work in the built in photo app).
I have no idea what is wrong with the code - I am posting it below. Perhaps someone here has an idea what could make this work? Thanks in advance.
public class CustomCameraView extends SurfaceView{
Camera mCamera;
SurfaceHolder mHolder;
public CustomCameraView(Context context){
super(context);
mHolder = this.getHolder();
mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
mHolder.addCallback(mSurfaceHolderListener);
}
SurfaceHolder.Callback mSurfaceHolderListener = new SurfaceHolder.Callback() {
public void surfaceCreated(SurfaceHolder holder) {
mCamera=Camera.open();
try {
mCamera.setPreviewDisplay(mHolder);
}
catch (Exception e){ }
}
public void surfaceChanged(SurfaceHolder holder, int format, int width,
int height)
{
Camera.Parameters params = mCamera.getParameters();
params.setColorEffect(Camera.Parameters.EFFECT_NEGATIVE);
mCamera.setParameters(params);
mCamera.startPreview();
}
public void surfaceDestroyed(SurfaceHolder arg0)
{
mCamera.stopPreview();
mCamera.release();
}
};
}
After some testing it turned out the problem could be related to the HTC Desire I was testing on (or maybe its OS version). The code works correctly on some other Samsung phones. I haven't figured out what could be the problem on the HTC.
UPDATE:
I have managed to get the effects working, but truly by accident, and I still don't understand why. But I will give the answer here - perhaps someone will find it useful, or maybe will be able to explain why it happens this way:
I added the following line to the surfaceChanged method because I was trying to decrease the size of the preview:
previewHolder.setFixedSize(width, height-1);
This had the result of making the selected effect visible.
When I changed this line to:
previewHolder.setFixedSize(width, height);
the effect was not visible any more once again. So odd.... it works for set height being anything less than the received height parameter.

I have been struggling with this as well. I found out that the HTC Desire its camera needs a strange order of executing the setParameters, setPreviewDisplay and startPreview for the color effect to work. The order is:
Camera.Parameters parameters = camera.getParameters();
//set the parameters
camera.setParameters(parameters);
camera.startPreview();
camera.setParameters(parameters);
camera.setPreviewDisplay(surfaceHolder);
Calling startPreview before setPreviewDisplay is documented in the Android SDK as a way of initializing the camera and the surfaceView in parallel.
Regarding your update about getting the effects to work by accident, the same happend to me! I assume for the same reason, some of my code got called twice in quick succesion (in my case due to a changing database object). This caused the method to (re)set the parameters and (re)start the preview to be called twice producing the desired result. After realising this and some more experimenting the above order seemed to work on both my HTC Desire and Acer Iconia A500 and I was quite happy with it.
However I have just received a comment for my application saying it produces corrupted images on the HTC Desire HD so I would recommend not using this order of camera initialization as a default but rather as a fix for the HTC Desire.

After setting new parameters to camera and starting preview invalidate() are calling on your SurfaceView . But it only Invalidate the whole view. If the view is visible, onDraw(android.graphics.Canvas) will be called at some point in the future. So there is no guarantees that onDraw() will be called immediately. But onDraw() are always invoking after calling onMeasure() with size differs from current. So it can be a reason of this odd behavior.

Simple answers use following type :
Camera camera = null;
camera = Camera.open();
if (camera != null) {
try {
Camera.Parameters parameters = camera.getParameters();
// Set all kind of stuffs here..
parameters.setSceneMode(Camera.Parameters.FLASH_MODE_AUTO);
parameters.setColorEffect(Camera.Parameters.EFFECT_SEPIA); // whatever effect you want
camera.setParameters(parameters);
camera.setPreviewDisplay(surface_holder);
camera.startPreview();
} catch (IOException exception) {
camera.release();
camera = null;
}
}