I am a first time android programmer.
The project I am working on requires me to do (simple?) real time video processing.
The app, once finished needs to do this:
When we click on the inbuilt camera application, it opens. I then proceed to choose the video recording option. Using that I can see the surroundings without needing to record. What I am trying to accomplish is to delay that display by a few hundred milliseconds. A colleague of mine could do this pretty easily with the delay option using the laptop webcam and openCV (for computers). I am trying to accomplish the same with an android phone.
Perhaps I am doing a poor job of explaining the situation. Kindly reply at the earliest.
I am working on the code now and being a first time programmer taking some time.
Excited to start with Android programming!
no idea if this task actually needs opencv ( might be a bit of overkill ) but if you opt for that, its fairly easy.
see all we do here is record frames continuously, and toggle between realtime/playback mode on some event (onTouch for simplicity here):
package com.berak.echo;
import java.util.ArrayList;
import java.util.List;
import android.os.Bundle;
import android.view.Menu;
import android.view.MotionEvent;
import android.view.View;
import android.view.View.OnTouchListener;
import android.app.Activity;
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.core.Point;
import org.opencv.core.Scalar;
import com.berak.echo.R;
public class EchoActivity extends Activity implements CvCameraViewListener2, OnTouchListener {
CameraBridgeViewBase mOpenCvCameraView;
List<Mat> ring = new ArrayList<Mat>(); // recording buffer
int delay = 100; // delay == length of buffer
boolean delayed = false; // state
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_echo);
mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.cam3_surface_view);
mOpenCvCameraView.setCvCameraViewListener(this);
mOpenCvCameraView.setOnTouchListener(this); // setup as touchlistener
}
// lots of boilerplate, ugly, but needed.
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
mOpenCvCameraView.enableView();
break;
default:
super.onManagerConnected(status);
break;
}
}
};
#Override
public void onResume() {;
super.onResume();
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_5,this, mLoaderCallback);
}
#Override
public void onPause() {
super.onPause();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
#Override
public void onDestroy() {
super.onDestroy();
}
#Override
public void onCameraViewStarted(int width, int height) { }
#Override
public void onCameraViewStopped() { }
// here's the bread & butter stuff:
#Override
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
Mat mRgba = inputFrame.rgba();
ring.add(mRgba.clone()); // add one at the end
if ( ring.size() >= delay ) { // pop one from the front
ring.get(0).release();
ring.remove(0);
}
Mat ret;
String txt;
if ( delayed && ring.size()>0 ) { // depending on 'delayed' return either playback
ret = ring.get(0); // return the 'oldest'
txt = "playback";
} else {
ret = mRgba; // or realtime frame
txt = "realtime";
}
Core.putText(ret, txt, new Point(20,20), Core.FONT_HERSHEY_PLAIN, 1.2, new Scalar(200,0,0));
return ret;
}
#Override
public boolean onTouch(View v, MotionEvent event) {
// just toggle between delayed an realtime view:
delayed = ! delayed;
return false;
}
}
Related
I want to perform image processing with OpenCV and android. In the first step, I need to change the camera properties like resolution, exposure, etc. By using OpenCV I only can change the resolution(mOpenCvCameraView.setMaxFrameSize(320,240);) and cannot change exposure.
By using OpenCV and camera2 When I run it, it is crashing(this code:pastebin.com/3XgvKGQN).
How can I change camera exposure?
package com.williams.drew.opencvtest;
import android.graphics.Paint;
import android.hardware.camera2.CameraDevice;
import android.hardware.camera2.CameraMetadata;
import android.hardware.camera2.CaptureRequest;
import android.os.Bundle;
import android.support.annotation.NonNull;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.SurfaceView;
import android.view.WindowManager;
import org.opencv.android.JavaCameraView;
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.Core;
import org.opencv.core.CvType;
import org.opencv.core.Mat;
import org.opencv.imgproc.Imgproc;
public class MainActivity extends AppCompatActivity implements CvCameraViewListener2 {
//Prefixes for logging success and failure messages
private static final String TAG = "OCVSample::Activity";
//Loads camera view of OpenCV for us to use. This lets us see using OpenCV
private CameraBridgeViewBase mOpenCvCameraView;
//Preview Builder which changes exposure (i think)
private CaptureRequest.Builder mPreviewRequestBuilder;
private CaptureRequest mPreviewRequest;
private long exposureTime = 1000,frameDuration = 1000;
private int sensitivity = 200;
//OPENCV Variables
Mat matRGBA;
public MainActivity() {
Log.i(TAG, "Instantiated new " + this.getClass());
}
#Override
protected void onCreate(Bundle savedInstanceState) {
Log.i(TAG, "called onCreate");
super.onCreate(savedInstanceState);
getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
setContentView(R.layout.show_camera);
mOpenCvCameraView = (JavaCameraView) findViewById(R.id.show_camera_activity_java_surface_view);
mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
mOpenCvCameraView.setCvCameraViewListener(this);
}
#Override
public void onPause() {
super.onPause();
if(mOpenCvCameraView != null) {
mOpenCvCameraView.disableView();
}
}
#Override
public void onResume() {
super.onResume();
if(!OpenCVLoader.initDebug()) {
Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for init");
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_1_0, this, mLoaderCallback);
}
else {
Log.d(TAG, "OpenCV library found inside package. Using it!");
mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
}
}
#Override
public void onDestroy() {
super.onDestroy();
if(mOpenCvCameraView != null) {
mOpenCvCameraView.disableView();
}
}
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
Log.i(TAG, "OpenCV loaded successfully");
mOpenCvCameraView.enableView();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
#Override
public void onCameraViewStarted(int width, int height) {
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_OFF);
mPreviewRequestBuilder.set(CaptureRequest.SENSOR_EXPOSURE_TIME, Long.valueOf(exposureTime));
mPreviewRequestBuilder.set(CaptureRequest.SENSOR_SENSITIVITY, Integer.valueOf(sensitivity));
mPreviewRequestBuilder.set(CaptureRequest.SENSOR_FRAME_DURATION, Long.valueOf(frameDuration));
matRGBA = new Mat(width, height, CvType.CV_8UC4);
}
#Override
public void onCameraViewStopped() {
matRGBA.release();
}
#Override
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
matRGBA = inputFrame.rgba();
return matRGBA;
}
}
Thank you for your answers.
Yes, OpenCV is not exposing all the camera parameters. You can modify JavaCameraView and add the function that calls setExposureCompensation().
You want to call that function like this:
Camera mCamera;
mCamera = Camera.open();
Camera.Parameters params = mCamera.getParameters();
params.setExposureCompensation(0);
I want to do some image processing jobs and I want to get real-time image data from my camera(during preview state) instead of after taking pictures.
I looked on this post but don't know how to use this.
After setting this,
SurfaceHolder.Callback surfaceCallback=new SurfaceHolder.Callback()
{
public void surfaceCreated(SurfaceHolder holder) {
camera.setPreviewCallback(previewCallback);
}
}
and this
private Camera.PreviewCallback previewCallback= new Camera.PreviewCallback()
{
#Override
public void onPreviewFrame(byte[] data,Camera cam)
{
Camera.Size previewSize = cam.getParameters().getPreviewSize();
YuvImage yuvImage = new YuvImage(data, ImageFormat.NV21,previewSize.width,previewSize.height, null);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(0,0,previewSize.width,previewSize.height),80,baos);
byte[] jdata = baos.toByteArray();
Bitmap bitmap = BitmapFactory.decodeByteArray(jdata,0,jdata.length);
}
};
How can I real-time get my image data? Could anyone please kindly give me an short example code?
Thanks
For doing image processing based operations you can use with OpenCV Library.
Here i given sample code to process the frames directly from camera using OpenCV
import android.Manifest;
import android.content.pm.PackageManager;
import android.os.Bundle;
import android.support.v4.app.ActivityCompat;
import android.support.v4.content.ContextCompat;
import android.support.v7.app.AppCompatActivity;
import android.util.Log;
import android.view.View;
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.JavaCameraView;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.Mat;
import static org.opencv.core.CvType.CV_8UC1;
public class MainActivity extends AppCompatActivity implements CameraBridgeViewBase.CvCameraViewListener2 {
private static final String TAG = "MainActivity";
JavaCameraView javaCameraView;
Mat frame;
public static final int CAMERA_PERMISSION_REQUEST_CODE = 3;
static {
System.loadLibrary("MyOpenCVLibs");
}
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch(status){
case LoaderCallbackInterface.SUCCESS:
{
javaCameraView.enableView();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
if (ContextCompat.checkSelfPermission(this,Manifest.permission.CAMERA)!= PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this,new String[]{Manifest.permission.CAMERA},CAMERA_PERMISSION_REQUEST_CODE);
}
javaCameraView = (JavaCameraView) findViewById(R.id.java_camera_view);
javaCameraView.setVisibility(View.VISIBLE);
javaCameraView.setCvCameraViewListener(this);
}
#Override
protected void onPause(){
super.onPause();
if(javaCameraView!=null)
javaCameraView.disableView();
}
#Override
protected void onDestroy(){
super.onDestroy();
if(javaCameraView!=null)
javaCameraView.disableView();
}
#Override
protected void onResume(){
super.onResume();
if (OpenCVLoader.initDebug()) {
Log.i(TAG, "OpenCV loaded successfully.");
mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
} else {
Log.i(TAG, "OpenCV not loaded.");
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_1_0, this, mLoaderCallback);
}
}
#Override
public void onCameraViewStarted(int width, int height) {
frame=new Mat(height,width,CV_8UC4);
}
#Override
public void onCameraViewStopped() {
frame.release();
}
#Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
frame=inputFrame.rgba();
}
}
This code checks OpenCV properly imported or not and Checks permission for Camera.
And each input frames are saved in frame variable.
Running openCV-Android samples does not work as expected. I have Android Studio on Windows, extracted OpenCV-Android 2.4.11 and the sample is tutorial-1-camerapreview (but other samples don't seem to work also)
If I use 'Import module' and run Tutorial1CameraView on a device, everything works fine. I have the 3 normal buttons (link) PLUS a 4th one, which opens a menu.
If I use 'New project' and do as if I coded it myself (copy/pasting the code and files from the tutorial), it runs the same app, but there is no 4th button. So I can't open the menu.
Here is exactly what I've done :
New project -> blank activity
Import the openCV libs (I followed the instructions on this page)
Copy/paste the source code of Tutorial1CameraView in my MainActivity
Merge the res folder from tutorial with res folder created by Android studio. I deleted some files, like the old menu folder
(generated when I created the project, and not used by the tutorial),
to be sure they don't interfere. (But it still doesn't work if I keep
them)
Renaming some trivial things like package/classes in the java and xml files to make it compile
if it helps, my java code is :
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.Mat;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
import android.app.Activity;
import android.os.Bundle;
import android.util.Log;
import android.view.Menu;
import android.view.MenuItem;
import android.view.SurfaceView;
import android.view.WindowManager;
import android.widget.Toast;
public class MainActivity extends Activity implements CvCameraViewListener2 {
private static final String TAG = "OCVSample::Activity";
private CameraBridgeViewBase mOpenCvCameraView;
private boolean mIsJavaCamera = true;
private MenuItem mItemSwitchCamera = null;
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
Log.i(TAG, "OpenCV loaded successfully");
mOpenCvCameraView.enableView();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
public MainActivity() {
Log.i(TAG, "Instantiated new " + this.getClass());
}
/** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState) {
Log.i(TAG, "called onCreate");
super.onCreate(savedInstanceState);
getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
setContentView(R.layout.tutorial1_surface_view);
if (mIsJavaCamera)
mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.tutorial1_activity_java_surface_view);
else
mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.tutorial1_activity_native_surface_view);
mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
mOpenCvCameraView.setCvCameraViewListener(this);
}
#Override
public void onPause()
{
super.onPause();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
#Override
public void onResume()
{
super.onResume();
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_3, this, mLoaderCallback);
}
public void onDestroy() {
super.onDestroy();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
#Override
public boolean onCreateOptionsMenu(Menu menu) {
Log.i(TAG, "called onCreateOptionsMenu");
mItemSwitchCamera = menu.add("Toggle Native/Java camera");
return true;
}
#Override
public boolean onOptionsItemSelected(MenuItem item) {
String toastMesage = new String();
Log.i(TAG, "called onOptionsItemSelected; selected item: " + item);
if (item == mItemSwitchCamera) {
mOpenCvCameraView.setVisibility(SurfaceView.GONE);
mIsJavaCamera = !mIsJavaCamera;
if (mIsJavaCamera) {
mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.tutorial1_activity_java_surface_view);
toastMesage = "Java Camera";
} else {
mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.tutorial1_activity_native_surface_view);
toastMesage = "Native Camera";
}
mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
mOpenCvCameraView.setCvCameraViewListener(this);
mOpenCvCameraView.enableView();
Toast toast = Toast.makeText(this, toastMesage, Toast.LENGTH_LONG);
toast.show();
}
return true;
}
public void onCameraViewStarted(int width, int height) {
}
public void onCameraViewStopped() {
}
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
return inputFrame.rgba();
}
}
I can join any other files needed (It's probably not related to java ...)
What could be the problem ? Why does the app work but only the menu does not ?
To solve this issue, if the theme attribute for your activity in your AndroidManifest file ends in .Fullscreen, remove the .Fullscreen.
EDIT: Apparently removing the .Fullscreen doesn't work, however replacing the theme with one that does not have the .Fullscreen/.NoActionBar attributes. In this case android:theme="#style/ThemeOverlay.AppCompat.ActionBar" did the trick.
delete android:theme="#style/Theme" in the manifest file.
I am new to opencv. I want to apply sobel filter on real time video using opencv in android but i am failed in doing so. when i use the built-in function of sobel in opencv it gives me the white blank screen.I have searched a lot but didn't find my error..
package com.example.bismanawaz.opencvpractice;
import android.app.Activity;
import android.os.Bundle;
import android.util.Log;
import android.view.Menu;
import android.view.MenuItem;
import android.view.SurfaceView;
import android.view.WindowManager;
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.Mat;
import org.opencv.imgproc.Imgproc;
public class MainActivity extends Activity implements CameraBridgeViewBase.CvCameraViewListener2{
private CameraBridgeViewBase mOpenCvCameraView;
private Mat sobell;
public static final int CV_8UC1=0;
private static final String TAG ="OpenCV" ;
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
Log.i(TAG, "OpenCV loaded successfully");
mOpenCvCameraView.enableView();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
#Override
public void onResume()
{
super.onResume();
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_7, this, mLoaderCallback);
}
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Log.i(TAG, "called onCreate");
super.onCreate(savedInstanceState);
getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
setContentView(R.layout.activity_main);
mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.HelloOpenCvView);
mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
mOpenCvCameraView.setCvCameraViewListener(this);
}
#Override
public void onPause()
{
super.onPause();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
public void onDestroy() {
super.onDestroy();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
#Override
public boolean onCreateOptionsMenu(Menu menu) {
// Inflate the menu; this adds items to the action bar if it is present.
getMenuInflater().inflate(R.menu.menu_main, menu);
return true;
}
#Override
public boolean onOptionsItemSelected(MenuItem item) {
// Handle action bar item clicks here. The action bar will
// automatically handle clicks on the Home/Up button, so long
// as you specify a parent activity in AndroidManifest.xml.
int id = item.getItemId();
//noinspection SimplifiableIfStatement
if (id == R.id.action_settings) {
return true;
}
return super.onOptionsItemSelected(item);
}
/**
* This method is invoked when camera preview has started. After this method is invoked
* the frames will start to be delivered to client via the onCameraFrame() callback.
*
* #param width - the width of the frames that will be delivered
* #param height - the height of the frames that will be delivered
*/
#Override
public void onCameraViewStarted(int width, int height) {
sobell = new Mat();
}
/**
* This method is invoked when camera preview has been stopped for some reason.
* No frames will be delivered via onCameraFrame() callback after this method is called.
*/
#Override
public void onCameraViewStopped() {
}
/**
* This method is invoked when delivery of the frame needs to be done.
* The returned values - is a modified frame which needs to be displayed on the screen.
*
* #param inputFrame
*/
#Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
Imgproc.Sobel(inputFrame.gray(),sobell,CV_8UC1,1,1,3);
return sobell;
}
}
This is my method which is doing Sobel.
public static Mat FindSobelEdges(Mat mat) {
Imgproc.Sobel(mat, mat, CvType.CV_8UC1, 1, 1);
return mat;
}
I think your problem is wrong parameters. Pls check the Imgproc.Sobel doc:
Imgproc.Sobel
How can I use OpenCV to process process some images saved on a smartphone, without using JavaCameraView?
I want to process an image saved on the SD card and then show the result of the process on the screen. I implemented my according to the tutorials from opencv4android libraries and they use the method onCameraFrame to show the image and implement the CameraViewListener and use CameraBridgeViewBase. However, I only want to process an image, I don´t want to use the camera to capture images and I think those elements may be unnecessary.
How can I change the opencv4android libraries and process stored images using OpenCV without using the camera?
If someone is still looking for an answer:
package com.example.ocv4androidwithoutcamera;
import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.InstallCallbackInterface;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;
import org.opencv.core.Mat;
import android.os.Bundle;
import android.app.Activity;
import android.util.Log;
import android.widget.Toast;
public class MainActivity extends Activity implements LoaderCallbackInterface {
protected BaseLoaderCallback mOpenCVCallBack = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
onOpenCVReady();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
}
#Override
protected void onResume()
{
super.onResume();
Log.i("DEMO", "Trying to load OpenCV library");
if (!OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_4, this, mOpenCVCallBack))
{
Log.e("DEMO", "Cannot connect to OpenCV Manager");
}
}
protected void onOpenCVReady(){
//this should crash if opencv is not loaded
Mat img = new Mat();
Toast.makeText(getApplicationContext(), "opencv ready", Toast.LENGTH_LONG).show();
}
#Override
public void onManagerConnected(int status) {
// TODO Auto-generated method stub
}
#Override
public void onPackageInstall(int operation,
InstallCallbackInterface callback) {
// TODO Auto-generated method stub
}
}
Don’t forget to add opencv library in Project Properties => Android => Library.