I am trying to use OpenCV 2.4.3.2 to create a camera app and do some opencv processing. I would like it to be able to have multiple UI orientations, not just Landscape.
The problem is that when I change the orientation to portrait, the image comes out sideways.
I understand that I could just rotate the input image before doing image processing (and thus leave the orientation as landscape only), which is fine and works, but doesn't solve the problem that the rest of my UI will be in the wrong orientation.
I have also tried using this code to rotate the camera 90deg, but it just doesn't seem to work.
mCamera.setDisplayOrientation(90);
It either has no effect, or sometimes just causes the preview to be blacked out
Has anyone done this successfully with OpenCV? My class extends from JavaCameraView.
Edit
I have made an improvement, which is that I have rotated the image inside of OpenCV as it is displayed in the CameraBridgeViewBase.java class.
In the deliver and draw frame method:
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
//canvas.drawBitmap(mCacheBitmap, (canvas.getWidth() - mCacheBitmap.getWidth()) / 2, (canvas.getHeight() - mCacheBitmap.getHeight()) / 2, null);
//Change to support portrait view
Matrix matrix = new Matrix();
matrix.preTranslate((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,(canvas.getHeight() - mCacheBitmap.getHeight()) / 2);
if(getResources().getConfiguration().orientation == Configuration.ORIENTATION_PORTRAIT)
matrix.postRotate(90f,(canvas.getWidth()) / 2,(canvas.getHeight()) / 2);
canvas.drawBitmap(mCacheBitmap, matrix, new Paint());
...
Basically, this just roatates the input image like so
This is better, but I obviously want this to be full screen.
I had the same problem trying to implement OpenCV. I was able to fix it by making the following changes to the deliverAndDrawFrame method.
Rotate the canvas object
Canvas canvas = getHolder().lockCanvas();
// Rotate canvas to 90 degrees
canvas.rotate(90f, canvas.getWidth()/2, canvas.getHeight()/2);
Resize the bitmap to fit entire size of canvas before drawing
// Resize
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
// Use bitmap instead of mCacheBitmap
canvas.drawBitmap(bitmap, new Rect(0,0,bitmap.getWidth(), bitmap.getHeight()), new Rect(
(int)((canvas.getWidth() - mScale*bitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*bitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*bitmap.getWidth()) / 2 + mScale*bitmap.getWidth()),
(int）((canvas.getHeight() - mScale*bitmap.getHeight()) / 2 + mScale*bitmap.getHeight()
)), null);
// Unlock canvas
getHolder().unlockCanvasAndPost(canvas);
I modified the CameraBridgeViewBase.java as follows:
protected Size calculateCameraFrameSize(List<?> supportedSizes, ListItemAccessor accessor, int surfaceWidth, int surfaceHeight) {
int calcWidth = 0;
int calcHeight = 0;
if(surfaceHeight > surfaceWidth){
int temp = surfaceHeight;
surfaceHeight = surfaceWidth;
surfaceWidth = temp;
}
And in the function "deliverAndDrawFrame":
if (mScale != 0) {
if(canvas.getWidth() > canvas.getHeight()) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, rotateMe(canvas, mCacheBitmap), null);
}
where rotateMe is defined as follows:
private Matrix rotateMe(Canvas canvas, Bitmap bm) {
// TODO Auto-generated method stub
Matrix mtx=new Matrix();
float scale = (float) canvas.getWidth() / (float) bm.getHeight();
mtx.preTranslate((canvas.getWidth() - bm.getWidth())/2, (canvas.getHeight() - bm.getHeight())/2);
mtx.postRotate(90,canvas.getWidth()/2, canvas.getHeight()/2);
mtx.postScale(scale, scale, canvas.getWidth()/2 , canvas.getHeight()/2 );
return mtx;
}
The preview FPS is slower because for computational overhead when compared to landscape mode.
Actually, you can just make width or height match parent (full screen).
if (canvas != null) {
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
canvas.rotate(90,0,0);
float scale = canvas.getWidth() / (float)bitmap.getHeight();
float scale2 = canvas.getHeight() / (float)bitmap.getWidth();
if(scale2 > scale){
scale = scale2;
}
if (scale != 0) {
canvas.scale(scale, scale,0,0);
}
canvas.drawBitmap(bitmap, 0, -bitmap.getHeight(), null);
...
Also, you can make the preview size larger than the screen. Just modify the scale.
Unfortunately Opencv4Android does not support portrait camera. But there's a way how to overcome it.
1)Write your custom Camera and set it's orientation to portrait.
2)Register for it's preview callback.
3)In onPreviewFrame(byte[]data, Camera camera) create Mat of preview bytes:
Mat mat = new Mat(previewSize.height, previewSize.width, CvType.CV_8UC1);
mat.put(0, 0, data);
Core.transpose(mat, mat);
Core.flip(mat, mat, -1); // rotates Mat to portrait
CvType depends on a preview format your camera is using.
PS. do not forget to release all the Mat instances you've created when you're done.
PPS. it's good to manage your camera on a separate thread in order not to overload UI thread while doing some detection.
As in other answers, I've written my personal version of deliverAndDrawFrame (I've also notified through comments where my code starts and ends):
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
if (BuildConfig.DEBUG) {
Log.d(TAG, "mStretch value: " + mScale);
}
// Start of the fix
Matrix matrix = new Matrix();
matrix.preTranslate( ( canvas.getWidth() - mCacheBitmap.getWidth() ) / 2f, ( canvas.getHeight() - mCacheBitmap.getHeight() ) / 2f );
matrix.postRotate( 90f, ( canvas.getWidth()) / 2f, canvas.getHeight() / 2f );
float scale = (float) canvas.getWidth() / (float) mCacheBitmap.getHeight();
matrix.postScale(scale, scale, canvas.getWidth() / 2f , canvas.getHeight() / 2f );
canvas.drawBitmap( mCacheBitmap, matrix, null );
// Back to original OpenCV code
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
getHolder().unlockCanvasAndPost(canvas);
}
}
}
Preview is now in Portrait mode, as you can see:
I have the same issue, i have had figure out it!! and there is my solution:
as the part of first, In CameraBridgeViewBase.Java, the two constructor, add initialization of WindowManager:
public CameraBridgeViewBase(Context context, int cameraId) {
super(context);
mCameraIndex = cameraId;
getHolder().addCallback(this);
mMaxWidth = MAX_UNSPECIFIED;
mMaxHeight = MAX_UNSPECIFIED;
windowManager = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
}
public CameraBridgeViewBase(Context context, AttributeSet attrs) {
super(context, attrs);
int count = attrs.getAttributeCount();
Log.d(TAG, "Attr count: " + Integer.valueOf(count));
TypedArray styledAttrs = getContext().obtainStyledAttributes(attrs, R.styleable.CameraBridgeViewBase);
if (styledAttrs.getBoolean(R.styleable.CameraBridgeViewBase_show_fps, false))
enableFpsMeter();
mCameraIndex = styledAttrs.getInt(R.styleable.CameraBridgeViewBase_camera_id, -1);
getHolder().addCallback(this);
mMaxWidth = MAX_UNSPECIFIED;
mMaxHeight = MAX_UNSPECIFIED;
windowManager = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
styledAttrs.recycle();
}
then ,You need to replace function deliverAndDrawFrame(CvCameraViewFrame frame) as follows ,
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch (Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
int rotation = windowManager.getDefaultDisplay().getRotation();
int degrees = 0;
// config degrees as you need
switch (rotation) {
case Surface.ROTATION_0:
degrees = 90;
break;
case Surface.ROTATION_90:
degrees = 0;
break;
case Surface.ROTATION_180:
degrees = 270;
break;
case Surface.ROTATION_270:
degrees = 180;
break;
}
Matrix matrix = new Matrix();
matrix.postRotate(degrees);
Bitmap outputBitmap = Bitmap.createBitmap(mCacheBitmap, 0, 0, mCacheBitmap.getWidth(), mCacheBitmap.getHeight(), matrix, true);
if (outputBitmap.getWidth() <= canvas.getWidth()) {
mScale = getRatio(outputBitmap.getWidth(), outputBitmap.getHeight(), canvas.getWidth(), canvas.getHeight());
} else {
mScale = getRatio(canvas.getWidth(), canvas.getHeight(), outputBitmap.getWidth(), outputBitmap.getHeight());
}
if (mScale != 0) {
canvas.scale(mScale, mScale, 0, 0);
}
Log.d(TAG, "mStretch value: " + mScale);
canvas.drawBitmap(outputBitmap, 0, 0, null);
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
getHolder().unlockCanvasAndPost(canvas);
}
}
}
and add this function extra,
private float getRatio(int widthSource, int heightSource, int widthTarget, int heightTarget) {
if (widthTarget <= heightTarget) {
return (float) heightTarget / (float) heightSource;
} else {
return (float) widthTarget / (float) widthSource;
}
}
it's okey ,and If this answer useful to you, please mark 'accepted' Help Reputation
All answers here are hacks. i prefer this solution :
change in JavaCameraView code :
mBuffer = new byte[size];
mCamera.setDisplayOrientation(90); //add this
mCamera.addCallbackBuffer(mBuffer);
Second Change :
// if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
// mSurfaceTexture = new SurfaceTexture(MAGIC_TEXTURE_ID);
// mCamera.setPreviewTexture(mSurfaceTexture);
// } else
// mCamera.setPreviewDisplay(null);
mCamera.setPreviewDisplay(getHolder());
It seems like the new OpenCV CameraBridgeViewBase.java class is too high-level and doesn't give enough control over the layout of the camera preview. Take a look at my sample code, which is based on some of the older OpenCV samples and uses pure Android code. To use the byte array passed in onPreviewFrame, put() it into a Mat and convert from YUV to RGB:
mYuv = new Mat(previewHeight + previewHeight/2, previewWidth, CvType.CV_8UC1);
mYuv.put(0, 0, mBuffer);
Imgproc.cvtColor(mYuv, mRgba, Imgproc.COLOR_YUV420sp2RGBA, 4);
You may be able to find the old OpenCV4Android samples on the internet, though they were taken out a few versions ago. However, the linked sample code and the snippet above should be enough to get you started.
If you're using the openCV 2.4.9 , try :
1) to copy the content of opencv tutorial-mixed processing in your code;
2) correct the mismatching errors (activity name and probably layout reference);
3)Modify your manifest by adding android:screenOrientation ="landscape"
4) correct minors errors and run !!!! bbaamm (it should work properly now)
Note: with this method the status bar appaear on the right side when the phone is in portrait position .
Since we are developing camera project, I advise you to remove the status bar from the preview.
Hope it helps !!!
You have to consider a few things:
onPreviewFrame() always delivers the raw camera data in its assambled rotation
getSupportedPreviewSizes() gives corresponding aspect ratios
Algorithm needs to analyze the frame in portrait to detect objects correct.
the Bitmap created (Java-side) to store the resulting frame also needs the correct aspect ratio
So, for a fast and high-resolution solution i changed JavaCameraView.java and my JNI-part.
in JavaCameraView.java:
...
if (sizes != null) {
/* Select the size that fits surface considering maximum size allowed */
Size frameSize;
if(width > height)
{
frameSize = calculateCameraFrameSize(sizes, new JavaCameraSizeAccessor(), width, height);
}else{
frameSize = calculateCameraFrameSize(sizes, new JavaCameraSizeAccessor(), height, width);
}
...
mCamera.setParameters(params);
params = mCamera.getParameters();
int bufFrameWidth, bufFrameHeight;
bufFrameWidth = params.getPreviewSize().width;
bufFrameHeight = params.getPreviewSize().height;
if(width > height) {
mFrameWidth = params.getPreviewSize().width;
mFrameHeight = params.getPreviewSize().height;
}else{
mFrameWidth = params.getPreviewSize().height;
mFrameHeight = params.getPreviewSize().width;
}
...
mFrameChain = new Mat[2];
mFrameChain[0] = new Mat(bufFrameHeight + (bufFrameHeight/2), bufFrameWidth, CvType.CV_8UC1);
mFrameChain[1] = new Mat(bufFrameHeight + (bufFrameHeight/2), bufFrameWidth, CvType.CV_8UC1);
AllocateCache();
mCameraFrame = new JavaCameraFrame[2];
mCameraFrame[0] = new JavaCameraFrame(mFrameChain[0], bufFrameWidth, bufFrameHeight);
mCameraFrame[1] = new JavaCameraFrame(mFrameChain[1], bufFrameWidth, bufFrameHeight);
With these changes, we made sure we are using the highest resultion available for portrait (switches height/width in calculateCameraFrameSize). We are still handling landscape as input from onPreviewFrame() but created a Bitmap to draw in portrait (AllocateCache).
Last, we need to give the algorithm the portrait-frame in order to let him detect "standing" objects and return it for saving and rendering the bitmap.
So following modifications to your Activity:
public Mat rot90(Mat matImage, int rotflag){
//1=CW, 2=CCW, 3=180
Mat rotated = new Mat();
if (rotflag == 1){
rotated = matImage.t();
flip(rotated, rotated, 1); //transpose+flip(1)=CW
} else if (rotflag == 2) {
rotated = matImage.t();
flip(rotated, rotated,0); //transpose+flip(0)=CCW
} else if (rotflag ==3){
flip(matImage, rotated,-1); //flip(-1)=180
} else if (rotflag != 0){ //if not 0,1,2,3:
Log.e(TAG, "Unknown rotation flag("+rotflag+")");
}
return rotated;
}
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
mRgba = rot90(inputFrame.rgba(), 1);
mGray = rot90(inputFrame.gray(), 1);
...
After a lot of searching around, I found this -
https://answers.opencv.org/question/23972/face-detect-with-portrait-mode-and-front-camera/
This works.
Also, make sure you set the portrait mode in AndroidManifest.xml
android:screenOrientation="portrait"
I`ve got portrait orientation with CameraBridgeViewBase, but I had to change JavaCameraView.java inside the OpenCV :( The idea is next: after camera init, do next
setDisplayOrientation(mCamera, 90);
mCamera.setPreviewDisplay(getHolder());
and setDisplayOrientation method
protected void setDisplayOrientation(Camera camera, int angle){
Method downPolymorphic;
try
{
downPolymorphic = camera.getClass().getMethod("setDisplayOrientation", new Class[] { int.class });
if (downPolymorphic != null)
downPolymorphic.invoke(camera, new Object[] { angle });
}
catch (Exception e1)
{
}
}
"jaiprakashgogi" developer answer is working for me. but the problem is the preview still saved as landscape only. that means if we set the preview to imageview then it displayed as landscape.
The above solution works upto showing the preview as portrait but not saved as portrait persistently.
I was resolved that issue as the following way.
convert the byte or mat data into bitmap
rotate the matrix to 90 degrees and apply to bitmap
convert bitmap to byte array and save it.
please see the my code here...
public String writeToSDFile(byte[] data, int rotation){
byte[] portraitData=null;
if(rotation==90){
Log.i(TAG,"Rotation is : "+rotation);
Bitmap bitmap= BitmapFactory.decodeByteArray(data,0,data.length);
Matrix matrix = new Matrix();
matrix.postRotate(90);
Bitmap rotatedBitmap = Bitmap.createBitmap(bitmap , 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
portraitData=bitmapToByte(rotatedBitmap);
}
File dir=getDirectory();
String imageTime=""+System.currentTimeMillis();
String fileName=Constants.FILE_NAME+imageTime+"."+Constants.IMAGE_FORMAT;
File file = new File(dir, fileName);
try {
FileOutputStream f = new FileOutputStream(file);
if(rotation==90){
f.write(portraitData);
}else {
f.write(data);
}
f.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
Log.i(TAG, "******* File not found. Did you" +
" add a WRITE_EXTERNAL_STORAGE permission to the manifest?");
} catch (IOException e) {
e.printStackTrace();
}
Log.i(TAG,"\n\nFile written to "+file);
return fileName;
}
// convert bitmap to Byte Array
public byte[] bitmapToByte(Bitmap bitmap){
ByteArrayOutputStream outputStream=new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG,100,outputStream);
byte[] array=outputStream.toByteArray();
return array;
}
It solves the my problem completely.
Thanks to #Kaye Wrobleski for his answer.
I have extended it to allow both landscape and portrait orientation.
This is basically just a little extra code to allow easily switching between the default code that gives landscape orientation, and his code for portrait.
Insert his code as a new method in CameraBridgeViewBase.java
protected void deliverAndDrawFramePortrait(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
// Rotate canvas to 90 degrees
canvas.rotate(90f, canvas.getWidth()/2, canvas.getHeight()/2);
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
Log.d(TAG, "mStretch value: " + mScale);
if (mScale != 0) {
// Resize
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
// Use bitmap instead of mCacheBitmap
canvas.drawBitmap(bitmap, new Rect(0,0,bitmap.getWidth(), bitmap.getHeight()), new Rect(
(int)((canvas.getWidth() - mScale*bitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*bitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*bitmap.getWidth()) / 2 + mScale*bitmap.getWidth()),
(int)((canvas.getHeight() - mScale*bitmap.getHeight()) / 2 + mScale*bitmap.getHeight())), null);
} else {
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
// Use bitmap instead of mCacheBitmap
canvas.drawBitmap(bitmap, new Rect(0,0,bitmap.getWidth(), bitmap.getHeight()), new Rect(
(int)((canvas.getWidth() - bitmap.getWidth()) / 2),
(int)((canvas.getHeight() - bitmap.getHeight()) / 2),
(int)((canvas.getWidth() - bitmap.getWidth()) / 2 + bitmap.getWidth()),
(int)((canvas.getHeight() - bitmap.getHeight()) / 2 + bitmap.getHeight())), null);
}
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
getHolder().unlockCanvasAndPost(canvas);
}
}
}
Then modify JavaCameraView.java
Add a new variable to track whether we are in portrait or landscape mode
private boolean portraitMode;
Add two methods to set the orientation mode
public void setLandscapeMode() {
portraitMode = false;
}
public void setPortraitMode() {
portraitMode = true;
}
Now replace these lines in the JavaCameraView CameraWorkerClass, run() method
if (!mFrameChain[1 - mChainIdx].empty())
deliverAndDrawFrame(mCameraFrame[1 - mChainIdx]);
With these lines:
if (!mFrameChain[1 - mChainIdx].empty()) {
if (!portraitMode) {
deliverAndDrawFrame(mCameraFrame[1 - mChainIdx]);
} else {
deliverAndDrawFramePortrait(mCameraFrame[1 - mChainIdx]);
}
}
To switch between orientations, simply call either setLandscapeMode() or setPortraitMode() on your JavaCameraView object.
Please note that reverse portrait and reverse landscape orientations will still be upside-down.
You will need to rotate them 180 degrees to get them right-side up, which is easily done with OpenCV's warpAffine() method.
Note when using the back camera (LENS_FACING_BACK), portrait mode will flip the images upside down.
Maybe this help anyone. tested on android 9 with opencv343. Now this full screen and DETECT face in portrait and landscape modes. small changes in CameraBridgeViewBase class:
private final Matrix matrix = new Matrix();
......
change deliverAndDrawFrame() method:
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
int currentOrientation = getResources().getConfiguration().orientation;
if (currentOrientation == Configuration.ORIENTATION_LANDSCAPE) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
if (BuildConfig.DEBUG)
Log.d(TAG, "mStretch value: " + mScale);
if (mScale != 0) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
(canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
}
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
getHolder().unlockCanvasAndPost(canvas);
}
} else {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
int saveCount = canvas.save();
canvas.setMatrix(matrix);
mScale = Math.max((float) canvas.getHeight() / mCacheBitmap.getWidth(), (float) canvas.getWidth() / mCacheBitmap.getHeight());
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
if (mScale != 0) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mCacheBitmap.getWidth()) - mCacheBitmap.getWidth())/2,
(int)(canvas.getHeight() - mScale*mCacheBitmap.getHeight() - mScale*mCacheBitmap.getHeight()/2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, new Rect(0, 0, mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
(canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
}
canvas.restoreToCount(saveCount);
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
getHolder().unlockCanvasAndPost(canvas);
}
}
}
}
and in MainActivity:
public Mat rotateMat(Mat matImage) {
Mat rotated = matImage.t();
Core.flip(rotated, rotated, 1);
return rotated;
}
#Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
MatOfRect faces = new MatOfRect();
int currentOrientation = getResources().getConfiguration().orientation;
if (currentOrientation == Configuration.ORIENTATION_LANDSCAPE) {
mRgba = inputFrame.rgba();
mGray = inputFrame.gray();
int height = mGray.rows();
if (Math.round(height * 0.2) > 0) {
mFaceSize = (int) Math.round(height * 0.2);
}
cascadeClassifier.detectMultiScale(mGray, faces, 1.1, 3, 2,
new Size(mFaceSize, mFaceSize));
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++) {
rectangle(mRgba, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 3);
}
} else {
mRgba = inputFrame.rgba();
mGray = rotateMat(inputFrame.gray());
if (mFaceSize == 0) {
int height = mGray.cols();
if (Math.round(height * 0.2) > 0) {
mFaceSize = (int) Math.round(height * 0.2);
}
}
Mat newMat = rotateMat(mRgba);
if(!isBackCameraOn){
flip(newMat, newMat, -1);
flip(mGray, mGray, -1);
}
if (cascadeClassifier != null)
cascadeClassifier.detectMultiScale(mGray, faces, 1.1, 3, 2, new Size(mFaceSize, mFaceSize));
mGray.release();
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++) {
rectangle(newMat, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 3);
}
Imgproc.resize(newMat, mRgba, new Size(mRgba.width(), mRgba.height()));
newMat.release();
}
if(!isBackCameraOn){
flip(mRgba, mRgba, 1);
flip(mGray, mGray, 1);
}
return mRgba;
}
Another solution. i think this better
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
if (BuildConfig.DEBUG)
Log.d(TAG, "mStretch value: " + mScale);
int currentOrientation = getResources().getConfiguration().orientation;
if (currentOrientation == Configuration.ORIENTATION_LANDSCAPE) {
if (mScale != 0) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
(canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
}
} else {
if (mScale != 0) {
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
canvas.drawBitmap(bitmap, new Rect(0,0,bitmap.getWidth(), bitmap.getHeight()), new Rect(
(int)((canvas.getWidth() - mScale*bitmap.getWidth()) / 2),
(int)(0),
(int)((canvas.getWidth() - mScale*bitmap.getWidth()) / 2 + mScale*bitmap.getWidth()),
(int)((canvas.getHeight()))), null);
} else {
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
canvas.drawBitmap(bitmap, new Rect(0,0,bitmap.getWidth(), bitmap.getHeight()), new Rect(
(int)((canvas.getWidth() - bitmap.getWidth()) / 2),
(int)(0),
(int)((canvas.getWidth() - bitmap.getWidth()) / 2 + bitmap.getWidth()),
(int)(canvas.getHeight())), null);
}
}
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
getHolder().unlockCanvasAndPost(canvas);
}
}
}
and...
#Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
MatOfRect faces = new MatOfRect();
int currentOrientation = getResources().getConfiguration().orientation;
if (currentOrientation == Configuration.ORIENTATION_LANDSCAPE) {
mRgba = inputFrame.rgba();
mGray = inputFrame.gray();
int height = mGray.rows();
if (Math.round(height * 0.2) > 0) {
mFaceSize = (int) Math.round(height * 0.2);
}
cascadeClassifier.detectMultiScale(mGray, faces, 1.1, 3, 2,
new Size(mFaceSize, mFaceSize));
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++) {
Point center = new Point(facesArray[i].x + facesArray[i].width / 2,
facesArray[i].y + facesArray[i].height / 2);
rectangle(mRgba, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 3);
}
} else {
mRgba = inputFrame.rgba();
mGray = inputFrame.gray();
Mat rotImage = Imgproc.getRotationMatrix2D(new Point(mRgba.cols() / 2,
mRgba.rows() / 2), 90, 1.0);
Imgproc.warpAffine(mRgba, mRgba, rotImage, mRgba.size());
Imgproc.warpAffine(mGray, mGray, rotImage, mRgba.size());
Core.flip(mRgba, mRgba, 1);
Core.flip(mGray, mGray, 1);
int height = mGray.rows();
if (Math.round(height * 0.2) > 0) {
mFaceSize = (int) Math.round(height * 0.2);
}
cascadeClassifier.detectMultiScale(mGray, faces, 1.1, 3, 2,
new Size(mFaceSize, mFaceSize));
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++) {
Point center = new Point(facesArray[i].x + facesArray[i].width / 2,
facesArray[i].y + facesArray[i].height / 2);
rectangle(mRgba, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 3);
}
}
return mRgba;
I don't think there is a way to do this, without some pixel manipulation. However, if we simply modified the matrix into which all of those pixels get drawn. The answer lies, in part, in the CameraBridgeViewBase.java file
1. Go to the CameraBridgeViewBase Class
2. Make Function update matrix
private final Matrix mMatrix = new Matrix();
private void updateMatrix() {
float mw = this.getWidth();
float mh = this.getHeight();
float hw = this.getWidth() / 2.0f;
float hh = this.getHeight() / 2.0f;
float cw = (float)Resources.getSystem().getDisplayMetrics().widthPixels; //Make sure to import Resources package
float ch = (float)Resources.getSystem().getDisplayMetrics().heightPixels;
float scale = cw / (float)mh;
float scale2 = ch / (float)mw;
if(scale2 > scale){
scale = scale2;
}
boolean isFrontCamera = mCameraIndex == CAMERA_ID_FRONT;
mMatrix.reset();
if (isFrontCamera) {
mMatrix.preScale(-1, 1, hw, hh); //MH - this will mirror the camera
}
mMatrix.preTranslate(hw, hh);
if (isFrontCamera){
mMatrix.preRotate(270);
} else {
mMatrix.preRotate(90);
}
mMatrix.preTranslate(-hw, -hh);
mMatrix.preScale(scale,scale,hw,hh);
}
3. Override onMeasure and layout function
#Override
public void layout(int l, int t, int r, int b) {
super.layout(l, t, r, b);
updateMatrix();
}
#Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
super.onMeasure(widthMeasureSpec, heightMeasureSpec);
updateMatrix();
}
4. Replace existing deliverAndDrawFrame function
protected void deliverAndDrawFrame(CvCameraViewFrame frame) { //replaces existing deliverAndDrawFrame
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
int saveCount = canvas.save();
canvas.setMatrix(mMatrix);
if (mScale != 0) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
(canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
}
//Restore canvas after draw bitmap
canvas.restoreToCount(saveCount);
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
getHolder().unlockCanvasAndPost(canvas);
}
}
}
I was able to extend from JavaCameraView and hack around to allow portrait and other orientations. This is what worked for me in OpenCV 4.5.3 (com.quickbirdstudios:opencv-contrib:4.5.3.0):
package com.reactnativemapexplorer.opencv;
import android.content.Context;
import android.graphics.Bitmap;
import org.opencv.android.CameraBridgeViewBase;
import org.opencv.android.JavaCameraView;
import org.opencv.core.Core;
import org.opencv.core.Mat;
import java.lang.reflect.Field;
public class CustomJavaCameraView extends JavaCameraView {
public enum Orientation {
LANDSCAPE_LEFT,
PORTRAIT,
LANDSCAPE_RIGHT;
boolean isLandscape() {
return this == LANDSCAPE_LEFT || this == LANDSCAPE_RIGHT;
}
boolean isLandscapeRight() {
return this == LANDSCAPE_RIGHT;
}
};
// scale camera by this coefficient - using mScale seems to more performant than upsizing frame Mat
// orientation is immutable because every attempt to change it dynamically failed with 'null pointer dereference' and similar exceptions
// tip: re-creating camera from the outside should allow changing orientation
private final Orientation orientation;
public CustomJavaCameraView(Context context, int cameraId, Orientation orientation) {
super(context, cameraId);
this.orientation = orientation;
}
#Override
protected void AllocateCache() {
if (orientation.isLandscape()) {
super.AllocateCache();
return;
}
try {
Field privateField = CameraBridgeViewBase.class.getDeclaredField("mCacheBitmap");
privateField.setAccessible(true);
privateField.set(this, Bitmap.createBitmap(mFrameHeight, mFrameWidth, Bitmap.Config.ARGB_8888));
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new RuntimeException(e);
}
}
private class CvCameraViewFrameImplHack implements CvCameraViewFrame {
private Mat rgbMat;
public CvCameraViewFrameImplHack(Mat rgbMat) {
this.rgbMat = rgbMat;
}
#Override
public Mat rgba() {
return this.rgbMat;
}
#Override
public Mat gray() {
return null;
}
}
private Mat rotateToPortrait(Mat mat) {
Mat transposed = mat.t();
Mat flipped = new Mat();
Core.flip(transposed, flipped, 1);
transposed.release();
return flipped;
}
private Mat rotateToLandscapeRight(Mat mat) {
Mat flipped = new Mat();
Core.flip(mat, flipped, -1);
return flipped;
}
#Override
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat frameMat = frame.rgba();
Mat rotated;
if (orientation.isLandscape()) {
if (orientation.isLandscapeRight()) {
rotated = rotateToLandscapeRight(frameMat);
} else {
rotated = frameMat;
}
mScale = (float)getWidth() / (float)frameMat.width();
} else {
rotated = rotateToPortrait(frameMat);
mScale = (float)getHeight() / (float)rotated.height();
}
CvCameraViewFrameImplHack hackFrame = new CvCameraViewFrameImplHack(rotated);
super.deliverAndDrawFrame(hackFrame);
}
}
I don't know well, but camera size is decided from screen width. Because screen width is low, camera height also is decided by low in the portrait orientation. Therefore, camera resolution is decided by low too. And preview image lay down (preview image rotation is decided as camera image's width and height in CameraBridgeViewBase.java).
As the solution, use landscape orientation (decide landscape mode in the manifest.xml as Activity). As a result, because screen width is high, height will be high also and your app decide high resolution. Also, you don't have to rotate the camera image and always full-screen mode. But the disadvantage is the origin point is different. I tried variety method about high-resolution image as the portrait orientation but I couldn't find a way.
My app : portrait orientation
my camera image is 720, 480 / landscape orientation 1280, 1080.
Modify your code in JavaCameraView.java as outlined on this page
It is really easy to fix.
Before
Log.d(TAG, "startPreview");
mCamera.startPreview();
After
Log.d(TAG, "startPreview");
setDisplayOrientation(mCamera, 90);
mCamera.setPreviewDisplay(getHolder());
mCamera.startPreview();
Related
I am using Google mobile vision text API for reading a text using the APIs provided.
My use cases :
Initially i am drawing a re-sizable rectangle to the surface of view.
Restrict the text that recognize by the google apis only with in the rectangle.
public class GraphicOverlay<T extends GraphicOverlay.Graphic> extends View
{
protected void onDraw(Canvas canvas)
{
super.onDraw(canvas);
synchronized (mLock)
{
if ((mPreviewWidth != 0) && (mPreviewHeight != 0))
{
mWidthScaleFactor = (float) canvas.getWidth() / (float) mPreviewWidth;
mHeightScaleFactor = (float) canvas.getHeight() / (float) mPreviewHeight;
}
for (Graphic graphic : mGraphics)
{
graphic.draw(canvas);
}
paint.setColor(Color.parseColor("#55000000"));
paint.setStyle(Paint.Style.FILL);
paint.setStrokeJoin(Paint.Join.ROUND);
// mPaint.setStrokeCap(Paint.Cap.ROUND);
paint.setStrokeWidth(5);
canvas.drawPaint(paint);
paint.setColor(Color.parseColor("#55FFFFFF"));
if (groupId == 1) {
mCurrentRect= new Rect(point2.x+colorballs.get(1).getWidthOfBall() / 2,
point4.y +colorballs.get(1).getWidthOfBall() / 2,
point4.x+colorballs.get(1).getWidthOfBall() / 2,
point2.y+colorballs.get(1).getWidthOfBall() / 2);
canvas.drawRect(mCurrentRect, paint);
} else {
mCurrentRect= new Rect(point2.x + colorballs.get(1).getWidthOfBall() / 2,
point4.y + colorballs.get(3).getWidthOfBall() / 2, point4.x
+ colorballs.get(3).getWidthOfBall() / 2, point2.y
+ colorballs.get(1).getWidthOfBall() / 2);
canvas.drawRect(mCurrentRect, paint);
}
BitmapDrawable mBitmap;
mBitmap = new BitmapDrawable();
// draw the balls on the canvas`enter code here`
for (ColorBall ball : colorballs) {
canvas.drawBitmap(ball.getBitmap(), ball.getX(), ball.getY(),
new Paint());
}
}
}
}
class CustomTextRecognizer extends Detector<TextBlock> {
private Detector<TextBlock> mDelegate;
private GraphicOverlay<OcrGraphic> mOcrGraphicOverlay;
CustomTextRecognizer(Detector<TextBlock> delegate, GraphicOverlay<OcrGraphic> ocrGraphicOverlay) {
mDelegate = delegate;
mOcrGraphicOverlay= ocrGraphicOverlay;
}
#Override
public void receiveFrame(Frame frame) {
Bitmap bt= frame.getBitmap();
super.receiveFrame(frame);
}
public SparseArray<TextBlock> detect(Frame frame) {
//How to compare the items that is inside my rectangle.
}
}
/*
* This method will call before each block detection and resolves only with valid detection that
* scanned with in the rectangle provided.
*/
public SparseArray<TextBlock> detect(Frame frame) {
SparseArray<TextBlock> validDetections = new SparseArray<TextBlock>();
Rect overlayRectangle = mOcrGraphicOverlay.getCurrentRectangle();
SparseArray<TextBlock> detectedItems = mDelegate.detect(frame);
for (int index = 0; index < detectedItems.size(); ++index) {
TextBlock detectedBlock = detectedItems.valueAt(index);
Rect boundingBox = detectedBlock.getBoundingBox();
// mOcrGraphicOverlay.getColorball(4).setX(overlayRectangle.left);
// mOcrGraphicOverlay.getColorball(4).setY(overlayRectangle.top);
/* if(((overlayRectangle.top <boundingBox.top) &&
(overlayRectangle.top+overlayRectangle.height())> (boundingBox.top +boundingBox.height())) &&
((overlayRectangle.left <boundingBox.left) &&
(overlayRectangle.left+overlayRectangle.width())> (boundingBox.left +boundingBox.width())))*/
mOcrGraphicOverlay.getColorball(4).setX(boundingBox.left);
mOcrGraphicOverlay.getColorball(4).setY(boundingBox.top);
if ((overlayRectangle.top < boundingBox.top) &&
(overlayRectangle.top + overlayRectangle.height()) > (boundingBox.top + boundingBox.height())) {
validDetections.put(index, detectedBlock);
}
}
return validDetections;
}
Found one solution.
Hello Android Programmers here, kindly check my code to fill the parameters about cropping the image detected.
I don't know what should be the values to put in parameters about cropping the face detected in android using createBitmap(Bitmapsource,x,y,width,height).
How can i get some integer for the x and y to fill the parameters in Bitmap function.
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
mRgba = inputFrame.rgba();
mGray = inputFrame.gray();
if (mAbsoluteFaceSize == 0) {
int height = mGray.rows();
if (Math.round(height * mRelativeFaceSize) > 0) {
mAbsoluteFaceSize = Math.round(height * mRelativeFaceSize);
}
mNativeDetector.setMinFaceSize(mAbsoluteFaceSize);
}
MatOfRect faces = new MatOfRect();
if (mDetectorType == JAVA_DETECTOR) {
if (mJavaDetector != null)
mJavaDetector.detectMultiScale(mGray, faces, 1.1, 2, 2, // TODO: objdetect.CV_HAAR_SCALE_IMAGE
new Size(mAbsoluteFaceSize, mAbsoluteFaceSize), new Size());
}
else if (mDetectorType == NATIVE_DETECTOR) {
if (mNativeDetector != null)
mNativeDetector.detect(mGray, faces);
}
else {
Log.e(TAG, "Detection method is not selected!");
}
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++)
Core.rectangle(mRgba, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 3);
//crop
//crop
viewWidth = part1.getMeasuredWidth();
viewHeight = part1.getMeasuredHeight();
//create the bitmap
bmp = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);
Resizebmp = Bitmap.createBitmap(bmp, x, y, viewWidth, viewHeight);
try {
Utils.matToBitmap(mRgba, bmp);
} catch(Exception e) {
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmp.recycle();
bmp = null;
}
return mRgba;
}
Correct me if I'm wrong. Do you mean that you had detected the face in a large bitmap and you want to crop the face out to a smaller bitmap?
For example, you have a 640 x 480 bitmap and you found the face inside rectangular with upper left x1, y1 and lower right x2, y2. Then it should be something like:
resizeBmp = Bitmap.createBitmap(bmp, x1, y1, x2 - x1, y2 - y1);
I am trying to use OpenCV face detection using the the byte[] data obtained from the onPreviewFrame() method of the Camera.PreviewCallback
I manage to convert the data into grayscale image using the codes below.
Mat matNew = new Mat(pHeight, pWidth, CvType.CV_8U);
matNew.put(0, 0, data);
Mat matrgb = new Mat();
Imgproc.cvtColor(matNew, matrgb, Imgproc.COLOR_YUV420sp2RGB, 4);
Mat matgray = new Mat();
Imgproc.cvtColor(matrgb, matgray, Imgproc.COLOR_RGB2GRAY, 0);
and I have set android:screenOrientation to "portrait" in the AndroidManifest file.
I am using OpenCV JavaDetector
mJavaDetector.detectMultiScale(matgray, faceDetected, 1.1, 3, 0,
new org.opencv.core.Size(0,0), new org.opencv.core.Size(matgray.width(), matgray.height()));
and drawing a rectangle over the faces detected using this
for (Rect rect : faceDetected.toArray()){
Core.rectangle(matgray, new Point(rect.x, rect.y),
new Point(rect.x + rect.width, rect.y + rect.height),
new Scalar(0, 255, 0));
}
However, in the resulting grayscale mat, face detection only happens when I hold my Android phone in landscape position. It does not work in portrait position.
The transpose and flip suggested in this post How to detect face by portrait mode? doesn't seem to work.
Is there any way to overcome this issue? I have used the Android FaceDetectionListener and that doesn't seem to have problem detecting faces in portrait mode. But, FaceDetectionListener's functions are limited compared to OpenCV.
Any help would be greatly appreciated. Thx.
Something like the following works for me with android:screenOrientation="portrait" in AndroidManifest.xml
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
mRgba = inputFrame.rgba();
mGray = inputFrame.gray();
MatOfRect faces = new MatOfRect();
Core.flip(mRgba.t(), mRgba, -1);
Core.flip(mGray.t(), mGray, -1);
if (mNativeDetector != null)
mNativeDetector.detect(mGray, faces);
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++)
Core.rectangle(mRgba, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 2);
return mRgba;
}
Flip the colored and gray image(Mat) clockwise for face/feature detection to work in portrait mode.
At the end of feature detection Logic, you flip counter clockwise the colored image(mRgba Mat) . As illustrated.
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
Core.flip(inputFrame.gray().t(),mGray,1); //rotate clockwise
Core.flip(inputFrame.rgba().t(),mRgba,1);
mRgba=Feature_DetectionNATIVE(mRgba,mGray);
Core.flip(mRgba.t(),mRgba,0); //rotate counter clockwise
//this is a solution for allowing face detection in portrait view if it isn't working at all.
return mRgba;
}
public Mat Feature_DetectionNATIVE(Mat mRgba2, final Mat Gray)
{
if (mAbsoluteFaceSize == 0)
{
int height = Gray.rows();
if (Math.round(height * mRelativeFaceSize) > 0)
{
mAbsoluteFaceSize = Math.round(height * mRelativeFaceSize);
}
mNativeDetector.setMinFaceSize(mAbsoluteFaceSize);
}
MatOfRect faces = new MatOfRect();
if (mDetectorType == JAVA_DETECTOR)
{
if (mJavaDetector != null)
mJavaDetector.detectMultiScale(Gray, faces, 1.1, 2, 2,
new Size(mAbsoluteFaceSize, mAbsoluteFaceSize), new Size());
}
else if (mDetectorType == NATIVE_DETECTOR)
{
if (mNativeDetector != null)
mNativeDetector.detect(Gray, faces);
}
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++)
{
Core.rectangle(mRgba2, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 3);
}
return mRgba2;
}
After that the camera will show face detection in landscape orientation to fix this you rotate the canvas clockwise by 90 in opencv's CameraBridgeViewBase main class or hack it.(Note this reduces FPS but face detection is still fast)
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
Log.d(TAG, "mStretch value: " + mScale);
canvas=rotateCanvas(canvas,mCacheBitmap);
getHolder().unlockCanvasAndPost(canvas);
}
}
}
protected Canvas rotateCanvas(final Canvas canvas, final Bitmap mCacheBitmap)
{
final CountDownLatch latch =new CountDownLatch(1);
final Mat[] mRgba=new Mat[1];
new Thread(new Runnable() {
#Override
public void run() {
try {
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
canvas.rotate(90,0,0);
mScale = canvas.getWidth() / (float)bitmap.getHeight();
float scale2 = canvas.getHeight() / (float)bitmap.getWidth();
if(scale2 > mScale){
mScale = scale2;
}
if (mScale != 0) {
canvas.scale(mScale, mScale,0,0);
}
canvas.drawBitmap(bitmap, 0, -bitmap.getHeight(), null);
}
catch (CvException e) { e.printStackTrace();}
latch.countDown();//setting //release await() in this thread
}
}).start();
try { latch.await(); //waits for countDown in the Thread inorder to obtain a value from the thread
} catch (InterruptedException e) { e.printStackTrace();}
return canvas;
}
This solution works when one is implementing Opencv's CameraBridgeViewBase class
(Tested on OpenCV 2.4.9)
I'm studying OpenCV recently.
OpenCV for Android sample code(2.4.6) is tested.
I have some wonder.
I can detect face by sample code(OpenCV Sample - face-detection). but, can't detect face by portrait mode(vertical mode) on android device.
First, I try set Front-Camera by portrait mode.
//added code in onCameraFrame() method.
Core.flip(mRgba, mRgba, 1);
Still do not recognize face.
How to detect face by portrait mode?
Thank you.
you need to do transpose and then flip:
int height = mGray.rows();
int faceSize = Math.round(height * 0.5F);
Mat temp = mGray.clone();
Core.transpose(mGray, temp);
Core.flip(temp, temp, -1);
MatOfRect rectFaces = new MatOfRect();
// java detector fast
mCascade.detectMultiScale(temp, rectFaces, 1.1, 1, 0, new Size(faceSize, faceSize), new Size());
Set the portrait mode in android's manifest first.
android:screenOrientation="portrait"
Flip the colored and gray image(Mat) clockwise for face/feature detection to work in portrait mode.
At the end of feature detection Logic, you flip counter clockwise the colored image(mRgba Mat) . As illustrated.
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
Core.flip(inputFrame.gray().t(),mGray,1); //rotate clockwise
Core.flip(inputFrame.rgba().t(),mRgba,1);
mRgba=Feature_DetectionNATIVE(mRgba,mGray);
Core.flip(mRgba.t(),mRgba,0); //rotate counter clockwise
//this is a solution for allowing face detection in portrait view if it isn't working at all.
return mRgba;
}
public Mat Feature_DetectionNATIVE(Mat mRgba2, final Mat Gray)
{
if (mAbsoluteFaceSize == 0)
{
int height = Gray.rows();
if (Math.round(height * mRelativeFaceSize) > 0)
{
mAbsoluteFaceSize = Math.round(height * mRelativeFaceSize);
}
mNativeDetector.setMinFaceSize(mAbsoluteFaceSize);
}
MatOfRect faces = new MatOfRect();
if (mDetectorType == JAVA_DETECTOR)
{
if (mJavaDetector != null)
mJavaDetector.detectMultiScale(Gray, faces, 1.1, 2, 2,
new Size(mAbsoluteFaceSize, mAbsoluteFaceSize), new Size());
}
else if (mDetectorType == NATIVE_DETECTOR)
{
if (mNativeDetector != null)
mNativeDetector.detect(Gray, faces);
}
else
{
Log.e(TAG, "Detection method is not selected!");
}
Rect[] facesArray = faces.toArray();
for (int i = 0; i < facesArray.length; i++)
{
Core.rectangle(mRgba2, facesArray[i].tl(), facesArray[i].br(), FACE_RECT_COLOR, 3);
}
return mRgba2;
}
After that the camera will show face detection in landscape orientation to fix this you rotate the canvas clockwise by 90 in opencv's CameraBridgeViewBase main class or hack it.(Note this reduces FPS but face detection is still fast)
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
Log.d(TAG, "mStretch value: " + mScale);
canvas=rotateCanvas(canvas,mCacheBitmap);
getHolder().unlockCanvasAndPost(canvas);
} } }
protected Canvas rotateCanvas(final Canvas canvas, final Bitmap mCacheBitmap)
{
final CountDownLatch latch =new CountDownLatch(1);
final Mat[] mRgba=new Mat[1];
new Thread(new Runnable() {
#Override
public void run() {
try {
Bitmap bitmap = Bitmap.createScaledBitmap(mCacheBitmap, canvas.getHeight(), canvas.getWidth(), true);
canvas.rotate(90,0,0);
mScale = canvas.getWidth() / (float)bitmap.getHeight();
float scale2 = canvas.getHeight() / (float)bitmap.getWidth();
if(scale2 > mScale){
mScale = scale2;
}
if (mScale != 0) {
canvas.scale(mScale, mScale,0,0);
}
canvas.drawBitmap(bitmap, 0, -bitmap.getHeight(), null);
}
catch (CvException e) { e.printStackTrace();}
latch.countDown();//setting //release await() in this thread
}
}).start();
try { latch.await(); //waits for countDown in the Thread inorder to obtain a value from the thread
} catch (InterruptedException e) { e.printStackTrace();}
return canvas;
}
(Using OpenCV 2.4.9)
I have a simple project that just show the camera with org.opencv.android.JavaCameraView.
my problem is that in default the camera is on landscape mode and I can't change this cause I need to define CameraBridgeViewBase instead of a regular camera intent.
this is a part of my code:
XML code:
<LinearLayout
android:layout_width="match_parent"
android:layout_height="match_parent" >
<org.opencv.android.JavaCameraView
android:layout_width="fill_parent"
android:layout_height="300dp"
android:visibility="gone"
android:id="#+id/HelloOpenCvView"
opencv:show_fps="true"
opencv:camera_id="1" />
</LinearLayout>
<LinearLayout
android:layout_width="match_parent"
android:layout_height="wrap_content" >
<Button
android:id="#+id/BtnVideo"
android:layout_marginLeft="2dp"
android:layout_marginRight="2dp"
android:layout_width="0dp"
style="#style/button"
android:layout_height="wrap_content"
android:layout_weight="1.00"
android:text="#string/videoBtn"
android:textSize="18dip" />
</LinearLayout>
Java Code :
CameraBridgeViewBase mOpenCvCameraView;
Button VideoButton;
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
overridePendingTransition(0, 0);
VideoButton = (Button) this.findViewById(R.id.BtnVideo);
VideoButton.setOnClickListener(onClickListener);
mOpenCvCameraView= (CameraBridgeViewBase) findViewById(R.id.HelloOpenCvView);
mOpenCvCameraView.setVisibility(SurfaceView.INVISIBLE);
}
private OnClickListener onClickListener = new OnClickListener() {
#Override
public void onClick(View v) {
switch (v.getId()){
case R.id.BtnVideo:
if(mOpenCvCameraView.getVisibility() == SurfaceView.VISIBLE)
{
mOpenCvCameraView.setVisibility(SurfaceView.INVISIBLE);
}
else
{
mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
}
break;
default :
break;
}
}
};
public void onResume() {
super.onResume();
overridePendingTransition(0, 0);
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_3, this, mLoaderCallback);
}
public void onPause()
{
super.onPause();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
public void onDestroy() {
super.onDestroy();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
public void onCameraViewStarted(int width, int height) {
}
public void onCameraViewStopped() {
}
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
return inputFrame.rgba();
}
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
//Log.i(TAG, "OpenCV loaded successfully");
mOpenCvCameraView.enableView();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
So how can I change the default orientation?
Thanks!
Ok, I found this as a solution:
First I get into JavaCameraView.java class in the OpenCV Library - 2.4.5
and then in initializeCamera() function before mCamera.startPreview(); I added these 2 function:
setDisplayOrientation(mCamera, 90);
mCamera.setPreviewDisplay(getHolder());
and the first function implemented like this:
protected void setDisplayOrientation(Camera camera, int angle){
Method downPolymorphic;
try
{
downPolymorphic = camera.getClass().getMethod("setDisplayOrientation", new Class[] { int.class });
if (downPolymorphic != null)
downPolymorphic.invoke(camera, new Object[] { angle });
}
catch (Exception e1)
{
e1.printStackTrace();
}
}
I just reminding that I work with OpenCV.
Hope this help someone.
The problem is that the code that paints doesn't check the camera params. The Mat is drawn on the Surface view in the function "deliverAndDrawFrame" in the class "CameraBridgeViewBase".
With a very simple modification in the CameraBridgeViewBase class, we can make a function that rotates the way the bitmap is drawn.
int userRotation= 0;
public void setUserRotation(int userRotation) {
this.userRotation = userRotation;
}
/**
* This method shall be called by the subclasses when they have valid
* object and want it to be delivered to external client (via callback) and
* then displayed on the screen.
* #param frame - the current frame to be delivered
*/
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
Mat modified;
if (mListener != null) {
modified = mListener.onCameraFrame(frame);
} else {
modified = frame.rgba();
}
boolean bmpValid = true;
if (modified != null) {
try {
Utils.matToBitmap(modified, mCacheBitmap);
} catch(Exception e) {
Log.e(TAG, "Mat type: " + modified);
Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
bmpValid = false;
}
}
if (bmpValid && mCacheBitmap != null) {
Canvas canvas = getHolder().lockCanvas();
if (canvas != null) {
canvas.drawColor(Color.parseColor("#8BC34A"), PorterDuff.Mode.SRC_IN);
//this is the rotation part
canvas.save();
canvas.rotate(userRotation, (canvas.getWidth()/ 2),(canvas.getHeight()/ 2));
if (mScale != 0) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
(canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
}
if (mFpsMeter != null) {
mFpsMeter.measure();
mFpsMeter.draw(canvas, 20, 30);
}
//remember to restore the canvas
canvas.restore();
getHolder().unlockCanvasAndPost(canvas);
}
}
}
I tried the most common solution that use the Core.flip function that rotates the Mat but consumes a lot of resources, this solution doesn't affect to the detection and doesn't affect to the performance, only changes the way the image is drawn on the canvas.
Hope this help.
I am using OpenCV 3.1, I fix it by apply transform when draw bitmap on deliverAndDrawFrame method of CameraBridgeViewBase class, Hope it helpful:
On CameraBridgeViewBase.java :
//I added new field
private final Matrix mMatrix = new Matrix();
//added updateMatrix method
private void updateMatrix() {
float hw = this.getWidth() / 2.0f;
float hh = this.getHeight() / 2.0f;
boolean isFrontCamera = Camera.CameraInfo.CAMERA_FACING_FRONT == mCameraIndex;
mMatrix.reset();
if (isFrontCamera) {
mMatrix.preScale(-1, 1, hw, hh);
}
mMatrix.preTranslate(hw, hh);
if (isFrontCamera)
mMatrix.preRotate(270);
else
mMatrix.preRotate(90);
mMatrix.preTranslate(-hw, -hh);
}
//then We need call updateMatrix on layout
#Override
public void layout(int l, int t, int r, int b) {
super.layout(l, t, r, b);
updateMatrix();
}
//I think we should also call updateMatrix on measure
#Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
super.onMeasure(widthMeasureSpec, heightMeasureSpec);
updateMatrix();
}
//then We need update deliverAndDrawFrame
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
//....Origin Code...
//Set matrix before OpenCV draw bitmap
int saveCount = canvas.save();
canvas.setMatrix(mMatrix);
//Begin OpenCv origin source
if (mScale != 0) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
(canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
}
//End OpenCv origin source
//Restore canvas after draw bitmap
canvas.restoreToCount(saveCount);
//....Origin code...
}
//After that We can see that the camera preview is so small, the easiest way is change mScale Value (should we change mScale to "private" instead "protected" ?)
protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
//....Origin Code...
//Set matrix before OpenCV draw bitmap to screen
int saveCount = canvas.save();
canvas.setMatrix(mMatrix);
//Change mScale to "Aspect to fill"
mScale = Math.max((float) canvas.getHeight() / mCacheBitmap.getWidth(), (float) canvas.getWidth() / mCacheBitmap.getHeight());
//Begin OpenCv origin source
if (mScale != 0) {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2),
(int)((canvas.getWidth() - mScale*mCacheBitmap.getWidth()) / 2 + mScale*mCacheBitmap.getWidth()),
(int)((canvas.getHeight() - mScale*mCacheBitmap.getHeight()) / 2 + mScale*mCacheBitmap.getHeight())), null);
} else {
canvas.drawBitmap(mCacheBitmap, new Rect(0,0,mCacheBitmap.getWidth(), mCacheBitmap.getHeight()),
new Rect((canvas.getWidth() - mCacheBitmap.getWidth()) / 2,
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2,
(canvas.getWidth() - mCacheBitmap.getWidth()) / 2 + mCacheBitmap.getWidth(),
(canvas.getHeight() - mCacheBitmap.getHeight()) / 2 + mCacheBitmap.getHeight()), null);
}
//End OpenCv origin source
//Restore canvas after draw bitmap
canvas.restoreToCount(saveCount);
//....Origin Code...
}
You can get full source code here:
https://gist.github.com/thongdoan/d73267eb58863f70c77d1288fe5cd3a4
Try this one on your onCameraFrame
mRgba = inputFrame.rgba();
Mat mRgbaT = mRgba.t();
Core.flip(mRgbaT, matRgbaFlip, 1);
Imgproc.resize(matRgbaFlip, matRgbaFlip, mRgba.size());
mRgbaT.release();
return matRgbaFlip;
EDIT: Fix memory leak, releasing the Mat created in the method and moving the other one as a global variable.
Firstly, do not create an instance from the base class, instead get the instance from the extended class
//CameraBridgeViewBase mOpenCvCameraView;
JavaCameraView mOpenCvCameraView;
It is worth mentioning that the CameraBridgeViewBase.java has already a surface holder, so let's use it instead of creating a surface texture.
Hence secondly, edit the JavaCameraView.java inside the function initializeCamera() by replacing the surface texture with a surface holder
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
//mSurfaceTexture = new SurfaceTexture(MAGIC_TEXTURE_ID);
//mCamera.setPreviewTexture(mSurfaceTexture);
mCamera.setPreviewDisplay(getHolder());
} else
mCamera.setPreviewDisplay(null);
The last step is to set the orientation WITHOUT adding any special functions. In the same function initializeCamera() just before startPreview() call setDisplayOrientation(degrees)
/* Finally we are ready to start the preview */
Log.d(TAG, "startPreview");
mCamera.setDisplayOrientation(90);
mCamera.startPreview();
In OpenCV version 3.4.3
I fixed it by doing below changes in initializeCamera() method in JavaCameraView.java class.
setDisplayOrientation(mCamera, 0);
Steps to follow:
Go to the JavaCameraView.java file.
Search for "setDisplayOrientation"
Set orientation angle to 0 (By default it was 90).
the android:screenOrientation value in the AndroidManifest.xml should help.
android:screenOrientation="portrait"