Develop Reference

Perspective Transform OpenCV

I'm new to OpenCV on Android and try to do Perspective Transform but I don't know how to use getperspectivetransform() and warpperspective() functions.I could detect rectangle from an image, but don't know how to warp.
Here is the detect rectangle function:
Mat tempMat = new Mat();
Mat src = new Mat();
Utils.bitmapToMat(image, tempMat);
Imgproc.cvtColor(tempMat, src, Imgproc.COLOR_BGR2RGB);
Mat blurred = src.clone();
Imgproc.medianBlur(src, blurred, 9);
Mat gray0 = new Mat(blurred.size(), CvType.CV_8U), gray = new Mat();
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
List<Mat> blurredChannel = new ArrayList<Mat>();
blurredChannel.add(blurred);
List<Mat> gray0Channel = new ArrayList<Mat>();
gray0Channel.add(gray0);
MatOfPoint2f approxCurve = new MatOfPoint2f();
double maxArea = 0;
int maxId = -1;
for (int c = 0; c < 3; c++) {
int ch[] = { c, 0 };
Core.mixChannels(blurredChannel, gray0Channel, new MatOfInt(ch));
int thresholdLevel = 1;
for (int t = 0; t < thresholdLevel; t++) {
if (t == 0) {
Imgproc.Canny(gray0, gray, 50, 50, 3, true); // true ?
Imgproc.dilate(gray, gray, new Mat(), new Point(-1, -1), 1); // 1
// ?
} else {
Imgproc.adaptiveThreshold(gray0, gray, thresholdLevel,
Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C,
Imgproc.THRESH_BINARY,
(src.width() + src.height()) / 200, t);
}
Imgproc.findContours(gray, contours, new Mat(),
Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
for (MatOfPoint contour : contours) {
MatOfPoint2f temp = new MatOfPoint2f(contour.toArray());
double area = Imgproc.contourArea(contour);
approxCurve = new MatOfPoint2f();
Imgproc.approxPolyDP(temp, approxCurve,
Imgproc.arcLength(temp, true) * 0.02, true);
if (approxCurve.total() == 4 && area >= maxArea) {
double maxCosine = 0;
List<Point> curves = approxCurve.toList();
for (int j = 2; j < 5; j++) {
double cosine = Math.abs(angle(curves.get(j % 4),
curves.get(j - 2), curves.get(j - 1)));
maxCosine = Math.max(maxCosine, cosine);
}
if (maxCosine < 0.45) {
maxArea = area;
maxId = contours.indexOf(contour);
}
}
}
}
}
I draw rectangle with this statement.
if (maxId >= 0) {
Rect rect = Imgproc.boundingRect(contours.get(maxId));
Imgproc.rectangle(src, rect.tl(), rect.br(), new Scalar(255, 0, 0,
.8), 4);
}
After that I convert mat to bitmap and show on an imageview.
Here is the screenshot
So my problem is warpping, How can I warp the rectangle and rotate it?
and If it is possible, how can I improve detecting rectangle? Any hints?
(OpenCV Android SDK Ver: 3.41, Android Studio Ver: 3.01)

If you are looking to warp the detected contour into rectangle,
Get the contours of the rectangle
find convex hull of the contour
Using approxPolyDP reduce the convex hull points into 4 points
fit line to consecutive points (ex, if pts is the array, lines are fit as follows l1 = line Between(pts[0], pts[1]), l2 = line Between(pts[1], pts[2]), l3 = line Between(pts[2], pts[3]), l4 = lineBetween(pts[3], pts[0])
find the intersection between these lines, you'll end up with four points
Order the points in clockwise order (inputCorners = TopLeft, TopRight, BottomRight, BottomLeft)
create an output image with needed resolution and make the corner points in the same clockwise order ((0,0), (0, cols), (rows, cols), (rows, 0))
find homography using the function
Mat homography = Calib3d.findHomography(inputCorners, imageCorners, Calib3d.RANSAC, 10);
using the output homography matrix, warp the input image using the function
Imgproc.warpPerspective(image, outputMat, homography, new Size(image.cols(), image.rows()));
you can refer to the following link

This is my kotlin extensin version you can use it in your projects.
fun Bitmap.perspectiveTransform(srcPoints: List<org.opencv.core.Point>) :
Bitmap{
val dstWidth = max(
srcPoints[0].distanceFrom(srcPoints[1]),
srcPoints[2].distanceFrom(srcPoints[3])
)
val dstHeight = max(
srcPoints[0].distanceFrom(srcPoints[2]),
srcPoints[1].distanceFrom(srcPoints[3])
)
val dstPoints: List<org.opencv.core.Point> = listOf(
org.opencv.core.Point(0.0, 0.0),
org.opencv.core.Point(dstWidth, 0.0),
org.opencv.core.Point(0.0, dstHeight),
org.opencv.core.Point(dstWidth, dstHeight)
)
return try {
val srcMat = Converters.vector_Point2d_to_Mat(srcPoints)
val dstMat = Converters.vector_Point2d_to_Mat(dstPoints)
val perspectiveTransformation =
Imgproc.getPerspectiveTransform(srcMat, dstMat)
val inputMat = Mat(this.height, this.width, CvType.CV_8UC1)
Utils.bitmapToMat(this, inputMat)
val outPutMat = Mat(dstHeight.toInt(), dstWidth.toInt(), CvType.CV_8UC1)
Imgproc.warpPerspective(
inputMat,
outPutMat,
perspectiveTransformation,
Size(dstWidth, dstHeight)
)
val outPut = Bitmap.createBitmap(
dstWidth.toInt(),
dstHeight.toInt(), Bitmap.Config.RGB_565
)
//Imgproc.cvtColor(outPutMat , outPutMat , Imgproc.COLOR_GRAY2BGR)
Utils.matToBitmap(outPutMat , outPut)
outPut
}
catch ( e : Exception){
e.printStackTrace()
this
}
}
To use distance from I write another extension function
fun org.opencv.core.Point.distanceFrom(srcPoint: org.opencv.core.Point):
Double {
val w1 = this.x - srcPoint.x
val h1 = this.y - srcPoint.y
val distance = w1.pow(2) + h1.pow(2)
return sqrt(distance)
}
Also in this answer the correct src Points indices are :
0 : topleft
1 : topRight
2 : bottomLeft
3 : bottomRight
Good luck

OpenCV, Java: Color Detection in a certain specified area

I currently have this image:
I managed to detect the black object. Now i want to detect the green object, but i only want the application to look for the green object below the black object. I already have the code to detect the green tape and its working. Just need to set it so that its only in the area below the black object.
Resulting image should still look like this:
P.S some of the variables are named "Blue", rest assured its using Green scalar values.
Code:
//Detect Black
private Bitmap findCombine(Bitmap sourceBitmap) {
Bitmap roiBitmap = null;
Scalar green = new Scalar(0, 255, 0, 255);
Mat sourceMat = new Mat(sourceBitmap.getWidth(), sourceBitmap.getHeight(), CvType.CV_8UC3);
Utils.bitmapToMat(sourceBitmap, sourceMat);
Mat roiTmp = sourceMat.clone();
bitmapWidth = sourceBitmap.getWidth();
Log.e("bitmapWidth", String.valueOf(bitmapWidth));
final Mat hsvMat = new Mat();
sourceMat.copyTo(hsvMat);
// convert mat to HSV format for Core.inRange()
Imgproc.cvtColor(hsvMat, hsvMat, Imgproc.COLOR_RGB2HSV);
Scalar lowerb = new Scalar(85, 50, 40); // lower color border for BLUE
Scalar upperb = new Scalar(135, 255, 255); // upper color border for BLUE
Scalar lowerblack = new Scalar(0, 0, 0); // lower color border for BLACK
Scalar upperblack = new Scalar(180, 255, 40); // upper color border for BLACK
Scalar testRunL = new Scalar(60, 50, 40); // lower Green 83 100 51
Scalar testRunU = new Scalar(90, 255, 255); // upper Green
Core.inRange(hsvMat, lowerblack, upperblack, roiTmp); // select only blue pixels
// find contours
List<MatOfPoint> contours = new ArrayList<>();
List<RotatedRect> boundingRects = new ArrayList<>();
Imgproc.findContours(roiTmp, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// find appropriate bounding rectangles
for (MatOfPoint contour : contours) {
MatOfPoint2f areaPoints = new MatOfPoint2f(contour.toArray());
RotatedRect boundingRect = Imgproc.minAreaRect(areaPoints);
double rectangleArea = boundingRect.size.area();
// test min ROI area in pixels
if (rectangleArea > 1300 && rectangleArea < 500000) {//400000
Point rotated_rect_points[] = new Point[4];
boundingRect.points(rotated_rect_points);
Rect rect3 = Imgproc.boundingRect(new MatOfPoint(rotated_rect_points));
Log.e("blackArea", String.valueOf(rect3.area()));
// test horizontal ROI orientation
if (rect3.height > rect3.width) {
Imgproc.rectangle(sourceMat, rect3.tl(), rect3.br(), green, 3);
xBlack = rect3.br().x;
xBlackCenter = (rect3.br().x+ rect3.tl().x) /2;
yBlack = rect3.br().y;//bottom
battHeight = (rect3.br().y - rect3.tl().y); //batt height in pixel
Log.e("BLACKBR, TL", String.valueOf(rect3.br().y) + "," + String.valueOf(rect3.tl().y));
}
}
}
roiBitmap = Bitmap.createBitmap(sourceMat.cols(), sourceMat.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(sourceMat, roiBitmap);
//Set area to detect green
Point leftPoint = new Point(0, yBlack); //far left, black object height
Point rightPoint = new Point(roiBitmap.getWidth(), roiBitmap.getHeight()); //btm right of entire bitmap
Rect bottomRect = new Rect(leftPoint, rightPoint);
double rectWidth = sourceBitmap.getWidth() - 0;
double rectHeight = sourceBitmap.getHeight() - yBlack;
Log.e("rectWidth", String.valueOf(rectWidth));
Log.e("rectHeight", String.valueOf(rectHeight));
Mat sourceMatT = new Mat(roiBitmap.getWidth(), roiBitmap.getHeight(), CvType.CV_8UC3);
Utils.bitmapToMat(roiBitmap,sourceMatT);
Bitmap C = Bitmap.createBitmap(sourceMatT.cols(), sourceMatT.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(sourceMatT, C);
Mat dumbMat = sourceMatT.clone();
Log.e("sourceMatT, BottomRect","SMT "+ String.valueOf(sourceMatT.size()) + " bottomRect " + String.valueOf(bottomRect.size()));
Mat cropMat = new Mat(dumbMat, bottomRect);
ImageView imgCropped = (ImageView) findViewById(R.id.cropped_image_view);
//Utils.matToBitmap(cropMat,C);
imgCropped.setImageBitmap(C);
//Detect Green
Bitmap roiBitmap2 = null;
Mat sourceMat2 = new Mat(C.getWidth(), C.getHeight(), CvType.CV_8UC3);
Utils.bitmapToMat(C, sourceMat2);
Mat roiTmp2 = sourceMat2.clone();
final Mat hsvMat2 = new Mat();
sourceMat.copyTo(hsvMat2);
// convert mat to HSV format for Core.inRange()
Imgproc.cvtColor(hsvMat2, hsvMat2, Imgproc.COLOR_RGB2HSV);
Core.inRange(hsvMat2, testRunL, testRunU, roiTmp2); // select only blue pixels
// find contours
List<MatOfPoint> contours2 = new ArrayList<>();
List<RotatedRect> boundingRects2 = new ArrayList<>();
Imgproc.findContours(roiTmp2, contours2, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// find appropriate bounding rectangles
for (MatOfPoint contour2 : contours2) {
MatOfPoint2f areaPoints2 = new MatOfPoint2f(contour2.toArray());
RotatedRect boundingRect2 = Imgproc.minAreaRect(areaPoints2);
double rectangleArea2 = boundingRect2.size.area();
// test min ROI area in pixels
if (rectangleArea2 > 40) { //214468.32402064091 // 20000
Point rotated_rect_points2[] = new Point[4];
boundingRect2.points(rotated_rect_points2);
Rect rect = Imgproc.boundingRect(new MatOfPoint(rotated_rect_points2));
Log.e("area green", String.valueOf(boundingRect2.size.area()));
// test vertical ROI orientation
if (rect.width > rect.height) {
if (numRect < 2) {
Imgproc.rectangle(sourceMat2, rect.tl(), rect.br(), green, 3);
xBlue = (rect.br().x + rect.tl().x) / 2; //center
yBlue = rect.br().y; //bottom
Log.e("GREEN br,tl", String.valueOf(rect.br().y) + " " + String.valueOf(rect.tl().y));
}
}
}
}
Point firstPoint = new Point(xBlackCenter, yBlack);
Point secondPoint = new Point(xBlackCenter, yBlue);
Point middlePoint = new Point(firstPoint.x,
firstPoint.y + 0.5 * (secondPoint.y - firstPoint.y));
Scalar lineColor = new Scalar(255, 0, 0, 255);
int lineWidth = 3;
Scalar textColor = new Scalar(255, 0, 0, 255);
//height of bounce = BattHeight IRL / battHeihgt Pixel * line Height Pixel
double lineHeightCm = (4.65 / battHeight) * findHeight(yBlack, yBlue);
Log.e("PixelBatt/PixelBounce", "BattH: " + battHeight + " find height " + String.valueOf(findHeight(xBlack, xBlue)) + "!");
Log.e("Blacky-blueY", String.valueOf(xBlue - xBlack));
Imgproc.line(sourceMat2, firstPoint, secondPoint, lineColor, lineWidth);
Imgproc.putText(sourceMat2, String.valueOf(lineHeightCm), middlePoint,
Core.FONT_HERSHEY_PLAIN, 3.5, textColor);
roiBitmap2 = Bitmap.createBitmap(sourceMat2.cols(), sourceMat2.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(sourceMat2, roiBitmap2);
TextView tvR = (TextView) findViewById(R.id.tvR);
tvR.setText("Bounce Height = " + lineHeightCm + "cm");
return roiBitmap2;
}
Error:
CvException [org.opencv.core.CvException: /build/master_pack-android/opencv/modules/java/generator/src/cpp/utils.cpp:97: error: (-215) src.dims == 2 && info.height == (uint32_t)src.rows && info.width == (uint32_t)src.cols in function void Java_org_opencv_android_Utils_nMatToBitmap2(JNIEnv*, jclass, jlong, jobject, jboolean)

There is no need to find green objects in specific area: You can find green contours on entire image, then just test it coordinates relative to black rectangle. Something like that:
At first - find black rectangle.
Rect blackRect = findBlackRect();
Then find contours of ALL green objects (same way as You find black):
// find green contours
List<MatOfPoint> greenContours = new ArrayList<>();
Imgproc.findContours(roiMat, greenContours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
Then test what of green contours laying below of black rect (has grater Y coord)
// find appropriate bounding rectangles
for (MatOfPoint contour : greenContours) {
MatOfPoint2f areaPoints = new MatOfPoint2f(contour.toArray());
RotatedRect boundingRect = Imgproc.minAreaRect(areaPoints);
Point rotated_rect_points[] = new Point[4];
boundingRect.points(rotated_rect_points);
Rect rect = Imgproc.boundingRect(new MatOfPoint(rotated_rect_points));
// test top left Y coord of bounding rectangle of green contour grater than
// Y coord of top left of black rectangle
if (rect.tl().y > blackRect.tl().y) {
// that is green contour under black rectangle
// just draw it
Imgproc.rectangle(sourceMat, rect.tl(), rect.br(), green, 3);
}
}
And so on...

Opencv(3.10) android- How to draw an object's trajectory

I have a project in college where I have to draw the trajectory of an object based on the color of it. Is being built based on colorBlobDetection. This is part of the code I have so far:
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
mRgba = inputFrame.rgba();
if (mIsColorSelected) {
mDetector.process(mRgba);
List<MatOfPoint> contours = mDetector.getContours();
Log.e(TAG, "Contours count: " + contours.size());
Imgproc.drawContours(mRgba, contours, -1, CONTOUR_COLOR);
List<Moments> mu = new ArrayList<Moments>(contours.size());
for (int i = 0; i < contours.size(); i++) {
mu.add(i, Imgproc.moments(contours.get(i), false));
Moments p = mu.get(i);
int xContour = (int) (p.get_m10() / p.get_m00());
int yContour = (int) (p.get_m01() / p.get_m00());
Point ponto = new Point(xContour, yContour);
//Imgproc.line(mRgba, ponto, ponto, CONTOUR_COLOR, 5, Imgproc.LINE_AA, 0);
}
At the moment I'm picking up the center of each contour.
Through the positions of the contours I want to draw their trajectories when they move
Ps. This 'Imgproc.line' was just to test if the center of each contour is correct.
Please help.

Why is pointPolygonTest() method of OpenCV4Android returning -1 for every pixel?

In the following code, I have carried out the following steps:
Loaded an image from sdcard.
Converted it to HSV format.
Used inRange function to mask out the red color.
Used findContours to find the contours.
Find the largest contour from those contours.
Created an ROI around the largest contour using boundingRect and submat functions.
Converted this ROI Mat to HSV format.
Iterated through the ROI Mat, and check for each pixel if it lies within the largest contour. I used the method pointPolygonTest to find this out, but it returns -1 for every pixel, as can be seen from the Log.i output I have pasted here. The question is why? How can I correct this.
private Scalar detectColoredBlob() {
rgbaFrame = Highgui.imread("/mnt/sdcard/DCIM/rgbaMat4Mask.bmp");
Mat hsvImage = new Mat();
Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/hsvImage.bmp", hsvImage);// check
Mat maskedImage = new Mat();
Core.inRange(hsvImage, new Scalar(0, 100, 100), new Scalar(10, 255, 255), maskedImage);
Highgui.imwrite("/mnt/sdcard/DCIM/maskedImage.bmp", maskedImage);// check
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(maskedImage, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// \/ We will use only the largest contour. Other contours (any other possible blobs of this color range) will be ignored.
MatOfPoint largestContour = contours.get(0);
double largestContourArea = Imgproc.contourArea(largestContour);
for (int i = 1; i < contours.size(); ++i) {// NB Notice the prefix increment.
MatOfPoint currentContour = contours.get(i);
double currentContourArea = Imgproc.contourArea(currentContour);
if (currentContourArea > largestContourArea) {
largestContourArea = currentContourArea;
largestContour = currentContour;
}
}
MatOfPoint2f largestContour2f = new MatOfPoint2f(largestContour.toArray());// Required on Line 289. See http://stackoverflow.com/questions/11273588/how-to-convert-matofpoint-to-matofpoint2f-in-opencv-java-api
Rect detectedBlobRoi = Imgproc.boundingRect(largestContour);
Mat detectedBlobRgba = rgbaFrame.submat(detectedBlobRoi);
Highgui.imwrite("/mnt/sdcard/DCIM/detectedBlobRgba.bmp", detectedBlobRgba);// check
Mat detectedBlobHsv = new Mat();
Imgproc.cvtColor(detectedBlobRgba, detectedBlobHsv, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/roiHsv.bmp", detectedBlobHsv);// check
for (int firstCoordinate = 0; firstCoordinate < detectedBlobHsv.rows(); firstCoordinate++) {
for (int secondCoordinate = 0; secondCoordinate < detectedBlobHsv.cols(); secondCoordinate++) {
Log.i(TAG, "HAPPY " + Arrays.toString(detectedBlobHsv.get(firstCoordinate, secondCoordinate)));
if (Imgproc.pointPolygonTest(largestContour2f, new Point(firstCoordinate, secondCoordinate), false) == -1) {
Log.i(TAG, "HAPPY ....................... OUTSIDE");
}
}
}
Highgui.imwrite("/mnt/sdcard/DCIM/processedcontoured.bmp", detectedBlobHsv);// check
EDIT:
I am doing this because I need to compute the average HSV color of pixels lying inside the contour (i.e. the average HSV color of the biggest red colored blob). If I computed the average color of the ROI detectedBlobHsv by the normal formula, I would do something like
Scalar averageHsvColor= new Scalar(256);
Scalar sumHsvOfPixels = new Scalar(256);
sumHsvOfPixels = Core.sumElems(detectedBlobHsv);
int numOfPixels = detectedBlobHsv.width() * detectedBlobHsv.height();
for (int channel=0; channel<sumHsvOfPixels.val.length; channel++) {
averageHsvColor = sumHsvOfPixels.val[channel]/numOfPixels;
}
So somebody here on SO (probably you?) had suggested me a way to exclude pixels outside my contour a while back. I'd implement that like:
//Giving pixels outside contour of interest an HSV value of `double[]{0,0,0}`, so that they don't affect the computation of `sumHsvOfPixels` while computing average,
//and while keeping track of the number of pixels removed from computation this way, so we can subtract that number from the `$numOfPixels` during computation of average.
int pixelsRemoved = 0;
for (int row=0; row<detectedBlobHsv.rows(); row++) {
for (int col=0; col<detectedBlobHsv.cols(); col++) {
if (Imgproc.pointPolygonTest(largestContour2f, new Point(row, col), false) == -1) {
detectedBlobHsv.put(row, col, new double[]{0,0,0});
pixelsRemoved++;
}
}
}
Then compute the average like
Scalar averageHsvColor= new Scalar(256);
Scalar sumHsvOfPixels = new Scalar(256);
sumHsvOfPixels = Core.sumElems(detectedBlobHsv); //This will now exclude pixels outside the contour
int numOfPixels = ( detectedBlobHsv.width()*detectedBlobHsv.height() )-pixelsRemoved;
for (int channel=0; channel<sumHsvOfPixels.val.length; channel++) {
averageHsvColor = sumHsvOfPixels.val[channel]/numOfPixels;
}
EDIT 1:
Towards the end of the following method, I have created the mask with a list of MatOfPoints which contains the largest contour only. When I wrote it to SDCard, I got
I don't know where I messed up!
private Scalar detectColoredBlob() {
//Highgui.imwrite("/mnt/sdcard/DCIM/rgbaFrame.jpg", rgbaFrame);// check
rgbaFrame = Highgui.imread("/mnt/sdcard/DCIM/rgbaMat4Mask.bmp");
//GIVING A UNIFORM VALUE OF 255 TO THE V CHANNEL OF EACH PIXEL (255 IS THE MAXIMUM VALUE OF V ALLOWED - Simulating a maximum light condition)
for (int firstCoordinate = 0; firstCoordinate < rgbaFrame.rows(); firstCoordinate++) {
for (int secondCoordinate = 0; secondCoordinate < rgbaFrame.cols(); secondCoordinate++) {
double[] pixelChannels = rgbaFrame.get(firstCoordinate, secondCoordinate);
pixelChannels[2] = 255;
rgbaFrame.put(firstCoordinate, secondCoordinate, pixelChannels);
}
}
Mat hsvImage = new Mat();
Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/hsvImage.bmp", hsvImage);// check
Mat maskedImage = new Mat();
Core.inRange(hsvImage, new Scalar(0, 100, 100), new Scalar(10, 255, 255), maskedImage);
Highgui.imwrite("/mnt/sdcard/DCIM/maskedImage.bmp", maskedImage);// check
// Mat dilatedMat = new Mat();
// Imgproc.dilate(maskedImage, dilatedMat, new Mat());
// Highgui.imwrite("/mnt/sdcard/DCIM/dilatedMat.jpg", dilatedMat);// check
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(maskedImage, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
//FINDING THE BIGGEST CONTOUR
// \/ We will use only the largest contour. Other contours (any other possible blobs of this color range) will be ignored.
MatOfPoint largestContour = contours.get(0);
double largestContourArea = Imgproc.contourArea(largestContour);
for (int i = 1; i < contours.size(); ++i) {// NB Notice the prefix increment.
MatOfPoint currentContour = contours.get(i);
double currentContourArea = Imgproc.contourArea(currentContour);
if (currentContourArea > largestContourArea) {
largestContourArea = currentContourArea;
largestContour = currentContour;
}
}
Rect detectedBlobRoi = Imgproc.boundingRect(largestContour);
Mat detectedBlobRgba = rgbaFrame.submat(detectedBlobRoi);
Highgui.imwrite("/mnt/sdcard/DCIM/detectedBlobRgba.bmp", detectedBlobRgba);// check
Mat detectedBlobHsv = new Mat();
Imgproc.cvtColor(detectedBlobRgba, detectedBlobHsv, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/roiHsv.bmp", detectedBlobHsv);// check
List<MatOfPoint> largestContourList = new ArrayList<>();
largestContourList.add(largestContour);
Mat roiWithMask = new Mat(detectedBlobHsv.rows(), detectedBlobHsv.cols(), CvType.CV_8UC3);
roiWithMask.setTo(new Scalar(0,0,0));
Imgproc.drawContours(roiWithMask, largestContourList, 0, new Scalar(0, 255, 255), -1);//TODO Using -1 instead of CV_FILLED.
Highgui.imwrite("/mnt/sdcard/DCIM/roiWithMask.bmp", roiWithMask);// check
// CALCULATING THE AVERAGE COLOR OF THE DETECTED BLOB
// STEP 1:
double [] averageHsvColor = new double[]{0,0,0};
int numOfPixels = 0;
for (int firstCoordinate = 0; firstCoordinate < detectedBlobHsv.rows(); ++firstCoordinate) {
for (int secondCoordinate = 0; secondCoordinate < detectedBlobHsv.cols(); ++secondCoordinate) {
double hue = roiWithMask.get(firstCoordinate, secondCoordinate)[0];
double saturation = roiWithMask.get(firstCoordinate, secondCoordinate)[1];
double value = roiWithMask.get(firstCoordinate, secondCoordinate)[2];
averageHsvColor[0] += hue;
averageHsvColor[1] += saturation;
averageHsvColor[2] += value;
numOfPixels++;
}
}
averageHsvColor[0] /= numOfPixels;
averageHsvColor[1] /= numOfPixels;
averageHsvColor[1] /= numOfPixels;
return new Scalar(averageHsvColor);
}
EDIT 2:
I corrected my 3 channel mask and made a single channel mask
Mat roiMask = new Mat(rgbaFrame.rows(), rgbaFrame.cols(), CvType.CV_8UC1);
roiMask.setTo(new Scalar(0));
Imgproc.drawContours(roiMask, largestContourList, 0, new Scalar(255), -1);
and this resulted in the correct roiMask:
Then, before the comment // CALCULATING THE AVERAGE COLOR OF THE DETECTED BLOB, I added:
Mat newImageWithRoi = new Mat(rgbaFrame.rows(), rgbaFrame.cols(), CvType.CV_8UC3);
newImageWithRoi.setTo(new Scalar(0, 0, 0));
rgbaFrame.copyTo(newImageWithRoi, roiMask);
Highgui.imwrite("/mnt/sdcard/DCIM/newImageWithRoi.bmp", newImageWithRoi);//check
This resulted in:
Now again I don't know how to proceed. :s

You don't need to use pointPolygonTest, because you already have the mask.
You can simply sum up the values that lies on the mask. Something along the lines of (not able to test this):
// Initialize at 0!!!
Scalar averageHsvColor= new Scalar(0,0,0);
int numOfPixels = 0;
for(int r=0; r<detectedBlobHsv.height(); ++r)
{
for(int c=0; c<detectedBlobHsv.width(); ++c)
{
if( /* value of mask(r,c) > 0 */)
{
int H = // get H value of pixel at (r, c)
int S = // get S value of pixel at (r, c)
int V = // get V value of pixel at (r, c)
// Sum values
averageHsvColor[0] += H;
averageHsvColor[1] += S;
averageHsvColor[2] += V;
// Increment number of pixels inside mask
numOfPixels ++;
}
}
}
// Compute average
averageHsvColor[0] /= numOfPixels ;
averageHsvColor[1] /= numOfPixels ;
averageHsvColor[2] /= numOfPixels ;

not able tot detect phisycal door or other big rectangle in open cv android

I wrote code to detect rectangles in open cv. And i am able to detect few object but i am not able to detect physical door or big rectangle. Please check my code and correct me if i am wrong somewhere . Another problem is this code is not able to detect rectangle constantly so when i draw rectangle it comes and go , comes and go and it looks bad. Any way to detect regularly in every frame.
Mat output= getGray(inputFrame.rgba(),inputFrame.rgba());
Imgproc.medianBlur(output, output, 5);
Imgproc.erode(output, output, new Mat());
Imgproc.dilate(output, output, new Mat());
Mat edges = new Mat();
Imgproc.Canny(output, output, 5, 50);
// Vector<MatOfPoint> vector=new Vector<MatOfPoint>();
// Imgproc.findContours(output, points, output, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
contours.clear();
Imgproc.findContours(output, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
MatOfPoint2f approxCurve = new MatOfPoint2f();
rgbImage=inputFrame.rgba();
mDrawnContours.clear();
> Blockquote
for(int i=0;i< contours.size();i++){
MatOfPoint tempContour=contours.get(i);
MatOfPoint2f newMat = new MatOfPoint2f( tempContour.toArray() );
int contourSize = (int)tempContour.total();
Imgproc.approxPolyDP(newMat, approxCurve, contourSize*0.15, true);
MatOfPoint points=new MatOfPoint(approxCurve.toArray());
if((Math.abs(Imgproc.contourArea(tempContour))<100) || !Imgproc.isContourConvex(points)){
Log.i(TAG, "::onCameraFrame:" + " too small");
appendLog("Too small");
continue;
}
else if(points.toArray().length >= 4 && points.toArray().length <= 6){
int vtc = points.toArray().length;
Vector<Double> cosList=new Vector<Double>();
for (int j = 2; j < vtc+1; j++){
cosList.add(angle(points.toArray()[j%vtc], points.toArray()[j-2], points.toArray()[j-1]));
}
double mincos = getMin(cosList);
double maxcos = getMax(cosList);
Log.i(TAG, "::onCameraFrame:" + "mincos:"+mincos+"maxcos:"+maxcos);
if (vtc == 4 && mincos >= -0.1 && maxcos <= 0.3)
{
mTotalSquare++;
Imgproc.drawContours(rgbImage, contours, i, new Scalar(0,0,255));
DrawnContours contours2=new DrawnContours();
contours2.setIndex(i);
mDrawnContours.add(contours2);
Log.i(TAG, "::onCameraFrame:" + "found");
appendLog("found");
}
else{
Log.i(TAG, "::onCameraFrame:" +" not found " +"mincos:"+mincos+"maxcos:"+maxcos);
appendLog("not found 1");
}
}
return rgbImage
Let me know if you have any questions.

I suppose, that large contours have more than 4 edges. Their contour consists of large number of short line segments (depends on approximation function parameter in line
Imgproc.approxPolyDP(newMat, approxCurve, contourSize*0.15, true);
).
And you have condition which check for edge numbers:
points.toArray().length <= 6