Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I want to make a an application like a cam scanner for cropping a document.
But I need same functionality like my two images..
First Images shown image captured by camera..
Second image recognize a captured image part like this..
I research more and more but not getting any out put so, I ask here if,any one done this tell me..
Thanks
I assume your problem is to detect the object to scan.
Object detection mechanisms like pattern matching or feature detection won't bring you the results you are looking for as you don't know what exactly is the object you are scanning.
Basically you search for a rectangular object in the picture.
A basic approach to this could be as following:
Run a canny edge detector on the image. It could help to blur the image a bit before doing this. The edges of the object should be clearly visible.
Now you want to do a Hough transform to find lines in the picture.
Search for lines with an angle around 90deg to each other. The problem would be to find the right ones. Maybe it is enough to use the lines closest to the frame of the picture that are reasonably parallel to them.
Find the intersecting points to define the edges of your object.
At least this should give you a hint where to research further.
As further steps in such an app you will have to calculate the projection of the points and do a affine transform of the object.
I hope this helps.
After writing all this i found this post. It should help you lot.
As my answer targets OpenCV you have to use the OpenCV library.
In Order to do this, you need to install the Android Native Development Kit (NDK).
There are some good tutorials on how to use OpenCV on Android on the OpenCV for Android page.
One thing to keep in mind is that almost each function of the Java wrapper calls a native method. That costs lots of time. So you want to do as much as possible in your native code before returning your results to the Java part.
I know I am too late to answer but it might be helpful to someone.
Try the following code.
#Override
protected void onDraw(Canvas canvas) {
super.onDraw(canvas);
path = new Path();
path.moveTo(x1, y1); // this should set the start point right
//path.lineTo(x1, y1); <-- this line should be drawn at the end of course,sorry
path.lineTo(x2, y2);
path.lineTo(x3, y3);
path.lineTo(x4, y4);
path.lineTo(x1, y1);
canvas.drawPath(path, currentPaint);
}
Pass your image mat in this method:
void findSquares(Mat image, List<MatOfPoint> squares) {
int N = 10;
squares.clear();
Mat smallerImg = new Mat(new Size(image.width() / 2, image.height() / 2), image.type());
Mat gray = new Mat(image.size(), image.type());
Mat gray0 = new Mat(image.size(), CvType.CV_8U);
// down-scale and upscale the image to filter out the noise
Imgproc.pyrDown(image, smallerImg, smallerImg.size());
Imgproc.pyrUp(smallerImg, image, image.size());
// find squares in every color plane of the image
Outer:
for (int c = 0; c < 3; c++) {
extractChannel(image, gray, c);
// try several threshold levels
Inner:
for (int l = 1; l < N; l++) {
Imgproc.threshold(gray, gray0, (l + 1) * 255 / N, 255, Imgproc.THRESH_BINARY);
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
// find contours and store them all as a list
Imgproc.findContours(gray0, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
MatOfPoint approx = new MatOfPoint();
// test each contour
for (int i = 0; i < contours.size(); i++) {
approx = approxPolyDP(contours.get(i), Imgproc.arcLength(new MatOfPoint2f(contours.get(i).toArray()), true) * 0.02, true);
// square contours should have 4 vertices after approximation
// relatively large area (to filter out noisy contours)
// and be convex.
// Note: absolute value of an area is used because
// area may be positive or negative - in accordance with the
// contour orientation
double area = Imgproc.contourArea(approx);
if (area > 5000) {
if (approx.toArray().length == 4 &&
Math.abs(Imgproc.contourArea(approx)) > 1000 &&
Imgproc.isContourConvex(approx)) {
double maxCosine = 0;
Rect bitmap_rect = null;
for (int j = 2; j < 5; j++) {
// find the maximum cosine of the angle between joint edges
double cosine = Math.abs(angle(approx.toArray()[j % 4], approx.toArray()[j - 2], approx.toArray()[j - 1]));
maxCosine = Math.max(maxCosine, cosine);
bitmap_rect = new Rect(approx.toArray()[j % 4], approx.toArray()[j - 2]);
}
// if cosines of all angles are small
// (all angles are ~90 degree) then write quandrange
// vertices to resultant sequence
if (maxCosine < 0.3)
squares.add(approx);
}
}
}
}
}
}
In this method you get four point of document then you can cut this image using below method:
public Bitmap warpDisplayImage(Mat inputMat) {
List<Point> newClockVisePoints = new ArrayList<>();
int resultWidth = inputMat.width();
int resultHeight = inputMat.height();
Mat startM = Converters.vector_Point2f_to_Mat(orderRectCorners(Previes method four poit list(like : List<Point> points)));
Point ocvPOut4 = new Point(0, 0);
Point ocvPOut1 = new Point(0, resultHeight);
Point ocvPOut2 = new Point(resultWidth, resultHeight);
Point ocvPOut3 = new Point(resultWidth, 0);
ocvPOut3 = new Point(0, 0);
ocvPOut4 = new Point(0, resultHeight);
ocvPOut1 = new Point(resultWidth, resultHeight);
ocvPOut2 = new Point(resultWidth, 0);
}
Mat outputMat = new Mat(resultWidth, resultHeight, CvType.CV_8UC4);
List<Point> dest = new ArrayList<Point>();
dest.add(ocvPOut3);
dest.add(ocvPOut2);
dest.add(ocvPOut1);
dest.add(ocvPOut4);
Mat endM = Converters.vector_Point2f_to_Mat(dest);
Mat perspectiveTransform = Imgproc.getPerspectiveTransform(startM, endM);
Imgproc.warpPerspective(inputMat, outputMat, perspectiveTransform, new Size(resultWidth, resultHeight), Imgproc.INTER_CUBIC);
Bitmap descBitmap = Bitmap.createBitmap(outputMat.cols(), outputMat.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(outputMat, descBitmap);
return descBitmap;
}
Related
I want to detect yellow colour objects and plot a centroid position on the largest detected yellow object.
I perform my steps in this order:
converted the input rgbaframe to hsv using cvtColor() method
perform color segmentation in HSV using inRange() method, bound it only to yellow color ranges & returns a binary threshold mask.
I perform morphology operation (specifically MORPH_CLOSE) to perform dilation then erosion on the mask to remove any noise.
I perform a gaussian blur to smooth the mask.
I apply canny algorithm to perform edge detection to make edges more obvious to prepare for Contour detection in the next step. (i'm starting to wonder if this step is of beneficial use at all?)
I apply findContour() algorithm to find the Contours in the image as well as find the hierarchy.
HERE I intend to use feature2d.FeatureDetection(SIMPLEBLOB)& pass in the blob area for detection as Params, however there seems to be no implementation supporting for Android, hence I had to work around the limitation and find largest blob using Imgproc.contourArea().
is there a way to do so?
I pass in the contours obtained previously from the findContours() method as parameter to Imgproc.moments to compute the Centroid Position of the objects detected.
HOWEVER, I would like to bring to everyone's attention that this current implementation will compute all centroids in EACH contour (yellow objects) detected. *PLS SEE/REFER to pic 1, 2 to see what is output onto the Frame back to the user.
What I would like to achieve is to find a way to use the contour of the largestblob (via largestContourArea) and pass that info on as a parameter into the ImgprocMoments() so that I will ONLY COMPUTE the centroid of that largest Contour(Object) detected, so I should only see 1 centroid Pos plotted on the screen at any particular point in time.
I have tried a few methods such as passing the contour of the Largest object as param into Imgproc.moments() but it didn't work either due to difference in data Type / if it worked, the output is not as desired, w multiple centroid points being plotted within or along the perimeter of the object, rather than 1 single point at the center of the largest contour object.
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
InputFrame = inputFrame.rgba();
Core.transpose(InputFrame,mat1); //transpose mat1(src) to mat2(dst), sorta like a Clone!
Imgproc.resize(mat1,mat2,InputFrame.size(),0,0,0); // params:(Mat src, Mat dst, Size dsize, fx, fy, interpolation) Extract the dimensions of the new Screen Orientation, obtain the new orientation's surface width & height. Try to resize to fit to screen.
Core.flip(mat2,InputFrame,-1); // mat3 now get updated, no longer is the Origi inputFrame.rgba BUT RATHER the transposed, resized, flipped version of inputFrame.rgba().
int rowWidth = InputFrame.rows();
int colWidth = InputFrame.cols();
Imgproc.cvtColor(InputFrame,InputFrame,Imgproc.COLOR_RGBA2RGB);
Imgproc.cvtColor(InputFrame,InputFrame,Imgproc.COLOR_RGB2HSV);
Lower_Yellow = new Scalar(21,150,150); //HSV color scale H to adjust color, S to control color variation, V is indicator of amt of light required to be shine on object to be seen.
Upper_Yellow = new Scalar(31,255,360); //HSV color scale
Core.inRange(InputFrame,Lower_Yellow, Upper_Yellow, maskForYellow);
final Size kernelSize = new Size(5, 5); //must be odd num size & greater than 1.
final Point anchor = new Point(-1, -1); //default (-1,-1) means that the anchor is at the center of the structuring element.
final int iterations = 1; //number of times dilation is applied. https://docs.opencv.org/3.4/d4/d76/tutorial_js_morphological_ops.html
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, kernelSize);
Imgproc.morphologyEx(maskForYellow, yellowMaskMorphed, Imgproc.MORPH_CLOSE, kernel, anchor, iterations); //dilate first to remove then erode. White regions becomes more pronounced, erode away black regions
Mat mIntermediateMat = new Mat();
Imgproc.GaussianBlur(yellowMaskMorphed,mIntermediateMat,new Size(9,9),0,0); //better result than kernel size (3,3, maybe cos reference area wider, bigger, can decide better whether inrange / out of range.
Imgproc.Canny(mIntermediateMat, mIntermediateMat, 5, 120); //try adjust threshold //https://stackoverflow.com/questions/25125670/best-value-for-threshold-in-canny
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(mIntermediateMat, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE, new Point(0, 0));
byte[] arr = new byte[100];
//List<double>hierarchyHolder = new ArrayList<>();
int cols = hierarchy.cols();
int rows = hierarchy.rows();
for (int i=0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
//hierarchyHolder.add(hierarchy.get(i,j));
//hierarchy.get(i,j) is a double[] type, not byte.
Log.d("hierarchy"," " + hierarchy.get(i,j).toString());
}
}
double maxArea1 = 0;
int maxAreaIndex1 = 0;
//MatOfPoint max_contours = new MatOfPoint();
Rect r = null;
ArrayList<Rect> rect_array = new ArrayList<Rect>();
for(int i=0; i < contours.size(); i++) {
//if(Imgproc.contourArea(contours.get(i)) > 300) { //Size of Mat contour # that particular point in ArrayList of Points.
double contourArea1 = Imgproc.contourArea(contours.get(i));
//Size of Mat contour # that particular point in ArrayList of Points.
if (maxArea1 < contourArea1){
maxArea1 = contourArea1;
maxAreaIndex1 = i;
}
//maxArea1 = Imgproc.contourArea(contours.get(i)); //assigned but nvr used
//max_contours = contours.get(i);
r = Imgproc.boundingRect(contours.get(maxAreaIndex1));
rect_array.add(r); //will only have 1 r in the array eventually, cos we will only take the one w largestContourArea.
}
Imgproc.cvtColor(InputFrame, InputFrame, Imgproc.COLOR_HSV2RGB);
if (rect_array.size() > 0) { //if got more than 1 rect found in rect_array, draw them out!
Iterator<Rect> it2 = rect_array.iterator(); //only got 1 though, this method much faster than drawContour, wont lag. =D
while (it2.hasNext()) {
Rect obj = it2.next();
//if
Imgproc.rectangle(InputFrame, obj.br(), obj.tl(),
new Scalar(0, 255, 0), 1);
}
}
//========= Compute CENTROID POS! WHAT WE WANT TO SHOW ON SCREEN EVENTUALLY!======================
List<Moments> mu = new ArrayList<>(contours.size()); //HUMoments
for (int i = 0; i < contours.size(); i++) {
mu.add(Imgproc.moments(contours.get(i)));
}
List<Point> mc = new ArrayList<>(contours.size()); //the Circle centre Point!
for (int i = 0; i < contours.size(); i++) {
//add 1e-5 to avoid division by zero
mc.add(new Point(mu.get(i).m10 / (mu.get(i).m00 + 1e-5), mu.get(i).m01 / (mu.get(i).m00 + 1e-5)));
}
for (int i = 0; i < contours.size(); i++) {
Scalar color = new Scalar(150, 150, 150);
Imgproc.circle(InputFrame, mc.get(i), 20, color, -1); //just to plot the small central point as a dot on the detected ImgObject.
}
Picture of output when view on CameraFrame:
1:
2:
Managed to resolve the issue, instead of looping thru the entire array of Contours for yellow objects detected, and pass each contour as parameter to Imgproc.moments, I now only assign the Contour at the particular index which the LargestContour is detected, so only 1 single contour is being processed now by Imgproc.moments to compute Centroid! The code correction is as shown below
//========= Compute CENTROID POS! WHAT WE WANT TO SHOW ON SCREEN EVENTUALLY!======================
List<Moments> mu = new ArrayList<>(contours.size());
mu.add(Imgproc.moments(contours.get(maxAreaContourIndex1))); //Just adding that 1 Single Largest Contour (largest ContourArea) to arryalist to be computed for MOMENTS to compute CENTROID POS!
List<Point> mc = new ArrayList<>(contours.size()); //the Circle centre Point!
//add 1e-5 to avoid division by zero
mc.add(new Point(mu.get(0).m10 / (mu.get(0).m00 + 1e-5), mu.get(0).m01 / (mu.get(0).m00 + 1e-5))); //index 0 cos there shld only be 1 contour now, the largest one only!
//notice that it only adds 1 point, the centroid point. Hence only 1 point in the mc list<Point>, so ltr reference that point w an index 0!
Scalar color = new Scalar(150, 150, 150);
Imgproc.circle(InputFrame, mc.get(0), 15, color, -1); //just to plot the small central point as a dot on the detected ImgObject.
I'm trying to overlay on the live frame a little image previosuly selected by the user. I already have the image's path from a previous activity. My problem is that I am not able to show the image on the frame.
I'm trying to detect a rectangle on the frame, and over the rectangle display the image selected. I could detect the rectangle, but now i can't display the image on any part of the frame (I don't care about the rectangle right now).
I've been trying to do it with the explanations from Adding Image Overlay OpenCV for Android and add watermark small image to large image opencv4android but it didn't worked for me.
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
Mat gray = inputFrame.gray();
Mat dst = inputFrame.rgba();
Imgproc.pyrDown(gray, dsIMG, new Size(gray.cols() / 2, gray.rows() / 2));
Imgproc.pyrUp(dsIMG, usIMG, gray.size());
Imgproc.Canny(usIMG, bwIMG, 0, threshold);
Imgproc.dilate(bwIMG, bwIMG, new Mat(), new Point(-1, 1), 1);
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
cIMG = bwIMG.clone();
Imgproc.findContours(cIMG, contours, hovIMG, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
for (MatOfPoint cnt : contours) {
MatOfPoint2f curve = new MatOfPoint2f(cnt.toArray());
Imgproc.approxPolyDP(curve, approxCurve, 0.02 * Imgproc.arcLength(curve, true), true);
int numberVertices = (int) approxCurve.total();
double contourArea = Imgproc.contourArea(cnt);
if (Math.abs(contourArea) < 100) {
continue;
}
//Rectangle detected
if (numberVertices >= 4 && numberVertices <= 6) {
List<Double> cos = new ArrayList<>();
for (int j = 2; j < numberVertices + 1; j++) {
cos.add(angle(approxCurve.toArray()[j % numberVertices], approxCurve.toArray()[j - 2], approxCurve.toArray()[j - 1]));
}
Collections.sort(cos);
double mincos = cos.get(0);
double maxcos = cos.get(cos.size() - 1);
if (numberVertices == 4 && mincos >= -0.3 && maxcos <= 0.5) {
//Small watermark image
Mat a = imread(img_path);
Mat bSubmat = dst.submat(0, dst.rows() -1 , 0, dst.cols()-1);
a.copyTo(bSubmat);
}
}
}
return dst;
}
NOTE: img_path is the path of the selected image I want to display over the frame. I got it from the previous activity.
By now, I just want to display the image over the frame. Later, I will try to display it on the same position where it found the rectangle.
Please, any suggestion or recommendation is welcome, as I am new with OpenCV. I'm sorry for my english, but feel free to ask me anything I didn't explain correctly. I'll do my best to explain it better.
Thanks a lot!
If you just want to display the image as an overlay, and not save it as part of the video, you may find it easier to simple display it in a separate view above the video view. This will likely use less processing, battery etc also.
If you want to draw onto the camera image bitmap then the following will allow you do that:
Bitmap cameraBitmap = BitmapFactory.decodeByteArray(bytes,0,bytes.length, opt);
Canvas camImgCanvas = new Canvas(cameraBitmap);
Drawable d = ContextCompat.getDrawable(getActivity(), R.drawable.myDrawable);
//Centre the drawing
int bitMapWidthCenter = cameraBitmap.getWidth()/2;
int bitMapheightCenter = cameraBitmap.getHeight()/2;
d.setBounds(bitMapWidthCenter, bitMapheightCenter, bitMapWidthCenter+d.getIntrinsicWidth(),
bitMapheightCenter+d.getIntrinsicHeight());
//And draw it...
d.draw(camImgCanvas);
I have tried 2 methodologies as follows:-
conversion of image to Mat
apply gaussian blur
then canny edge detection
find contours
The problem with this method is:
too many contours are detected
mostly open contours
doesn't detect what I want to detect
Then I changed my approach and tried adaptive thresholding after gaussian blur/median blur and it is much better and I am able to detect the corners in 50% cases
The current problem I am facing is that the page detection requires contrasting and plain background without any reflections. I think it's too idealistic for real world use.
This is where I would like some help. Even a direction towards the solution is highly appreciated especially in java. Thanks in anticipation
works absolutely fine with a significant contrasting background like this
Detected 4 corners
This picture gives troubles because the background isn't exactly the most contrasting
Initial largest contour found
Update: median blur did not help much so I traced the cause and found that the page boundary was detected in bits and pieces and not a single contour so it detected the biggest contour as a part of the page boundary Therefore performed some morphological operations to close relatively small gaps and the resultant largest contour is definitely improved but its its not optimum. Any ideas how I can improve the big gaps?
morphed original picture
largest contour found in the morphed image
PS morphing the image in ideal scenarios has led to detection of false contour boundaries. Any condition which can be checked before morphing an image is also a bonus. Thank you
If you use methods like that:
public static RotatedRect getBestRectByArea(List<RotatedRect> boundingRects) {
RotatedRect bestRect = null;
if (boundingRects.size() >= 1) {
RotatedRect boundingRect;
Point[] vertices = new Point[4];
Rect rect;
double maxArea;
int ixMaxArea = 0;
// find best rect by area
boundingRect = boundingRects.get(ixMaxArea);
boundingRect.points(vertices);
rect = Imgproc.boundingRect(new MatOfPoint(vertices));
maxArea = rect.area();
for (int ix = 1; ix < boundingRects.size(); ix++) {
boundingRect = boundingRects.get(ix);
boundingRect.points(vertices);
rect = Imgproc.boundingRect(new MatOfPoint(vertices));
if (rect.area() > maxArea) {
maxArea = rect.area();
ixMaxArea = ix;
}
}
bestRect = boundingRects.get(ixMaxArea);
}
return bestRect;
}
private static Bitmap findROI(Bitmap sourceBitmap) {
Bitmap roiBitmap = Bitmap.createBitmap(sourceBitmap.getWidth(), sourceBitmap.getHeight(), Bitmap.Config.ARGB_8888);
Mat sourceMat = new Mat(sourceBitmap.getWidth(), sourceBitmap.getHeight(), CV_8UC3);
Utils.bitmapToMat(sourceBitmap, sourceMat);
final Mat mat = new Mat();
sourceMat.copyTo(mat);
Imgproc.cvtColor(mat, mat, Imgproc.COLOR_RGB2GRAY);
Imgproc.threshold(mat, mat, 146, 250, Imgproc.THRESH_BINARY);
// find contours
List<MatOfPoint> contours = new ArrayList<>();
List<RotatedRect> boundingRects = new ArrayList<>();
Imgproc.findContours(mat, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// find appropriate bounding rectangles
for (MatOfPoint contour : contours) {
MatOfPoint2f areaPoints = new MatOfPoint2f(contour.toArray());
RotatedRect boundingRect = Imgproc.minAreaRect(areaPoints);
boundingRects.add(boundingRect);
}
RotatedRect documentRect = getBestRectByArea(boundingRects);
if (documentRect != null) {
Point rect_points[] = new Point[4];
documentRect.points(rect_points);
for (int i = 0; i < 4; ++i) {
Imgproc.line(sourceMat, rect_points[i], rect_points[(i + 1) % 4], ROI_COLOR, ROI_WIDTH);
}
}
Utils.matToBitmap(sourceMat, roiBitmap);
return roiBitmap;
}
you can achieve for your source images results like this:
or that:
If you adjust threshold values and apply filters you can achieve even better results.
You can pick a single contour by using one or both of:
Use BoundingRect and ContourArea to evaluate the squareness of each contour. boundingRect() returns orthogonal rects., to handle arbitrary rotation better use minAreaRect() which returns optimally rotated ones.
Use Cv.ApproxPoly iteratively to reduce to a 4 sided shape
var approxIter = 1;
while (true)
{
var approxCurve = Cv.ApproxPoly(largestContour, 0, null, ApproxPolyMethod.DP, approxIter, true);
var approxCurvePointsTmp = new[] { approxCurve.Select(p => new CvPoint2D32f((int)p.Value.X, (int)p.Value.Y)).ToArray() }.ToArray();
if (approxCurvePointsTmp[0].Length == 4)
{
corners = approxCurvePointsTmp[0];
break;
}
else if (approxCurvePointsTmp[0].Length < 4) throw new InvalidOperationException("Failed to decimate corner points");
approxIter++;
}
However neither of these will help if the contour detection gives you two separate contours due to noise / contrast.
I think it would be possible to use the hough line transformation to help detect cases where a line has been split into two contours.
If so the search could be repeated for all combinations of joined contours to see if a bigger / more rectangular match is found.
Stop relying on edge detection, the worst methodology in the universe, and switch to some form of image segmentation.
The paper is white, the background is contrasted, this is the information that you should use.
This might have been answered but I desperately need an answer for this. I want to find the largest square or rectangle in an image using OpenCV in Android. All of the solutions that I found are C++ and I tried converting it but it doesn't work and I do not know where I'm wrong.
private Mat findLargestRectangle(Mat original_image) {
Mat imgSource = original_image;
Imgproc.cvtColor(imgSource, imgSource, Imgproc.COLOR_BGR2GRAY);
Imgproc.Canny(imgSource, imgSource, 100, 100);
//I don't know what to do in here
return imgSource;
}
What I am trying to accomplish in here is to create a new image that is based on the largest square found in the original image (return value Mat image).
This is what I want to happen:
1 http://img14.imageshack.us/img14/7855/s7zr.jpg
It's also okay that I just get the four points of the largest square and I think I can take it from there. But it would be better if I can just return the cropped image.
After canny
1- you need to reduce noises with gaussian blur and find all the contours
2- find and list all the contours' areas.
3- the largest contour will be nothing but the painting.
4- now use perpective transformation to transform your shape to a rectangle.
check sudoku solver examples to see the similar processing problem. (largest contour + perspective)
Took me a while to convert the C++ code to Java, but here it is :-)
Warning ! Raw code, totally not optimized and all.
I decline any liability in cases of injury or lethal accident
List<MatOfPoint> squares = new ArrayList<MatOfPoint>();
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
if (Math.random()>0.80) {
findSquares(inputFrame.rgba().clone(),squares);
}
Mat image = inputFrame.rgba();
Imgproc.drawContours(image, squares, -1, new Scalar(0,0,255));
return image;
}
int thresh = 50, N = 11;
// helper function:
// finds a cosine of angle between vectors
// from pt0->pt1 and from pt0->pt2
double angle( Point pt1, Point pt2, Point pt0 ) {
double dx1 = pt1.x - pt0.x;
double dy1 = pt1.y - pt0.y;
double dx2 = pt2.x - pt0.x;
double dy2 = pt2.y - pt0.y;
return (dx1*dx2 + dy1*dy2)/Math.sqrt((dx1*dx1 + dy1*dy1)*(dx2*dx2 + dy2*dy2) + 1e-10);
}
// returns sequence of squares detected on the image.
// the sequence is stored in the specified memory storage
void findSquares( Mat image, List<MatOfPoint> squares )
{
squares.clear();
Mat smallerImg=new Mat(new Size(image.width()/2, image.height()/2),image.type());
Mat gray=new Mat(image.size(),image.type());
Mat gray0=new Mat(image.size(),CvType.CV_8U);
// down-scale and upscale the image to filter out the noise
Imgproc.pyrDown(image, smallerImg, smallerImg.size());
Imgproc.pyrUp(smallerImg, image, image.size());
// find squares in every color plane of the image
for( int c = 0; c < 3; c++ )
{
extractChannel(image, gray, c);
// try several threshold levels
for( int l = 1; l < N; l++ )
{
//Cany removed... Didn't work so well
Imgproc.threshold(gray, gray0, (l+1)*255/N, 255, Imgproc.THRESH_BINARY);
List<MatOfPoint> contours=new ArrayList<MatOfPoint>();
// find contours and store them all as a list
Imgproc.findContours(gray0, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
MatOfPoint approx=new MatOfPoint();
// test each contour
for( int i = 0; i < contours.size(); i++ )
{
// approximate contour with accuracy proportional
// to the contour perimeter
approx = approxPolyDP(contours.get(i), Imgproc.arcLength(new MatOfPoint2f(contours.get(i).toArray()), true)*0.02, true);
// square contours should have 4 vertices after approximation
// relatively large area (to filter out noisy contours)
// and be convex.
// Note: absolute value of an area is used because
// area may be positive or negative - in accordance with the
// contour orientation
if( approx.toArray().length == 4 &&
Math.abs(Imgproc.contourArea(approx)) > 1000 &&
Imgproc.isContourConvex(approx) )
{
double maxCosine = 0;
for( int j = 2; j < 5; j++ )
{
// find the maximum cosine of the angle between joint edges
double cosine = Math.abs(angle(approx.toArray()[j%4], approx.toArray()[j-2], approx.toArray()[j-1]));
maxCosine = Math.max(maxCosine, cosine);
}
// if cosines of all angles are small
// (all angles are ~90 degree) then write quandrange
// vertices to resultant sequence
if( maxCosine < 0.3 )
squares.add(approx);
}
}
}
}
}
void extractChannel(Mat source, Mat out, int channelNum) {
List<Mat> sourceChannels=new ArrayList<Mat>();
List<Mat> outChannel=new ArrayList<Mat>();
Core.split(source, sourceChannels);
outChannel.add(new Mat(sourceChannels.get(0).size(),sourceChannels.get(0).type()));
Core.mixChannels(sourceChannels, outChannel, new MatOfInt(channelNum,0));
Core.merge(outChannel, out);
}
MatOfPoint approxPolyDP(MatOfPoint curve, double epsilon, boolean closed) {
MatOfPoint2f tempMat=new MatOfPoint2f();
Imgproc.approxPolyDP(new MatOfPoint2f(curve.toArray()), tempMat, epsilon, closed);
return new MatOfPoint(tempMat.toArray());
}
There are some related questions here in SO. Check them out:
OpenCV C++/Obj-C: Detecting a sheet of paper / Square Detection
How do I recognize squares in this image?
There is also an example shipped with OpenCV:
https://code.ros.org/trac/opencv/browser/trunk/opencv/samples/cpp/squares.cpp?rev=4079
Once you have the rectangle, you can align the picture by computing the homography with the rectangle corners and applying a perspective transform.
This might have been answered but I desperately need an answer for this. I want to find the largest square or rectangle in an image using OpenCV in Android. All of the solutions that I found are C++ and I tried converting it but it doesn't work and I do not know where I'm wrong.
private Mat findLargestRectangle(Mat original_image) {
Mat imgSource = original_image;
Imgproc.cvtColor(imgSource, imgSource, Imgproc.COLOR_BGR2GRAY);
Imgproc.Canny(imgSource, imgSource, 100, 100);
//I don't know what to do in here
return imgSource;
}
What I am trying to accomplish in here is to create a new image that is based on the largest square found in the original image (return value Mat image).
This is what I want to happen:
1 http://img14.imageshack.us/img14/7855/s7zr.jpg
It's also okay that I just get the four points of the largest square and I think I can take it from there. But it would be better if I can just return the cropped image.
After canny
1- you need to reduce noises with gaussian blur and find all the contours
2- find and list all the contours' areas.
3- the largest contour will be nothing but the painting.
4- now use perpective transformation to transform your shape to a rectangle.
check sudoku solver examples to see the similar processing problem. (largest contour + perspective)
Took me a while to convert the C++ code to Java, but here it is :-)
Warning ! Raw code, totally not optimized and all.
I decline any liability in cases of injury or lethal accident
List<MatOfPoint> squares = new ArrayList<MatOfPoint>();
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
if (Math.random()>0.80) {
findSquares(inputFrame.rgba().clone(),squares);
}
Mat image = inputFrame.rgba();
Imgproc.drawContours(image, squares, -1, new Scalar(0,0,255));
return image;
}
int thresh = 50, N = 11;
// helper function:
// finds a cosine of angle between vectors
// from pt0->pt1 and from pt0->pt2
double angle( Point pt1, Point pt2, Point pt0 ) {
double dx1 = pt1.x - pt0.x;
double dy1 = pt1.y - pt0.y;
double dx2 = pt2.x - pt0.x;
double dy2 = pt2.y - pt0.y;
return (dx1*dx2 + dy1*dy2)/Math.sqrt((dx1*dx1 + dy1*dy1)*(dx2*dx2 + dy2*dy2) + 1e-10);
}
// returns sequence of squares detected on the image.
// the sequence is stored in the specified memory storage
void findSquares( Mat image, List<MatOfPoint> squares )
{
squares.clear();
Mat smallerImg=new Mat(new Size(image.width()/2, image.height()/2),image.type());
Mat gray=new Mat(image.size(),image.type());
Mat gray0=new Mat(image.size(),CvType.CV_8U);
// down-scale and upscale the image to filter out the noise
Imgproc.pyrDown(image, smallerImg, smallerImg.size());
Imgproc.pyrUp(smallerImg, image, image.size());
// find squares in every color plane of the image
for( int c = 0; c < 3; c++ )
{
extractChannel(image, gray, c);
// try several threshold levels
for( int l = 1; l < N; l++ )
{
//Cany removed... Didn't work so well
Imgproc.threshold(gray, gray0, (l+1)*255/N, 255, Imgproc.THRESH_BINARY);
List<MatOfPoint> contours=new ArrayList<MatOfPoint>();
// find contours and store them all as a list
Imgproc.findContours(gray0, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
MatOfPoint approx=new MatOfPoint();
// test each contour
for( int i = 0; i < contours.size(); i++ )
{
// approximate contour with accuracy proportional
// to the contour perimeter
approx = approxPolyDP(contours.get(i), Imgproc.arcLength(new MatOfPoint2f(contours.get(i).toArray()), true)*0.02, true);
// square contours should have 4 vertices after approximation
// relatively large area (to filter out noisy contours)
// and be convex.
// Note: absolute value of an area is used because
// area may be positive or negative - in accordance with the
// contour orientation
if( approx.toArray().length == 4 &&
Math.abs(Imgproc.contourArea(approx)) > 1000 &&
Imgproc.isContourConvex(approx) )
{
double maxCosine = 0;
for( int j = 2; j < 5; j++ )
{
// find the maximum cosine of the angle between joint edges
double cosine = Math.abs(angle(approx.toArray()[j%4], approx.toArray()[j-2], approx.toArray()[j-1]));
maxCosine = Math.max(maxCosine, cosine);
}
// if cosines of all angles are small
// (all angles are ~90 degree) then write quandrange
// vertices to resultant sequence
if( maxCosine < 0.3 )
squares.add(approx);
}
}
}
}
}
void extractChannel(Mat source, Mat out, int channelNum) {
List<Mat> sourceChannels=new ArrayList<Mat>();
List<Mat> outChannel=new ArrayList<Mat>();
Core.split(source, sourceChannels);
outChannel.add(new Mat(sourceChannels.get(0).size(),sourceChannels.get(0).type()));
Core.mixChannels(sourceChannels, outChannel, new MatOfInt(channelNum,0));
Core.merge(outChannel, out);
}
MatOfPoint approxPolyDP(MatOfPoint curve, double epsilon, boolean closed) {
MatOfPoint2f tempMat=new MatOfPoint2f();
Imgproc.approxPolyDP(new MatOfPoint2f(curve.toArray()), tempMat, epsilon, closed);
return new MatOfPoint(tempMat.toArray());
}
There are some related questions here in SO. Check them out:
OpenCV C++/Obj-C: Detecting a sheet of paper / Square Detection
How do I recognize squares in this image?
There is also an example shipped with OpenCV:
https://code.ros.org/trac/opencv/browser/trunk/opencv/samples/cpp/squares.cpp?rev=4079
Once you have the rectangle, you can align the picture by computing the homography with the rectangle corners and applying a perspective transform.