Related
i am working on an Android app that will recognize a GO board and create a SGF file of it.
i made a version that is able to detect a board and warp the perspective to make it square ( code and example image below) unfortunately it gets a bit harder when adding stones.(image below)
Important things about a average go board:
round black and white stones
black lines on the board
board color ranges from white to light brown and sometimes with a wood grain
stones are placed on intersections of two lines
correct me if i am wrong but i think my current approach is not a good one.
Has somebody a general idea on how i can separate the stones and lines from the rest of the picture?
My code:
Mat input = inputFrame.rgba(); //original image
Mat gray = new Mat(); //grayscale image
//convert image to grayscale
Imgproc.cvtColor( input, gray, Imgproc.COLOR_RGB2GRAY);
//try to improve histogram (more contrast)
equalizeHist(gray, gray);
//blur image
Size s = new Size(5,5);
GaussianBlur(gray, gray, s, 0);
//apply adaptive treshold
adaptiveThreshold( gray, gray, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY,11,2);
//adding secondary treshold, removes a lot of noise
threshold(gray, gray, 0, 255, Imgproc.THRESH_BINARY + Imgproc.THRESH_OTSU);
Some images:
(source: eightytwo.axc.nl)
(source: eightytwo.axc.nl)
EDIT: 05-03-2016
Yay! managed to detect lines stones and color correctly. precondition the picture has to be only the board itself, without any other background visible.
I use houghLinesP (60lines) and houghCircles (17circles), duration on my phone(1th gen Moto G) about 5 seconds.
Detecting board and warp it turns out to be quite a challenge when it has to be working under different angles and lightning conditions.. still working on that
Suggestions for different approaches are still welcome!!
(source: eightytwo.axc.nl)
EDIT: 15-03-2016
i found a nice way to get line intersects with cross type morphological transformations, works amazing when the picture is taken directly above the board unfortunately not while at an angle (see below)
(source: eightytwo.axc.nl)
In my last update i showed line and stone detection with a picture taken from directly above since then i have been working on detecting the board and warping it in a way that my line and stone detection becomes useful.
harris corner detection
I struggled to get the right parameter settings and i am still not sure if they are optimal, can't find much information on how to optimize image before using harris corners. right now it detects to many corners to be useful. though it feels like it could work. (upper line with pictures in example)
Mat corners = new Mat();
Imgproc.cornerHarris(image, corners, 5, 3, 0.03);
Mat mask = new Mat(corners.size(), CvType.CV_8U, new Scalar(1));
Core.MinMaxLocResult maxVal = Core.minMaxLoc(corners);
Core.inRange(corners, new Scalar(maxVal.maxVal * 0.01), new Scalar(maxVal.maxVal), mask);
cross type morphological transformations
works great when picture is taken directly from above, used from an angle or with a rotated board does not work (middle line with pictures in example)
Imgproc.GaussianBlur(image, image, new Size(5, 5), 0);
Imgproc.adaptiveThreshold(image, image, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY_INV, 11, 2);
int morph_elem = 1; //0: Rect - 1: Cross - 2: Ellipse
int morph_size = 5;
int morph_operator = 0; //0: Opening - 1: Closing \n 2: Gradient - 3: Top Hat \n 4: Black Hat
Mat element = getStructuringElement( morph_elem, new Size(2 * morph_size + 1, 2 * morph_size + 1), new Point( morph_size, morph_size ));
morphologyEx(image, image, morph_operator + 2, element);
contour and houghlines
if there are no stones on the outer boardline and light conditions not to harsh it works pretty well. contours are only part of the board quite often(lower line with pictures in example)
Imgproc.GaussianBlur(image, image, new Size(5, 5), 0);
Imgproc.adaptiveThreshold(image, image, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY_INV, 11, 2);
Mat hierarchy = new Mat();
MatOfPoint biggest = null;
int contourId = 0;
double biggestArea = 0;
double minSize = 2000;
List<MatOfPoint> contours = new ArrayList<>();
findContours(InvertedImage, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
//find biggest
for( int x = 0; x < contours.size() ; x++ ){
double area = Imgproc.contourArea(contours.get(x));
if( area > minSize && area > biggestArea ){
biggestArea = area;
biggest = contours.get(x);
contourId = x;
}
}
providing the right picture all three the methods work but not good enough to be reliable. any thoughts on parameters, image pre-processing, different approaches or anything that might improve the detection are welcome=)
link to picture
EDIT: 31-03-2016
detecting lines and stones is pretty much solved so i will close this question. created a new one for detecting and warping accurately.
anybody interested in my progress: this is my GOSU Snap Alpha channel don't expect to much of it right now!
EDIT: 16-10-2016
Update: i saw some people are still following this question.
I tested some more stuff and started using Tensorflow, my neural network looks promising, you can have a look at it here.
A lot of work has to be done still, my current image dataset is awful and right now i am working on getting a big dataset.
the app works best using a square board with thick lines and decent lightning.
Assuming that you don't want to "force" your end user to take a cleanest pictures (like using an overlay like some of the QR code scanner for example)
Perhaps you could use some morphological transformations with differents kernels :
Opening and closing with a rectangular kernel for the lines
Opening and closing with an ellipse kernel to get the stones (it should be possible to invert the image at some point to get back the white or the black one)
Take a look at http://docs.opencv.org/2.4/doc/tutorials/imgproc/opening_closing_hats/opening_closing_hats.html (sorry this one is in C++ but I think this is almost the same in Java)
I had try these operations to remove a grid from a Sudoku to avoid noise in cell extraction and it worked like a charm.
Let me know of these informations were usefull for you (this is for sure a very interesting case)
I'm working on same program. I avoid finding lines at all.
First use perspective transform to get the board into a square as you have done. Find the edges of the 19x19 grid. Then assuming the board is 19x19 you can just compute the position of the lines. This works well for me. Then you compute the closest intersection of the center of the stone to determine which row and col line the stone is on. Works pretty well for me. Only probably is calibrating program for different lighting conditions and different color stones and boards.
On the website colorizer.org, they have an HSV range of H=0-360, S=0-100, V=0-100. We are also aware that the HSV range in OpenCV is H=0-180, S=0-255, V=0-255.
I wanted to select a range for any shade of (what we perceive as) blue color, so I looked at colorizer.org, and saw that blue Hue ranges roughly from 170 to 270. So I scaled this Hue range to OpenCV by dividing by 2, which gives 85-135.
Now, I took the following screenshot of color [H=216, S=96, V=67] from the preview at the website
Then I run the app on my phone and captured the following camera frame from the laptop screen. I understand that the HSV channel values will differ from those in website to some extent because there are other conditions like additional light (V in HSV) in the room when I captured the camera frame, etc.
Then I converted this Mat to HSV color space by Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_RGB2HSV_FULL);, which resulted in the following image.
Then I called the inRange function:
Core.inRange(hsvImage, new Scalar(85, 50, 40), new Scalar(135, 255, 255), maskedImage);
which resulted in the following maskedImage.
The question is that why isn't it detecting the blue color when I have included all the Hue Range possible for blue color really?
IMPORTANT: Except the first original image, all the images were stored in sdcard using Highgui.imwrite function, so that I could move them to my computer in order to upload them on Stackoverflow. You must have noticed that the blue color in the first original screenshot is converted to red color in the second image. The reason is that the frame captured by the camera (that is the photo/frame of the first screenshot captured by the mobile phone camera) is an RGBA image. But OpenCV converts all images to BRG by default when it saves them to sdcard of something. So be assured that the original image is RGBA, and it is only converted to BGR internally by OpenCV for saving into sdcard. That's why red appears blue.
using this code does work for me (C++):
cv::Mat input = cv::imread("../inputData/HSV_RGB.jpg");
//assuming your image to be in RGB format after loading:
cv::Mat hsv;
cv::cvtColor(input,hsv,CV_RGB2HSV);
// hue range:
cv::Mat mask;
inRange(hsv, cv::Scalar(85, 50, 40), cv::Scalar(135, 255, 255), mask);
cv::imshow("blue mask", mask);
I used this input image (saved and loaded in BGR format although it in fact is a RGB image, that's why we have to use RGB2HSV instead of BGR2HSV):
resulting in this mask:
The difference to your code is that I used CV_RGB2HSV instead of CV_RGB2HSV_FULL. Flag CV_RGB2HSV_FULL uses the whole byte to store the hue values, so range 0 .. 360 degrees will be scaled to 0 .. 255 instead of 0 .. 180 as in CV_RGB2HSV
I could verify this by using this part of the code:
// use _FULL flag:
cv::cvtColor(input,hsv,CV_RGB2HSV_FULL);
// but scale the hue values accordingly:
double hueScale = 2.0/1.41176470588;
cv::Mat mask;
// scale hue values:
inRange(hsv, cv::Scalar(hueScale*85, 50, 40), cv::Scalar(hueScale*135, 255, 255), mask);
giving this result:
For anyone who wants to test with the "right" image:
Here's the input converted to BGR: If you want to use that directly you have to switch conversion from RGB2HSV to BGR2HSV. But I thought it would be better to show the BGR version of the input, too...
I am working in a project where I have to show some portion of the image clear and make rest part of the image blur. The blur should be managed by slider. Means it can be increase or decrease. The final result image should look alike below.
During my research for this I found below links useful
http://blog.neteril.org/blog/2013/08/12/blurring-images-on-android/
https://github.com/kikoso/android-stackblur
http://blog.neteril.org/blog/2013/08/12/blurring-images-on-android/
But the issue in above links is they all make complete image blur. Not some part of image.
Kindly suggest some solution to achieve this. Thanks in advance.
do a masked blur few times ....
create mask
0 means blur (black) and >=1 means not blur (white). Init this part by big enough value for example w=100 pixels
create masked blur function
just a common convolution with some matrix like
0.0 0.1 0.0
0.1 0.6 0.1
0.0 0.1 0.0
but do it only for target pixels where mask is ==0 after image is blurred blur also the mask. This should enlarge the white area a bit (by pixel per iteration but losing magnitude on borders that is why w>1).
loop bullet #2 N times
N determines blur/non-blur gradient depth the w is only to assure that burred mask will grow... Each time the blur mask will increase its white part
That should do the trick, You can also use dilatation of the mask instead of blurring it.
[edit1] implementation
Have played with this a bit today and found out that the mask is not growing enough with smooth so I change the algo a bit (here mine code C++):
picture pic0,pic1,pic2;
// pic0 - source
// pic1 - output
// pic2 - mask
int x0=400,y0=330,r0=100,dr=200;
// x0,y0,r0 - masked area
// dr - blur gradient size
int i,r;
// init output as sourceimage
pic1=pic0;
// init mask (size of source image) with gradient circles
pic2.resize(pic0.xs,pic0.ys);
pic2.clear(0);
for (i=1;i<=255;i++)
{
r=r0+dr-((dr*i)>>8);
pic2.bmp->Canvas->Brush->Color=TColor(i<<16); // shifted because GDI has inverse channel layout then direct pixel access
pic2.bmp->Canvas->Pen ->Color=TColor(i<<16);
pic2.bmp->Canvas->Ellipse(x0-r,y0-r,x0+r,y0+r);
}
for (i=1;i<255;i+=10) pic1.rgb_smooth_masked(pic2,i);
here the smooth function:
//---------------------------------------------------------------------------
void picture::rgb_smooth_masked(const picture &mask,DWORD treshold)
{
int i,x,y;
color *q0,*q1,*m0,c0,c1,c2;
if ((xs<2)||(ys<2)) return;
for (y=0;y<ys-1;y++)
{
q0=p[y ]; m0=mask.p[y];
q1=p[y+1];
for (x=0;x<xs-1;x++)
if (m0[x].dd<treshold)
{
c0=q0[x];
c1=q0[x+1];
c2=q1[x];
for (i=0;i<4;i++)
q0[x].db[i]=DWORD((DWORD(c0.db[i])+DWORD(c0.db[i])+DWORD(c1.db[i])+DWORD(c2.db[i]))>>2);
}
}
}
//---------------------------------------------------------------------------
create gradient mask with circles increasing in color from 1 to 255
rest is black the gradient width is dr and determine the smoothing sharpness.
create smooth masked with mask and threshold
smooth all pixels where mask pixel is < threshold. See the function rgb_smooth_masked. It uses 2x2 convolution matrix
0.50,0.25
0.25,0.00
loop threshold from 1 to 255 by some step
the step determines the image blur strength.
And finally here some visual results this is source image I taken with my camera:
And here the output on the left and mask on the right:
the blue color means values < 256 (B is lowest 8 bits of color)
I use my own picture class for images so some members are:
xs,ys size of image in pixels
p[y][x].dd is pixel at (x,y) position as 32 bit integer type
clear(color) - clears entire image
resize(xs,ys) - resizes image to new resolution
Main idea is to allow user to recolor to specific wall based user selection.
Currently i have implemented this feature using cvFloodFill (helps to prepare mask image) which can help me to change relative HSV value for wall so i can retain edges. but problem with this solution is that it works on color and all walls are repainted instead of single wall selected by user.
i have also tried canny edge detection but it just able to detect edge but not able to convert it to area.
Please find below code which i am currently using for repaint function
Prepare mask
cvFloodFill(mask, new CvPoint(295, 75), new CvScalar(255, 255, 255,0), cvScalarAll(1), cvScalarAll(1), null, 4, null);
split channel
cvSplit(hsvImage, hChannel, sChannel, vChannel, null);
change color
cvAddS(vChannel, new CvScalar(255*(0.76-0.40),0,0,0), vChannel, mask);
How can we detect edges and corresponding area from the image.
i am looking for solution which can be other than opencv but should be possible for iPhone and android
Edit
i am able to achieve somewhat result as below image using below steps
cvCvtColor(image, gray, CV_BGR2GRAY);
cvSmooth(gray,smooth,CV_GAUSSIAN,7,7,0,0);
cvCanny(smooth, canny, 10, 250, 5);
there are two problem with this output not sure how to resolve them
1. close near by edges
2. remove small edges
You could try something like :
Mat imageOut = Mat::zeros(imageIn.rows, imageIn.cols, CV_8UC3);
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
findContours( imageIn, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
for( int idx = 0; idx >= 0; idx = hierarchy[idx][0] )
{
Scalar color( rand()&255, rand()&255, rand()&255 );
drawContours( imageOut, contours, idx, color, CV_FILLED, 8, hierarchy );
}
It should draw the walls in different colors. If it works, that means that in "hierarchy" each wall is identified as a contour, you then will have to find out which one the user selected on his touch screen and do your color tuning processing.
You may have to change the different parameters in "findContours" link.
You will also need to smooth the input image before the contour detection to avoid being annoyed with the details or textures.
Hope that helps,
Thomas
I think I might have the solution for you!
There is a sample file called watershed.cpp in OpenCV, just run it and you'll get this result :
You can make your user draw on his screen to discriminate each wall.
Then if you want something more precise you can outline the areas (without touching other lines) like this :
And TADA! :
With a little work you can make it user-friendly (cancel last line, connect areas etc...)
Hope that helps!
I think you can use Canny Edge Detection algorithm to find edge difference. Some links
StackOverFlow
StackOverFlow
OpenCV QA
OpenCV
Native Tutorial
I hope this can help you out. Thanks.
Here is some OpenCV4Android code to find the largest contour in a Mat called image, which we'll assume is in the RGBA colour space. To find contours, it's first necessary to threshold or binarize the image (convert to black and white). Using a Gaussian Blur on the image before thresholding reduces the number of small contours that are produced. The size parameters to the blur and threshold must be odd numbers; you can play around to find which value gives the best results (here, I've used 7 for both).
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat BW = new Mat();
Mat hierarchy = new Mat();
MatOfPoint largestContour;
Imgproc.cvtColor(image, image, Imgproc.COLOR_RGBA2GRAY); // convert to grayscale
Imgproc.GaussianBlur(image, BW, new Size(7,7), 0);
Imgproc.adaptiveThreshold(BW, BW, 255,
Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY_INV, 7, 2.0);
Imgproc.findContours(BW, contours, hierarchy, Imgproc.RETR_EXTERNAL,
Imgproc.CHAIN_APPROX_SIMPLE);
double maxArea = 0;
for (MatOfPoint contour : contours) {
double area = Imgproc.contourArea(contour);
if (area > maxArea) {
maxArea = area;
largestContour = contour;
}
}
there are two problem with this output not sure how to resolve them 1. close near by edges 2. remove small edges
You can use morphologic operations to close the edges. Look for the dilation and closing operators.
You can remove small edges by doing labeling. Count the number of pixels in each region (connected white pixels). Remove any region with a number of pixels less than some threshold. I don't use opencv, but most libraries have a labeling function that will create an image where each set of touching pixels of a single color are assigned a unique color in the output image.
Please we need help urgently, we are using openCv in Android (Java).
We are facing a lot of problems:
convertTo() doesn't work so we can't convert 3 channel image to 1 channel without passing it on cvtColor().
grayImg.convertTo(grayImg, CvType.CV_8UC1);
cvtColor() gives a weird output:
Imgproc.cvtColor(src, grayImg, Imgproc.COLOR_RGB2GRAY);
Output of this line is the image repeated 4 times!
The only way to get rid of this repetition is to add this line and the output is a white and black image but 3 channel so it crashes any coming function because it needs 1 channel image.
Imgproc.cvtColor(grayImg, grayImg, Imgproc.COLOR_GRAY2RGB,3);
canny() for edge detection:
Imgproc.Canny(grayImg, grayImg, 10, 100,3,true);
findContours() counts a horrible number of contours while number of objects in the image is only 2 input image is 3 channel bmp image and we convert it to Mat.
output image:
https://dl.dropbox.com/u/36214963/canny.jpg
Thanks for your concern
Try BGR2GRAY rather than RGB2GRAY.I had the same problem and I solved it through this.There is also a note in the documentation about this
Converts an image from one color space to another.
The function cvtColor converts an input image from one color space to another. In case of a transformation to-from RGB color space, the order of the channels should be specified explicitly (RGB or BGR). Note that the default color format in OpenCV is often referred to as RGB but it is actually BGR (the bytes are reversed). So the first byte in a standard (24-bit) color image will be an 8-bit Blue component, the second byte will be Green, and the third byte will be Red. The fourth, fifth, and sixth bytes would then be the second pixel (Blue, then Green, then Red), and so on.
If I understand your first question correctly, you have two options to convert RGB images to grayscale ones.
Option 1: Convert the 3 channel image to 1 channel as you are trying to do.
IplImage *RGB_image = cvLoadImage("my_colored_image.jpg");
IplImage *GRAY_IMAGE = cvCreateImage(cvGetSize(RGB_image), IPL_DEPTH_8U, 1);
cvCvtColor(RGB_image, GRAY_IMAGE, CV_RGB2GRAY);
Option 2: Read the colored image as a grayscale image directly.
IplImage* GRAY_IMAGE = cvLoadImage("my_colored_image.jpg", CV_LOAD_IMAGE_GRAYSCALE);
I hope this suits you.
I haven't actually used opencv before, but I don't think convertTo is the answer your looking for.
By looking at the opencv documentation I found this:
cvtColor - Converts an image from one color space to another
Mat color; // the input image
Mat gray(color.rows, color.cols, color.depth());
cvtColor(color, gray, CV_BGR2GRAY);
Or simply (and the function cvtColor will create the image internally):
Mat color;
Mat gray;
cvtColor(color, gray, CV_BGR2GRAY);