Develop Reference

Edge detection on colored background using OpenCV

I am using following code to detect edges from given document.
private Mat edgeDetection(Mat src) {
Mat edges = new Mat();
Imgproc.cvtColor(src, edges, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur(edges, edges, new Size(5, 5), 0);
Imgproc.Canny(edges, edges, 10, 30);
return edges;
}
And then I can find the document from this edges by finding largest contour from this.
My problem is I can find the document from following pic:
but not from following pic:
How can I improve this edge detection?

I use Python, but the main idea is the same.
If you directly do cvtColor: bgr -> gray for img2, then you must fail. Because the gray becames difficulty to distinguish the regions:
Related answers:
How to detect colored patches in an image using OpenCV?
Edge detection on colored background using OpenCV
OpenCV C++/Obj-C: Detecting a sheet of paper / Square Detection
In your image, the paper is white, while the background is colored. So, it's better to detect the paper is Saturation(饱和度) channel in HSV color space. For HSV, refer to https://en.wikipedia.org/wiki/HSL_and_HSV#Saturation.
Main steps:
Read into BGR
Convert the image from bgr to hsv space
Threshold the S channel
Then find the max external contour(or do Canny, or HoughLines as you like, I choose findContours), approx to get the corners.
This is the first result:
This is the second result:
The Python code(Python 3.5 + OpenCV 3.3):
#!/usr/bin/python3
# 2017.12.20 10:47:28 CST
# 2017.12.20 11:29:30 CST
import cv2
import numpy as np
##(1) read into bgr-space
img = cv2.imread("test2.jpg")
##(2) convert to hsv-space, then split the channels
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)
##(3) threshold the S channel using adaptive method(`THRESH_OTSU`) or fixed thresh
th, threshed = cv2.threshold(s, 50, 255, cv2.THRESH_BINARY_INV)
##(4) find all the external contours on the threshed S
cnts = cv2.findContours(threshed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]
canvas = img.copy()
#cv2.drawContours(canvas, cnts, -1, (0,255,0), 1)
## sort and choose the largest contour
cnts = sorted(cnts, key = cv2.contourArea)
cnt = cnts[-1]
## approx the contour, so the get the corner points
arclen = cv2.arcLength(cnt, True)
approx = cv2.approxPolyDP(cnt, 0.02* arclen, True)
cv2.drawContours(canvas, [cnt], -1, (255,0,0), 1, cv2.LINE_AA)
cv2.drawContours(canvas, [approx], -1, (0, 0, 255), 1, cv2.LINE_AA)
## Ok, you can see the result as tag(6)
cv2.imwrite("detected.png", canvas)

In OpenCV there is function called dilate this will darker the lines. so try the code like below.
private Mat edgeDetection(Mat src) {
Mat edges = new Mat();
Imgproc.cvtColor(src, edges, Imgproc.COLOR_BGR2GRAY);
Imgproc.dilate(edges, edges, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(10, 10)));
Imgproc.GaussianBlur(edges, edges, new Size(5, 5), 0);
Imgproc.Canny(edges, edges, 15, 15 * 3);
return edges;
}

Manipulating the black and white pixels in Android bitmap

I am developing an app for clicked an image and convert the bitmap to binary. Here is some of my code of converting the image to binary:
File file = new File(Environment.getExternalStorageDirectory().getAbsolutePath()+"/Watershed/WatershedImages", "Image_"+Save.MyDate+".jpg");
Mat origianlImage = Imgcodecs.imread(file.getAbsolutePath());
Mat image1Mat = new Mat();
Imgproc.cvtColor(origianlImage,image1Mat, Imgproc.COLOR_BGR2HSV);
Mat mat3 = new Mat();
Core.inRange(image1Mat, new Scalar(0, 0, 0), new Scalar(50, 255, 255), mat3);
Mat mat4 = new Mat();
Imgproc.threshold(mat3, mat4, 0, 255, Imgproc.THRESH_BINARY);
Bitmap threshBitmap = Bitmap.createBitmap(mat4.cols(), mat4.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mat4, threshBitmap);
The image is converted to bitmap. I have checked the color of pixels and found out that black pixel value is -16777216 and white is -1. For further processing, I need to change these values to original java type (0 for black and 255 for white).
How to manipulate the pixel values from -16777216 to 0 and -1 to 255 WITHOUT changing the colors? Is there any way? Please help.

How to set transparent background to grabcut output image with open cv?

> Hi, I am using OpenCV android library grabcut() method to extract an image from background, but the problem is that the output bitmap contains background Same as original image and object become white .I need Object as its same as original image and background transparent
I am using this code
private static Bitmap makeBlackTransparent(Bitmap image) {
// convert image to matrix
Mat src = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
Utils.bitmapToMat(image, src);
// init new matrices
Mat dst = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
Mat tmp = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
Mat alpha = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
// convert image to grayscale
Imgproc.cvtColor(src, tmp, Imgproc.COLOR_BGR2GRAY);
// threshold the image to create alpha channel with complete transparency in black background region and zero transparency in foreground object region.
Imgproc.threshold(tmp, alpha, 100, 255, Imgproc.THRESH_BINARY);
// split the original image into three single channel.
List<Mat> rgb = new ArrayList<Mat>(3);
Core.split(src, rgb);
// Create the final result by merging three single channel and alpha(BGRA order)
List<Mat> rgba = new ArrayList<Mat>(4);
rgba.add(rgb.get(0));
rgba.add(rgb.get(1));
rgba.add(rgb.get(2));
rgba.add(alpha);
Core.merge(rgba, dst);
// convert matrix to output bitmap
Bitmap output = Bitmap.createBitmap(image.getWidth(), image.getHeight(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(dst, output);
return output;
}

There are 2 problems in your code:
Firstly you need to segment out the white background, so adjust your thresh to be near 220 - 240 and also use THRESH_BINARY_INV instead of THRESH_BINARY :
Imgproc.threshold(tmp, alpha, 230, 255, Imgproc.THRESH_BINARY_INV);
Secondly, you must pre-multiply the ARGB layers, as Android ImageView behaves weird without premultiplication, for that you need to use cvtColor with COLOR_RGBA2mRGBA flag:
// Create the final result by merging three single channel and alpha(BGRA order)
List<Mat> rgba = new ArrayList<Mat>(4);
rgba.add(rgb.get(0));
rgba.add(rgb.get(1));
rgba.add(rgb.get(2));
rgba.add(alpha);
Core.merge(rgba, dst);
Imgproc.cvtColor(dst, dst, Imgproc.COLOR_RGBA2mRGBA);

Why is my copyTo function Not copying the mask to the correct Mat?

I started with the following image, named rgbaMat4Mask.bmp:
Then I converted it to HSV, and then did inRange() to find contours, and got the following Mat named maskedMat:
Then I went on to draw the first contour (the bigger one), on a newly created empty Mat named newMatWithMask, which has been given the same size as that of the first image I started with:
So far so good, but the problem starts now. I created a new Mat and gave it the same size as that of the first contour (the bigger one), and then set its background color to new Scalar(120, 255, 255). Then I copied the newMat4MaskFinished to it using copyTo function. But neither is the size of the resulting Mat same as that of the contour, nor is its background color set to new Scalar(120, 255, 255) which is blue.
It is rather an image with size same as that of the entire mask, and has a black background. why? What am I doing wrong?
public void doProcessing(View view) {
// READING THE RGBA MAT
Mat rgbaMat4Mask = Highgui.imread("/mnt/sdcard/DCIM/rgbaMat4Mask.bmp");
// CONVERTING TO HSV
Mat hsvMat4Mask = new Mat();
Imgproc.cvtColor(rgbaMat4Mask, hsvMat4Mask, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/hsvMat4Mask.bmp", hsvMat4Mask);//check
// CREATING A FILTER/MASK FOR RED COLORED BLOB
Mat maskedMat = new Mat();
Core.inRange(hsvMat4Mask, new Scalar(0, 100, 100), new Scalar(10, 255, 255), maskedMat);
Highgui.imwrite("/mnt/sdcard/DCIM/maskedMat.bmp", maskedMat);// check
// COPYING THE MASK TO AN EMPTY MAT
// STEP 1:
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(maskedMat, contours, new Mat(), Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);
//STEP 2:
Mat newMat4Mask = new Mat(rgbaMat4Mask.rows(), rgbaMat4Mask.cols(), CvType.CV_8UC1);
newMat4Mask.setTo(new Scalar(0));
Imgproc.drawContours(newMat4Mask, contours, 0, new Scalar(255), -1);//TODO Using -1 instead of CV_FILLED.
Highgui.imwrite("/mnt/sdcard/DCIM/newMatWithMask.bmp", newMat4Mask);// check
//STEP 3
Log.i(TAG, "HAPPY rows:"+contours.get(0).rows()+" columns:"+contours.get(0).cols());
Mat newMatwithMaskFinished = new Mat(contours.get(0).rows(), contours.get(0).cols(), CvType.CV_8UC3);
newMatwithMaskFinished.setTo(new Scalar(120, 255, 255));
rgbaMat4Mask.copyTo(newMatwithMaskFinished, newMat4Mask);
Highgui.imwrite("/mnt/sdcard/DCIM/newMatwithMaskFinished.bmp", newMatwithMaskFinished);//check*/
}

Your newMatwithMaskFinished should have the same size as rgbaMat4Mask and newMat4Mask.
Mat newMatwithMaskFinished = new Mat(rgbaMat4Mask.rows(), rgbaMat4Mask.cols(), CvType.CV_8UC3);
If you want to have a Mat of the bigger circle only, with transparent background, then you need to:
1) create newMatwithMaskFinished with type CV_8UC4
Mat newMatwithMaskFinished = new Mat(rgbaMat4Mask.rows(), rgbaMat4Mask.cols(), CvType.CV_8UC4);
2) set a transparent background:
newMatwithMaskFinished.setTo(new Scalar(0, 0, 0, 0));
3) Compute the bounding box box of the contour you're interested in, with boundingRect.
4) Convert rgbaMat4Mask to 4 channels (unless it's already), with cvtColor(..., COLOR_BGR2BGRA), let's call this rgba
5) Copy rgba to newMatwithMaskFinished, with mask newMat4Mask.
6) Crop newMatwithMaskFinished on box, using submat method

Using OpenCV, how to find the values of HSV channels of the "masked region" in a masked image?

Using OpenCV4Android, how can I get the HSV channels of the first pixel of the masked region in a masked image (dilatedMat in the following snippet)? I know that we'd get the HSV channel values of first pixel by hsvMat.get(0,0) but I don't know how to apply this to the masked region only, rather than the entire Mat.
For example, following is a function to which a camera frame is passed as an argument, and I have generated a mask, but how should I proceed from there?
NOTE: Please keep in mind that the masked region is Not a rectangle, it has an irregular shape.
private void detectColoredBlob (Mat rgbaFrame) {
Mat hsvImage = new Mat();
Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_RGB2HSV_FULL);
Mat maskedImage = new Mat();
Scalar lowerThreshold = new Scalar(85, 50, 20);
Scalar upperThreshold = new Scalar(135, 255, 77);
Core.inRange(hsvImage, lowerThreshold, upperThreshold, maskedImage);
Mat dilatedMat= new Mat();
Imgproc.dilate(maskedImage, dilatedMat, new Mat() );
//****************WHAT NOW???**************
}