I'm new to openCV, I've been getting into the samples provided for Android.
My goals is to detect color-blobs so I started with color-blob-detection sample.
I'm converting color image to grayscale and then thresholding using a binary threshold.
The background is white, blobs are black. I want to detect those black blobs. Also, I would like to draw their contour in color but I'm not able to do it because image is black and white.
I've managed to accomplish this in grayscale but I don't prefer how the contours are drawn, it's like color tolerance is too high and the contour is bigger than the actual blob (maybe blobs are too small?). I guess this 'tolerance' I talk about has something to do with setHsvColor but I don't quite understand that method.
Thanks in advance! Best Regards
UPDATE MORE INFO
The image I want to track is of ink splits. Imagine a white piece of paper with black ink splits. Right now I'm doing it in real-time (camera view). The actual app would take a picture and analyse that picture.
As I said above, I took color-blob-detection sample (android) from openCV GitHub repo. And I add this code in the onCameraFrame method (in order to convert it to black and white in real-time) The convertion is made so I don't mind if ink is black, blue, red:
mRgba = inputFrame.rgba();
/**************************************************************************/
/** BLACK AND WHITE **/
// Convert to Grey
Imgproc.cvtColor(inputFrame.gray(), mRgba, Imgproc.COLOR_GRAY2RGBA, 4);
Mat blackAndWhiteMat = new Mat ( H, W, CvType.CV_8U, new Scalar(1));
double umbral = 100.0;
Imgproc.threshold(mRgba, blackAndWhiteMat , umbral, 255, Imgproc.THRESH_BINARY);
// convert back to bitmap for displaying
Bitmap resultBitmap = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(), Bitmap.Config.ARGB_8888);
blackAndWhiteMat.convertTo(blackAndWhiteMat, CvType.CV_8UC1);
Utils.matToBitmap(blackAndWhiteMat, resultBitmap);
/**************************************************************************/
This may not be the best way but it works.
Now I want to detect black blobs (ink splits). I guess they are detected because the Logcat (log entry of sample app) throws the number of contours detected, but I'm not able to see them because the image is black and white and I want the contour to be red, for example.
Here's an example image:-
And here is what I get using RGB (color-blob-detection as is, not black and white image). Notice how small blobs are not detected. (Is it possible to detect them? or are they too small?)
Thanks for your help! If you need more info I would gladly update this question
UPDATE: GitHub repo of color-blob-detection sample (second image)
GitHub Repo of openCV sample for Android
The solution is based on a combination of adaptive Image thresholding and use of the connected-component algorithm.
Assumption - The paper is the most lit area of the image whereas the ink spots on the paper are darkest regions.
from random import Random
import numpy as np
import cv2
def random_color(random):
"""
Return a random color
"""
icolor = random.randint(0, 0xFFFFFF)
return [icolor & 0xff, (icolor >> 8) & 0xff, (icolor >> 16) & 0xff]
#Read as Grayscale
img = cv2.imread('1-input.jpg', 0)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
# Gaussian to remove noisy region, comment to see its affect.
img = cv2.medianBlur(img,5)
#Find average intensity to distinguish paper region
avgPixelIntensity = cv2.mean( img )
print "Average intensity of image: ", avgPixelIntensity[0]
# Generate mask to distinguish paper region
#0.8 - used to ignore ill-illuminated region of paper
mask = cv2.inRange(img, avgPixelIntensity[0]*0.8, 255)
mask = 255 - mask
cv2.imwrite('2-maskedImg.jpg', mask)
#Approach 1
# You need to choose 4 or 8 for connectivity type(border pixels)
connectivity = 8
# Perform the operation
output = cv2.connectedComponentsWithStats(mask, connectivity, cv2.CV_8U)
# The first cell is the number of labels
num_labels = output[0]
# The second cell is the label matrix
labels = output[1]
# The third cell is the stat matrix
stats = output[2]
# The fourth cell is the centroid matrix
centroids = output[3]
cv2.imwrite("3-connectedcomponent.jpg", labels)
print "Number of labels", num_labels, labels
# create the random number
random = Random()
for i in range(1, num_labels):
print stats[i, cv2.CC_STAT_LEFT], stats[i, cv2.CC_STAT_TOP], stats[i, cv2.CC_STAT_WIDTH], stats[i, cv2.CC_STAT_HEIGHT]
cv2.rectangle(cimg, (stats[i, cv2.CC_STAT_LEFT], stats[i, cv2.CC_STAT_TOP]),
(stats[i, cv2.CC_STAT_LEFT] + stats[i, cv2.CC_STAT_WIDTH], stats[i, cv2.CC_STAT_TOP] + stats[i, cv2.CC_STAT_HEIGHT]), random_color(random), 2)
cv2.imwrite("4-OutputImage.jpg", cimg)
The Input Image
Masked Image from thresholding and invert operation.
Use of connected component.
Overlaying output of connected component on input image.
Related
Hello stackoverflow community I would like if someone can guide me a little regarding my next question, I want to make an application that takes a photo when it detects a sheet with 3 marks (black squares in the corners) similar to what a QR would have. I have read a little about opencv that I think could help me more however I am not very clear yet.
Here my example
Once you obtain your binary image, you can find contours and filter using contour approximation and contour area. If the approximated contour has a length of four then it must be a square and if it is within a lower and upper area range then we have detected a mark. We keep a counter of the mark and if there are three marks in the image, we can take the photo. Here's the visualization of the process.
We Otsu's threshold to obtain a binary image with the objects to detect in white.
From here we find contours using cv2.findContours and filter using contour approximation cv2.approxPolyDP in addition to contour area cv2.contourArea.
Detected marks highlighted in teal
I implemented it in Python but you can adapt the same approach
Code
import cv2
# Load image, grayscale, Otsu's threshold
image = cv2.imread('1.png')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
# Find contours and filter using contour approximation and contour area
marks = 0
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
area = cv2.contourArea(c)
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.04 * peri, True)
if len(approx) == 4 and area > 250 and area < 400:
x,y,w,h = cv2.boundingRect(c)
cv2.rectangle(image, (x, y), (x + w, y + h), (200,255,12), 2)
marks += 1
# Sheet has 3 marks
if marks == 3:
print('Take photo')
cv2.imshow('thresh', thresh)
cv2.imshow('image', image)
cv2.waitKey()
I am kind of stuck with this problem, and I know there are so many questions about it on stack overflow but in my case. Nothing gives the expected result.
The Context:
Am using Android OpenCV along with Tesseract so I can read the MRZ area in the passport. When the camera is started I pass the input frame to an AsyncTask, the frame is processed, the MRZ area is extracted succesfully, I pass the extracted MRZ area to a function prepareForOCR(inputImage) that takes the MRZ area as gray Mat and Will output a bitmap with the thresholded image that I will pass to Tesseract.
The problem:
The problem is while thresholding the Image, I use adaptive thresholding with blockSize = 13 and C = 15, but the result given is not always the same depending on the lighting of the image and the conditions in general from which the frame is taken.
What I have tried:
First I am resizing the image to a specific size (871,108) so the input image is always the same and not dependant on which phone is used.
After resizing, I try with different BlockSize and C values
//toOcr contains the extracted MRZ area
Bitmap toOCRBitmap = Bitmap.createBitmap(bitmap);
Mat inputFrame = new Mat();
Mat toOcr = new Mat();
Utils.bitmapToMat(toOCRBitmap, inputFrame);
Imgproc.cvtColor(inputFrame, inputFrame, Imgproc.COLOR_BGR2GRAY);
TesseractResult lastResult = null;
for (int B = 11; B < 70; B++) {
for (int C = 11; C < 70; C++){
if (IsPrime(B) && IsPrime(C)){
Imgproc.adaptiveThreshold(inputFrame, toOcr, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY, B ,C);
Bitmap toOcrBitmap = OpenCVHelper.getBitmap(toOcr);
TesseractResult result = TesseractInstance.extractFrame(toOcrBitmap, "ocrba");
if (result.getMeanConfidence()> 70) {
if (MrzParser.tryParse(result.getText())){
Log.d("Main2Activity", "Best result with " + B + " : " + C);
return result;
}
}
}
}
}
Using the code below, the thresholded result image is a black on white image which gives a confidence greater than 70, I can't really post the whole image for privacy reasons, but here's a clipped one and a dummy password one.
Using the MrzParser.tryParse function which adds checks for the character position and its validity within the MRZ, am able to correct some occurences like a name containing a 8 instead of B, and get a good result but it takes so much time, which is normal because am thresholding almost 255 images in the loop, adding to that the Tesseract call.
I already tried getting a list of C and B values which occurs the most but the results are different.
The question:
Is there a way to define a C and blocksize value so that it s always giving the same result, maybe adding more OpenCV calls so The input image like increasing contrast and so on, I searched the web for 2 weeks now I can't find a viable solution, this is the only one that is giving accurate results
You can use a clustering algorithm to cluster the pixels based on color. The characters are dark and there is a good contrast in the MRZ region, so a clustering method will most probably give you a good segmentation if you apply it to the MRZ region.
Here I demonstrate it with MRZ regions obtained from sample images that can be found on the internet.
I use color images, apply some smoothing, convert to Lab color space, then cluster the a, b channel data using kmeans (k=2). The code is in python but you can easily adapt it to java. Due to the randomized nature of the kmeans algorithm, the segmented characters will have label 0 or 1. You can easily sort it out by inspecting cluster centers. The cluster-center corresponding to characters should have a dark value in the color space you are using.
I just used the Lab color space here. You can use RGB, HSV or even GRAY and see which one is better for you.
After segmenting like this, I think you can even find good values for B and C of your adaptive-threshold using the properties of the stroke width of the characters (if you think the adaptive-threshold gives a better quality output).
import cv2
import numpy as np
im = cv2.imread('mrz1.png')
# convert to Lab
lab = cv2.cvtColor(cv2.GaussianBlur(im, (3, 3), 1), cv2.COLOR_BGR2Lab)
im32f = np.array(im[:, :, 1:3], dtype=np.float32)
k = 2 # 2 clusters
term_crit = (cv2.TERM_CRITERIA_EPS, 30, 0.1)
ret, labels, centers = cv2.kmeans(im32f.reshape([im.shape[0]*im.shape[1], -1]),
k, None, term_crit, 10, 0)
# segmented image
labels = labels.reshape([im.shape[0], im.shape[1]]) * 255
Some results:
I want to remove black borders around License Plate. I am using opencv + android.
Please reply with code using which i can remove the borders.
I have also attached the image.image 1
You can perform (DoG) Difference of Gaussians to detect the high frequency details in your image. By high frequency in an image I mean distinct edges and corners.
Here is the code as requested. The explanations are placed as comments by the side:
import cv2
img = cv2.imread('number_plate.jpg') #---Reading the image---
img1 = img.copy() #----The final contour will be drawn on the copy of the original image---
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #---converting to gray scale---
Before performing DoG, I enhanced the gray sale image by applying Adaptive histogram equalization:
clahe = cv2.createCLAHE(clipLimit=3.0, tileGridSize=(8,8))
enhanced = clahe.apply(gray_img)
cv2.imshow(enhanced_gray_img', enhanced)
Now I performed Gaussian blur using two separate kernels and subtracted the resulting images as follows:
blur1 = cv2.GaussianBlur(enhanced, (15, 15), 0)
blur2 = cv2.GaussianBlur(enhanced, (25, 25), 0)
difference = blur2 - blur1
cv2.imshow('Difference_of_Gaussians', difference)
Then I performed binary threshold on the image above and found contours. I drew the contour having the largest area:
ret, th = cv2.threshold(difference, 127,255, 0) #---performed binary threshold ---
_, contours, hierarchy = cv2.findContours(th, cv2.RETR_EXTERNAL, 1) #---Find contours---
cnts = contours
max = 0 #----Variable to keep track of the largest area----
c = 0 #----Variable to store the contour having largest area---
for i in range(len(contours)):
if (cv2.contourArea(cnts[i]) > max):
max = cv2.contourArea(cnts[i])
c = i
rep = cv2.drawContours(img1, contours[c], -1, (0,255,0), 3) #----Draw the contour having the largest area on the image---
cv2.imshow('Final_Image.jpg', rep)
And voila!!! There you go.
Now you can obtain bounding rectangles for the contours you found and fed those coordinates as regions to the OCR to extract the text present
I have a problem with conversion from BGR to HSV.
I'm programming with Android Studio and testing with my Xperia Z5.
In my code snippet, I'm getting totally wrong colour values:
Scalar LOWER_RED = (0,0,0);
Scalar HIGHER_RED = (30,255,255);
Mat src = new Mat(Bitmap.getHeight(), Bitmap.getWidth(),CvType.CV_8UC4);
Mat hsv = new Mat(Bitmap.getHeight(), Bitmap.getWidth(),CvType.CV_8UC4);
Utils bitmapToMat(Bitmap, src);
Imgproc.cvtColor(src,hsv,Imgproc.COLOR_BGR2HSV);
Core.inRange(hsv, LOWER_RED, HIGHER_RED, hsv);
Utils.matToBitmap(hsv,Bitmap);
I want to capture red colour. What did I do wrong?
Edit:
I tried with all advices and my Code Snippet looks now this way:
Scalar LOWER_RED = (0,10,100);
Scalar HIGHER_RED = (10,255,255);
Mat src = new Mat(Bitmap.getHeight(), Bitmap.getWidth(),CvType.CV_8UC3);
Mat hsv = new Mat(Bitmap.getHeight(), Bitmap.getWidth(),CvType.CV_8UC3);
Utils bitmapToMat(Bitmap, src);
Imgproc.cvtColor(src,hsv,Imgproc.COLOR_BGR2HSV);
Core.inRange(hsv, LOWER_RED, HIGHER_RED, hsv);
Utils.matToBitmap(hsv,Bitmap);
The Outcome is a black screen ( no matches )
with
Core.inRange(hsv,New Scalar(0,0,0),New Scalar(10,255,255),HighRedRange);
Core.inRange(hsv,New Scalar(160,100,100),New Scalar(179,255,255),LowRedRange);
Core.addWeighted(LowRedRange,1.0,HighredRange,1.0,0.0,hsv);
The vegetables are black and the white background is white in hsv
0,0,0 - 10,255,255 AND 160,100,100 - 179,255,255
If I use a Scalar from 110,100,100 until 135,255,255, then the red pepper is white and the back ground black ( correctly detected ).
Source Picture:
And I dont understand all this...
There is a good tutorial here. It's for C++ but the general idea is the same. I tried the general idea and it surely works. The problem is that your range is too broad. In OpenCV, Hue range is in 0-180. Meaning that your higher limit goes to 30*2 = 60 which includes nearly all yellow range too.
I set the range from 0 to 10 for Hue, but remember you may also want to get 160 - 179 range which also includes some part of red. For this, you just need a second mask and then combine them with simple addition.
The example code in Python:
import cv2
import numpy as np
img = cv2.imread('peppers.jpg',1)
im_hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
thresh_low = np.array([0,100,100])
thresh_high = np.array([10,255,255])
mask = cv2.inRange(im_hsv, thresh_low, thresh_high)
im_masked = cv2.bitwise_and(img,img, mask= mask)
cv2.imshow('Masked',im_masked)
cv2.waitKey(0)
Original image:
Result:
I know now my problem
it's this:
Imgproc.cvtColor(src,hsv,Imgproc.COLOR_BGR2HSV);
With RGB2HSV all values are correct.
I thought on Android Smartphones there is BGR used ?
However, Big thanks for all answers.
I wish all of you a great day :)
I am working on an app that is expected to remove image backgrounds using opencv, at first I tried using grabcut but it was too slow and the results were not always accurate, then I tried using threshold, although the results are not yet close th grabcut, its very fast and looks like a better, So my code is first looking at the image hue and analying which portion of it appears more, that portion is taken in as the background, the issue is at times its getting the foreground as background below is my code:
private Bitmap backGrndErase()
{
Bitmap bitmap = BitmapFactory.decodeResource(getResources(), R.drawable.skirt);
Log.d(TAG, "bitmap: " + bitmap.getWidth() + "x" + bitmap.getHeight());
bitmap = ResizeImage.getResizedBitmap(bitmap, calculatePercentage(40, bitmap.getWidth()), calculatePercentage(40, bitmap.getHeight()));
Mat frame = new Mat();
Utils.bitmapToMat(bitmap, frame);
Mat hsvImg = new Mat();
List<Mat> hsvPlanes = new ArrayList<>();
Mat thresholdImg = new Mat();
// int thresh_type = Imgproc.THRESH_BINARY_INV;
//if (this.inverse.isSelected())
int thresh_type = Imgproc.THRESH_BINARY;
// threshold the image with the average hue value
hsvImg.create(frame.size(), CvType.CV_8U);
Imgproc.cvtColor(frame, hsvImg, Imgproc.COLOR_BGR2HSV);
Core.split(hsvImg, hsvPlanes);
// get the average hue value of the image
double threshValue = this.getHistAverage(hsvImg, hsvPlanes.get(0));
Imgproc.threshold(hsvPlanes.get(0), thresholdImg, threshValue, mThresholdValue, thresh_type);
// Imgproc.adaptiveThreshold(hsvPlanes.get(0), thresholdImg, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 11, 2);
Imgproc.blur(thresholdImg, thresholdImg, new Size(5, 5));
// dilate to fill gaps, erode to smooth edges
Imgproc.dilate(thresholdImg, thresholdImg, new Mat(), new Point(-1, -1), 1);
Imgproc.erode(thresholdImg, thresholdImg, new Mat(), new Point(-1, -1), 3);
Imgproc.threshold(thresholdImg, thresholdImg, threshValue, mThresholdValue, Imgproc.THRESH_BINARY);
//Imgproc.adaptiveThreshold(thresholdImg, thresholdImg, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 11, 2);
// create the new image
Mat foreground = new Mat(frame.size(), CvType.CV_8UC3, new Scalar(255, 255, 255));
frame.copyTo(foreground, thresholdImg);
Utils.matToBitmap(foreground,bitmap);
//return foreground;
alreadyRun = true;
return bitmap;
}
the method responsible for Hue:
private double getHistAverage(Mat hsvImg, Mat hueValues)
{
// init
double average = 0.0;
Mat hist_hue = new Mat();
// 0-180: range of Hue values
MatOfInt histSize = new MatOfInt(180);
List<Mat> hue = new ArrayList<>();
hue.add(hueValues);
// compute the histogram
Imgproc.calcHist(hue, new MatOfInt(0), new Mat(), hist_hue, histSize, new MatOfFloat(0, 179));
// get the average Hue value of the image
// (sum(bin(h)*h))/(image-height*image-width)
// -----------------
// equivalent to get the hue of each pixel in the image, add them, and
// divide for the image size (height and width)
for (int h = 0; h < 180; h++)
{
// for each bin, get its value and multiply it for the corresponding
// hue
average += (hist_hue.get(h, 0)[0] * h);
}
// return the average hue of the image
average = average / hsvImg.size().height / hsvImg.size().width;
return average;
}
A sample of the input and output:[
Input Image 2 and Output:
Input Image 3 and Output:
Indeed, as others have said you are unlikely to get good results just with a threshold on hue. You can use something similar to GrabCut, but faster.
Under the hood, GrabCut calculates foreground and background histograms, then calculates the probability of each pixel being FG/BG based on these histograms, and then optimizes the resulting probability map using graph cut to obtain a segmentation.
Last step is most expensive, and it may be ignored depending on the application. Instead, you may apply the threshold to the probability map to obtain a segmentation. It may (and will) be worse than GrabCut, but will be better than your current approach.
There are some points to consider for this approach. The choice of histogram model would be very important here. You can either consider 2 channels in some space like YUV or HSV, consider 3 channels of RGB, or consider 2 channels of normalized RGB. You also have to select an appropriate bin size for those histograms. Too small bins would lead to 'overtraining', while too large will reduce the precision. The tradeoffs between those are a topic for a separate discussion, in brief - I would advice using RGB with 64 bins per channel for start and then see what changes are better for your data.
Also, you can get better results for coarse binning if you use interpolation to get values between bins. In past I have used trilinear interpolation and it was kind of good, compared to no interpolation at all.
But remember that there are no guarantees that your segmentation will be correct without prior knowledge on object shape, either with GrabCut, thresholding or this approach.
I would try again Grabcut, it is one of the best segmentation methods available. This is the result I get
cv::Mat bgModel,fgModel; // the models (internally used)
cv::grabCut(image,// input image
object_mask,// segmentation result
rectang,// rectangle containing foreground
bgModel,fgModel, // models
5,// number of iterations
cv::GC_INIT_WITH_RECT); // use rectangle
// Get the pixels marked as likely foreground
cv::compare(object_mask,cv::GC_PR_FGD,object_mask,cv::CMP_EQ);
cv::threshold(object_mask, object_mask, 0,255, CV_THRESH_BINARY); //ensure the mask is binary
The only problem of Grabcut is that you have to give as an input a rectangle containing the object you want to extract. Apart from that it works pretty well.
Your method of finding average hue is WRONG! As you most probably know, hue is expressed as angle and takes value in [0,360] range. Therefore, a pixel with hue 360 essentially has same colour as a pixel with hue 0 (both are pure red). In the same way, a pixel with hue 350 is actually closer to a pixel with hue 10 than a pixel with hue, say for example, 300.
As for opencv, cvtColor function actually divides calculated hue value by 2 to fit it in 8 bit integer. Thus, in opencv, hue values wrap after 180. Now, consider we have two red(ish) pixels with hues 10 and 170. If we take their average, we will get 90 — hue of pure cyan, the exact opposite of red — which is not our desired value.
Therefore, to correctly find the average hue, you need to first find average pixel value in RGB colour space, then calculate the hue from this RGB value. You can create 1x1 matrix with average RGB pixel and convert it to HSV/HSL.
Following the same reasoning, applying threshold to hue image doesn't work flawlessly. It does not consider wrapping of hue values.
If I understand correctly, you want to find pixels with similar hue as the background. Assuming we know the colour of background, I would do this segmentation in RGB space. I would introduce some tolerance variable. I would use the background pixel value as centre and this tolerance as radius and thus define a sphere in RGB colour space. Now, rest is inspecting each pixel value, if it falls inside this sphere, then classify as background; otherwise, regard it as foreground pixel.