Develop Reference

Blurring a Rect within a screenshot

I'm developing an Android app which uses a background Service to programmatically capture a screenshot of whatever is on the screen currently. I obtain the screenshot as a Bitmap.
Next, I successfully imported OpenCV into my Android project.
What I need to do now is blur a subset of this image, i.e. not the entire image itself, but a [rectangular] area or sub-region within the image. I have an array of Rect objects representing the rectangular regions that I need to blur within the screenshot.
I've been looking around for a tutorial on doing this with OpenCV in Java, and I haven't found a clear answer. The Mat and Imgproc classes are obviously the ones of interest, and there's the Mat.submat() method, but I've been unable to find a clear, straightforward tutorial on getting this done.
I've googled a lot, and none of the examples I've found are complete. I need to do this in Java, within the Android runtime.
What I need is: Bitmap >>> Mat >>> Imgproc>>> Rect >>> Bitmap with ROI
blurred.
Any experienced OpenCV devs out here, can you point me in the right direction? This is the only thing I'm stuck at.
Related:
Gaussian blurring with OpenCV: only blurring a subregion of an image?.
How to blur a rectagle with OpenCv.
How to blur some portion of Image in Android?.

The C++ code to achieve this task is shared below with comments and sample images:
// load an input image
Mat img = imread("C:\\elon_tusk.png");
img:
// extract subimage
Rect roi(113, 87, 100, 50);
Mat subimg = img(roi);
subimg:
// blur the subimage
Mat blurred_subimage;
GaussianBlur(subimg, blurred_subimage, Size(0, 0), 5, 5);
blurred_subimage:
// copy the blurred subimage back to the original image
blurred_subimage.copyTo(img(roi));
img:
Android equivalent:
Mat img = Imgcodecs.imread("elon_tusk.png");
Rect roi = new Rect(113, 87, 100, 50);
Mat subimage = img.submat(roi).clone();
Imgproc.GaussianBlur(subimg, subimg, new Size(0,0), 5, 5);
subimg.copyTo(img.submat(roi));

You could just implement your own helper function, let's call it roi (region of interest).
Since images in opencv are numpy ndarrays, you can do something like this:
def roi(image: np.ndarray, region: QRect) -> np.ndarray:
a1 = region.upperLeft().x()
b1 = region.bottomRight().y()
a2 = region.upperLeft().x()
b2 = region.bottomRight().y()
return image[a1:a2, b1:b2]
And just use this helper function to extract the subregions of the image that you are interested, blur them and put the result back on the original picture.

Thresholding in Android using opencv

Not sure if this is the right way to ask, but please help. I have an image of a dented car. I have to process it and highlight the dents and return the number of dents. I was able to do it reasonably well with the following result:
The matlab code is:
img2=rgb2gray(i1);
imshow(img2);
img3=imtophat(img2,strel('disk',15));
img4=imadjust(img3);
layer=img4(:,:,1);
img5=layer>100 & layer<250;
img6=imfill(img5,'holes');
img7=bwareaopen(img6,5);
[L,ans]=bwlabeln(img7);
imshow(img7);
I=imread(i1);
Ians=CarDentIdentification(I);
However, when I try to do this using opencv, I get this:
With the following code:
Imgproc.cvtColor(source, middle, Imgproc.COLOR_RGB2GRAY);
Imgproc.equalizeHist(middle, middle);
Imgproc.threshold(middle, middle, 150, 255, Imgproc.THRESH_OTSU);
Please tell me how can I obtain better results in opencv, and also how to count the dents? I tried findcontour() but it gives a very large number. I tried on other images as well, but I'm not getting proper results.
Please help.

So you basically from the MATLAB site, imtophat does - Top-hat filtering computes the morphological opening of the image (using imopen) and then subtracts the result from the original image.
You could do this in OpenCV with the following steps:
Step 1: Get the disk structuring element
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15, 15))
Step 2: Compute opening of the image and then subtract the result from the original image
tophat = cv2.morphologyEx(v, cv2.MORPH_TOPHAT, kernel)
This gives following result -
Step 3 - Now you could just manually threshold it or use Otsu -
ret, thresh = cv2.threshold(tophat, 17, 255, 0)
which gives you the following image -
Since the OP wants the code in Java, here is the probable code in Java:
private Mat topHat(Mat image)
{
Mat element = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(15, 15), new Point (0, 0));
Mat dst = new Mat;
Imgproc.morphologyEx(image, dst, Imgproc.MORPH_TOPHAT, element, new Point(0, 0));
return dst;
}
Make sure you do this on a gray scale image (CvType.8UC1) and then you can threshold suitably.

Why is the drawContour() in OpenCV generating this strange Mask?

I started by reading in this Mat.
Then I converted it to Greyscale and applied Imgproc.canny() to it, getting the following mask.
Then I used Imgproc.findContours() to find the contours, Imgproc.drawContours(), and Core.putText() to label the contours with numbers:
Then I did Rect boundingRect = Imgproc.boundingRect(contours.get(0));
Mat submatrix = new Mat();
submatrix = originalMat.submat(boundingRect); to get following submatrix:
So far so good. The Problem starts hereafter:
NOW I NEEDED A MASK OF THE submatrix. So I decided to use Imgproc.drawContours() to get the mask:
Mat mask = new Mat(submatrix.rows(), submatrix.cols(), CvType.CV_8UC1);
List<MatOfPoint> contourList = new ArrayList<>();
contourList.add(contours.get(0));
Imgproc.drawContours(mask, contourList, 0, new Scalar(255), -1);
I got the following mask:
WHAT I WAS EXPECTING was a filled (in white color) diamond shape on black background.
WHy am I getting this unexpected result?
EDIT:
When I replaced Mat mask = new Mat(submatrix.rows(),
submatrix.cols(), CvType.CV_8UC1); by Mat mask =
Mat.zeros(submatrix.rows(), submatrix.cols(), CvType.CV_8UC1);,
the last mask with white colored garbage was replaced by an empty
black mask withOUT any white color on it. I got the following submat
and mask:
I was getting the first contour in the list of contours (named
contours) by contours.get(0), and using this first contour to
calculate Imgproc.boundingRect() as well as in
contourList.add(contours.get(0)); later (where contourList is
the list of just one contour which will be used in the last
drawContours()).
Then I went ahead to change contours.get(0) to
contours.get(1) in Imgproc.boundingRect() as well as in contourList.add(); (just before Imgproc.drawContours()). That
resulted in this submat and mask:
Then I changed back to contours.get(0) in
Imgproc.boundingRect(); and let
contourList.add(contours.get(1)); be there. Got the following
submat and mask:
NOW I am completely Unable to Understand what is happening here.

I am not sure how this is handle in JAVA (I usually use OpenCV in c++ or python), but there is an error in your code...
The contours list will have a list of list of points. This points will refer to the original image. So, this mean that if the figure one is in lets say, x=300, y= 300, width= 100, height=100 then when you get your submatrix it will try to draw those points in a smaller image... so when it tries to draw point (300,300) in a 100 x 100 image, it will simply fail... probably throws an error or simply doesn't draw anything...
A solution for this is, do a for loop and substract to each point of the contour the initial point of the bounding rect (in my example (300,300)).
As, why there is some garbage drawn... well you never initialize the matrix. Not sure in JAVA, but in c++ you have to set them to 0.
I think it should be something like this:
Mat mask = new Mat(submatrix.rows(), submatrix.cols(), CvType.CV_8UC1, new Scalar(0));
I hope this helps :)
EDIT
I think I did not explain myself clearly before.
Your contours are an array of points (x,y). These are the coordinates of the points that represent each contour in the original image. This image has a size, and your submatrix has a smaller size. The points are outside of this small image boundaries....
you should do something like this to fix it:
for (int j = 0; j < contours[0].length; j++) {
contours[0][j].x -= boundingrect.x;
contours[0][j].y -= boundingrect.y;
}
and then you can draw the contours, since they will be in boundaries of the submat.
I think in java it is also possible to subtract the opencv points directly:
for (int j = 0; j < contours[0].length; j++) {
contours[0][j] -= boundingrect.tl();
}
but in this case I am not sure, since I have tried it in c++ only
boundingrect.tl() -> gives you the top left point of the rect

why detected object jumps using OpenCV

Ok I have a strange problem. I'll try to describe it as best as I can.
I've learned my app to detect a car when looking on it from the side
Imgproc.cvtColor(aInputFrame, grayscaleImage, Imgproc.COLOR_RGBA2RGB);
MatOfRect objects = new MatOfRect();
// Use the classifier to detect cars
if (cascadeClassifier != null) {
cascadeClassifier.detectMultiScale(grayscaleImage, objects, 1.1, 1,
2, new Size(absoluteObjectSize, absoluteObjectSize),
new Size());
}
for (int i = 0; i < dataArray.length; i++) {
Core.rectangle(aInputFrame, dataArray[i].tl(), dataArray[i].br(),
new Scalar(0, 255, 0, 255), 3);
mRenderer.setCameraPosition(-5, 5, 60f);
}
Now, this code works nice. I mean that i detects cars and it marks them with green rectangle. The problem is that the marked rectangle jumps like hell. I mean even when the phone is hold still the rectangle jumps from left to right to middle. There is never one still rectangle. I hope I've described the problem properly. I would like to stabilizy the marking cause I want to draw an overlay based on it and I can't make it to jump like this

See(1) the parameters for detectMultiScale and it expects
image of type CV_8U. You will need to convert to gray scale image
with COLOR_RGBA2GRAY instead of COLOR_RGBA2RGB
In detectMultiScale, increase the number of neighbours parameter to avoid false positives.
Suggestion: If input is a video stream, don't run
detectMultiScale on every frame. It is slow even if you use LBP
cascades. Try detection in one frame, followed by tracking techniques.

How to use OpenCV to process image so that the text become sharp and clear?

Wanted to achieve something like this: http://www.leptonica.com/binarization.html
While searching for solutions, most of the answers were general instructions such as advise to look at adaptive filter, gaussian blur, dilation and erosion but none of them provide any sample code to start with (so can play around with the values)..
I know different image require different methods and values to achieve optimum clarity, but I just need some general filter so that the image at least slightly sharper and less noisy compare to the original, before doing any OCR on it.
this is what I've tried so far..
Mat imageMat = new Mat();
Utils.bitmapToMat(photo, imageMat);
Imgproc.cvtColor(imageMat, imageMat, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur(imageMat, imageMat, new Size(3, 3), 0);
Imgproc.adaptiveThreshold(imageMat, imageMat, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY_INV, 5, 4);
but being an image processing newb, obviously I don't know what I'm doing XD
original image:
after applying the above:
How to do it correctly?
UPDATE: got it much closer thanks to metsburg, berak and Aurelius
Using the medianBlur method since cvSmooth with CV_MEDIAN is deprecated and replaced with medianBlur:
Imgproc.medianBlur(imageMat, imageMat, 3);
Imgproc.threshold(imageMat, imageMat, 0, 255, Imgproc.THRESH_OTSU);
Result:
Using back the GaussianBlur method, the result actually is slightly better:
Imgproc.GaussianBlur(imageMat, imageMat, new Size(3, 3), 0);
Imgproc.threshold(imageMat, imageMat, 0, 255, Imgproc.THRESH_OTSU);
Result:
For this image the difference is not noticable, so I tried another image which is a photo taken off the computer screen. Computer screen gives a lot of noises (wavy lines) so it is very hard to remove the noise.
Example original image:
Directly applying otsu:
using medianBlur before otsu:
using GaussianBlur before otsu:
Seems like gaussian blur is slightly better, however I'm still playing with the settings..
If anyone can advise on how to improve the computer screen photo further, please, let us know :)
One more thing.. using this method on the image inside the top link yields horrible results :( see it here: http://imgur.com/vOZAaE0

Well, you're almost there. Just try these modifications:
Instead of
Imgproc.GaussianBlur(imageMat, imageMat, new Size(3, 3), 0);
try:
cvSmooth(imageMat, imageMat, CV_MEDIAN, new Size(3, 3), 0);
check the syntax, may not exactly match
The link you posted uses thresholding of Otsu, so try this:
Imgproc.threshold(imageMat, imageMat, 0, 255, Imgproc.THRESH_OTSU);
for thresholding.
Try tweaking the parameters here and there, you should get something pretty close to your desired outcome.

Instead of using Imgproc.THRESH_BINARY_INV use Imgproc.THRESH_BINARY only as _INV is inverting your image after binarisations and resulted is the said output shown above in your example.
correct code:
Imgproc.adaptiveThreshold(imageMat, imageMat, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 5, 4);