I started by reading in this Mat.
Then I converted it to Greyscale and applied Imgproc.canny() to it, getting the following mask.
Then I used Imgproc.findContours() to find the contours, Imgproc.drawContours(), and Core.putText() to label the contours with numbers:
Then I did Rect boundingRect = Imgproc.boundingRect(contours.get(0));
Mat submatrix = new Mat();
submatrix = originalMat.submat(boundingRect); to get following submatrix:
So far so good. The Problem starts hereafter:
NOW I NEEDED A MASK OF THE submatrix. So I decided to use Imgproc.drawContours() to get the mask:
Mat mask = new Mat(submatrix.rows(), submatrix.cols(), CvType.CV_8UC1);
List<MatOfPoint> contourList = new ArrayList<>();
contourList.add(contours.get(0));
Imgproc.drawContours(mask, contourList, 0, new Scalar(255), -1);
I got the following mask:
WHAT I WAS EXPECTING was a filled (in white color) diamond shape on black background.
WHy am I getting this unexpected result?
EDIT:
When I replaced Mat mask = new Mat(submatrix.rows(),
submatrix.cols(), CvType.CV_8UC1); by Mat mask =
Mat.zeros(submatrix.rows(), submatrix.cols(), CvType.CV_8UC1);,
the last mask with white colored garbage was replaced by an empty
black mask withOUT any white color on it. I got the following submat
and mask:
I was getting the first contour in the list of contours (named
contours) by contours.get(0), and using this first contour to
calculate Imgproc.boundingRect() as well as in
contourList.add(contours.get(0)); later (where contourList is
the list of just one contour which will be used in the last
drawContours()).
Then I went ahead to change contours.get(0) to
contours.get(1) in Imgproc.boundingRect() as well as in contourList.add(); (just before Imgproc.drawContours()). That
resulted in this submat and mask:
Then I changed back to contours.get(0) in
Imgproc.boundingRect(); and let
contourList.add(contours.get(1)); be there. Got the following
submat and mask:
NOW I am completely Unable to Understand what is happening here.
I am not sure how this is handle in JAVA (I usually use OpenCV in c++ or python), but there is an error in your code...
The contours list will have a list of list of points. This points will refer to the original image. So, this mean that if the figure one is in lets say, x=300, y= 300, width= 100, height=100 then when you get your submatrix it will try to draw those points in a smaller image... so when it tries to draw point (300,300) in a 100 x 100 image, it will simply fail... probably throws an error or simply doesn't draw anything...
A solution for this is, do a for loop and substract to each point of the contour the initial point of the bounding rect (in my example (300,300)).
As, why there is some garbage drawn... well you never initialize the matrix. Not sure in JAVA, but in c++ you have to set them to 0.
I think it should be something like this:
Mat mask = new Mat(submatrix.rows(), submatrix.cols(), CvType.CV_8UC1, new Scalar(0));
I hope this helps :)
EDIT
I think I did not explain myself clearly before.
Your contours are an array of points (x,y). These are the coordinates of the points that represent each contour in the original image. This image has a size, and your submatrix has a smaller size. The points are outside of this small image boundaries....
you should do something like this to fix it:
for (int j = 0; j < contours[0].length; j++) {
contours[0][j].x -= boundingrect.x;
contours[0][j].y -= boundingrect.y;
}
and then you can draw the contours, since they will be in boundaries of the submat.
I think in java it is also possible to subtract the opencv points directly:
for (int j = 0; j < contours[0].length; j++) {
contours[0][j] -= boundingrect.tl();
}
but in this case I am not sure, since I have tried it in c++ only
boundingrect.tl() -> gives you the top left point of the rect
Related
I am kind of stuck with this problem, and I know there are so many questions about it on stack overflow but in my case. Nothing gives the expected result.
The Context:
Am using Android OpenCV along with Tesseract so I can read the MRZ area in the passport. When the camera is started I pass the input frame to an AsyncTask, the frame is processed, the MRZ area is extracted succesfully, I pass the extracted MRZ area to a function prepareForOCR(inputImage) that takes the MRZ area as gray Mat and Will output a bitmap with the thresholded image that I will pass to Tesseract.
The problem:
The problem is while thresholding the Image, I use adaptive thresholding with blockSize = 13 and C = 15, but the result given is not always the same depending on the lighting of the image and the conditions in general from which the frame is taken.
What I have tried:
First I am resizing the image to a specific size (871,108) so the input image is always the same and not dependant on which phone is used.
After resizing, I try with different BlockSize and C values
//toOcr contains the extracted MRZ area
Bitmap toOCRBitmap = Bitmap.createBitmap(bitmap);
Mat inputFrame = new Mat();
Mat toOcr = new Mat();
Utils.bitmapToMat(toOCRBitmap, inputFrame);
Imgproc.cvtColor(inputFrame, inputFrame, Imgproc.COLOR_BGR2GRAY);
TesseractResult lastResult = null;
for (int B = 11; B < 70; B++) {
for (int C = 11; C < 70; C++){
if (IsPrime(B) && IsPrime(C)){
Imgproc.adaptiveThreshold(inputFrame, toOcr, 255, Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C, Imgproc.THRESH_BINARY, B ,C);
Bitmap toOcrBitmap = OpenCVHelper.getBitmap(toOcr);
TesseractResult result = TesseractInstance.extractFrame(toOcrBitmap, "ocrba");
if (result.getMeanConfidence()> 70) {
if (MrzParser.tryParse(result.getText())){
Log.d("Main2Activity", "Best result with " + B + " : " + C);
return result;
}
}
}
}
}
Using the code below, the thresholded result image is a black on white image which gives a confidence greater than 70, I can't really post the whole image for privacy reasons, but here's a clipped one and a dummy password one.
Using the MrzParser.tryParse function which adds checks for the character position and its validity within the MRZ, am able to correct some occurences like a name containing a 8 instead of B, and get a good result but it takes so much time, which is normal because am thresholding almost 255 images in the loop, adding to that the Tesseract call.
I already tried getting a list of C and B values which occurs the most but the results are different.
The question:
Is there a way to define a C and blocksize value so that it s always giving the same result, maybe adding more OpenCV calls so The input image like increasing contrast and so on, I searched the web for 2 weeks now I can't find a viable solution, this is the only one that is giving accurate results
You can use a clustering algorithm to cluster the pixels based on color. The characters are dark and there is a good contrast in the MRZ region, so a clustering method will most probably give you a good segmentation if you apply it to the MRZ region.
Here I demonstrate it with MRZ regions obtained from sample images that can be found on the internet.
I use color images, apply some smoothing, convert to Lab color space, then cluster the a, b channel data using kmeans (k=2). The code is in python but you can easily adapt it to java. Due to the randomized nature of the kmeans algorithm, the segmented characters will have label 0 or 1. You can easily sort it out by inspecting cluster centers. The cluster-center corresponding to characters should have a dark value in the color space you are using.
I just used the Lab color space here. You can use RGB, HSV or even GRAY and see which one is better for you.
After segmenting like this, I think you can even find good values for B and C of your adaptive-threshold using the properties of the stroke width of the characters (if you think the adaptive-threshold gives a better quality output).
import cv2
import numpy as np
im = cv2.imread('mrz1.png')
# convert to Lab
lab = cv2.cvtColor(cv2.GaussianBlur(im, (3, 3), 1), cv2.COLOR_BGR2Lab)
im32f = np.array(im[:, :, 1:3], dtype=np.float32)
k = 2 # 2 clusters
term_crit = (cv2.TERM_CRITERIA_EPS, 30, 0.1)
ret, labels, centers = cv2.kmeans(im32f.reshape([im.shape[0]*im.shape[1], -1]),
k, None, term_crit, 10, 0)
# segmented image
labels = labels.reshape([im.shape[0], im.shape[1]]) * 255
Some results:
I wanted to find the convex hull in order to even the edges of a hand-drawn triangle on paper. Smoothing using image processing was not enough because i needed to detect this triangle too and a hand drawn triangle tends to have more than three points if the approxPolyDP function is used. A convex hull of a triangle is correctly identified by the approxPolyDP function.
The problem is, i have other shapes in the image too on which a convex hull is created.
Before convex hull is used: Notice the contour labelled 3
After convex hull is used: the end points have been joined and the contour labelled 3 forms a triangle
Now i wanted to somehow exclude contour 3 from being detected as a triangle.
To do this my strategy was to remove this contour altogether from the ArrayList named hullMop. This is because my triangle detection function uses the contours from hullMop and so it wouldnt even check the contour labelled 3.
extcontours are the contours before convex hull is used.
This function checks if a point from hullMop is inside extcontours. If it isn't, then that must be removed from hullMop because they are the extra set of points generated because of the convex hull, or in other words, the red line in the second image.
Now at this point I feel there is a hole in my concept. The openCV documentation says that the convex Hull returns the subset of the points of the original array, in other words, subset of the points of extcontours.
My question is, how do i get the points of the red line created by the convexHull function. I dont want to use findContours because i feel there is a better way.
private void RemoveFalseHullTriangles(ArrayList<MatOfPoint> extcontours, ArrayList<MatOfPoint> hullMop, int width, int height) {
//if every single point of hullmop doesnt touch or isn't inside extcontours, then that point must be the red line
MatOfPoint2f Contours2f = new MatOfPoint2f();
double [] newA = new double[2];
int hullCounter = 0;
A: for(int i =0;i<extcontours.size();i++) {
MatOfPoint ExtCnt = extcontours.get(i);
MatOfPoint HullCnt = hullMop.get(hullCounter);
ExtCnt.convertTo(Contours2f, CvType.CV_32F);
B: for (int j = 0; j < HullCnt.rows(); j++) {
double[] pt = new double[2];
pt[0] = HullCnt.get(j,0)[0];
pt[1] = HullCnt.get(j,0)[1];
if (Math.abs(Imgproc.pointPolygonTest(Contours2f, new Point(pt), true)) > 40) {
//Remove index from HullMop
hullMop.remove(hullCounter);
hullCounter--;
break B;
}
}
hullCounter++;
}
}
Because the hullMop only has a subset of the points of extcontours, i may never know the points of the red line of the contour labelled 3 after convex hull is used.
Is there anyway to get coordinates of that red line generated by convex hull other than using findContours?
As referenced by Alexandar Reynolds, the problem really was detecting open contours first and excluding those contours before finding the convex hull.
The method to find open contours is explained here:
Recognize open and closed shapes opencv
Basically, if an outer contour has no child contour in the hierarchy, then it is an open contour and must be excluded before finding convex hull ( for my case).
Not sure if this is the right way to ask, but please help. I have an image of a dented car. I have to process it and highlight the dents and return the number of dents. I was able to do it reasonably well with the following result:
The matlab code is:
img2=rgb2gray(i1);
imshow(img2);
img3=imtophat(img2,strel('disk',15));
img4=imadjust(img3);
layer=img4(:,:,1);
img5=layer>100 & layer<250;
img6=imfill(img5,'holes');
img7=bwareaopen(img6,5);
[L,ans]=bwlabeln(img7);
imshow(img7);
I=imread(i1);
Ians=CarDentIdentification(I);
However, when I try to do this using opencv, I get this:
With the following code:
Imgproc.cvtColor(source, middle, Imgproc.COLOR_RGB2GRAY);
Imgproc.equalizeHist(middle, middle);
Imgproc.threshold(middle, middle, 150, 255, Imgproc.THRESH_OTSU);
Please tell me how can I obtain better results in opencv, and also how to count the dents? I tried findcontour() but it gives a very large number. I tried on other images as well, but I'm not getting proper results.
Please help.
So you basically from the MATLAB site, imtophat does - Top-hat filtering computes the morphological opening of the image (using imopen) and then subtracts the result from the original image.
You could do this in OpenCV with the following steps:
Step 1: Get the disk structuring element
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15, 15))
Step 2: Compute opening of the image and then subtract the result from the original image
tophat = cv2.morphologyEx(v, cv2.MORPH_TOPHAT, kernel)
This gives following result -
Step 3 - Now you could just manually threshold it or use Otsu -
ret, thresh = cv2.threshold(tophat, 17, 255, 0)
which gives you the following image -
Since the OP wants the code in Java, here is the probable code in Java:
private Mat topHat(Mat image)
{
Mat element = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(15, 15), new Point (0, 0));
Mat dst = new Mat;
Imgproc.morphologyEx(image, dst, Imgproc.MORPH_TOPHAT, element, new Point(0, 0));
return dst;
}
Make sure you do this on a gray scale image (CvType.8UC1) and then you can threshold suitably.
I am searching for how to extract a digital number from the image in android. I have to take a picture then i need to get numbers from image. OpenCV is a option . can we convert opencv into android ? Kindly suggest me any proper way. I will be grateful to you.
There are many OCR for Android
check there links
https://github.com/rmtheis/android-ocr
https://github.com/GautamGupta/Simple-Android-OCR
http://www.abbyy.com/mobileocr/android/
best OCR (Optical character recognition) example in android
OpenCV supports Android platform. You have to set up OpenCV4Android, it's instructions step by step here.
http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/O4A_SDK.html
However OpenCV is not an option but only a step. Then you have to use a character recognition engine. Most popular one is Tesseract-ocr. But it is really not easy task.
Also, they often recognize all characters. If you could achieve it, extracting the digits will be the easiest part in Java.
this works for me you just need to specify the number size` ArrayList output=new ArrayList<>();
cvtColor(input,input,COLOR_BGRA2GRAY);
Mat img_threshold = new Mat();
threshold(input, img_threshold, 60, 255,THRESH_BINARY_INV);
Mat img_contours =copy(img_threshold);
//Find contours of possibles characters
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
findContours(img_contours, contours,new Mat(), Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE); // all pixels of each contours
contours=sort(contours);
// Draw blue contours on a white image
Mat result=copy(img_threshold);
cvtColor(result, result, COLOR_GRAY2BGR);
drawContours(result,contours,
-1, // draw all contours
new Scalar(0,0,255), // in blue
1); // with a thickness of 1
//Start to iterate to each contour founded
ListIterator<MatOfPoint> itc = contours.listIterator();
//Remove patch that are no inside limits of aspect ratio and area.
while (itc.hasNext())
{
//Create bounding rect of object
MatOfPoint mp = new MatOfPoint(itc.next().toArray());
Rect mr = boundingRect(mp);
rectangle(result,new Point(mr.x,mr.y),new Point(mr.x+mr.width,mr.y+mr.height),new Scalar(0,255,0));
Mat auxRoi=new Mat(img_threshold,mr);
if (OCR_verifySizes(auxRoi))
{
output.add(preprocessChar(auxRoi));
}
}
return output;`
I come from the Qt world and i am porting an application to Android. I am bit confused, i am banging my head for a few days now on something that must be so trivial that i cannot find why it's not working.
Some background: i have a C++ engine which i use trough NDK and JNI. This engine creates some bitmaps and passes them to the Java side, the Java side must display them on a View and let the user interact with them (drag and such).
The engine works properly, because i use it under Qt with full success. This is the workflow:
1- Java loads a big Bitmap from a custom data file (the C++ engine expects it to be in ARGB format, but it's compressed JPG data)
Bitmap.Config fmt = Bitmap.Config.ARGB_8888;
Bitmap bitmap = BitmapFactory.decodeByteArray(buffer, 0, size).copy( fmt , false);
2- initialize the C++ engine passing the bitmap. The C++ engine "breaks" the bitmap in smaller tiles. For tile it builds a rather complex alpha mask and stores it into the first byte of the bitmap (the "a" byte). This alpha mask only uses two values: 0xFF for opaque and 0x00 for transparent.
init_C_engine( this.fullImage );
3- The Java side then allocates all the tiles bitmaps, i do in two steps because before init i dont know which size will the tiles be. The engine will populate the tile_width and tile_height arrays:
Bitmap.Config fmt = Bitmap.Config.ARGB_8888;
for (int t = 0; t < this.puzzle_size; t++ ){
tile_data[ t ] = Bitmap.createBitmap( tile_width[t], tile_height[t], fmt);
4- Last step,inside the C++ engine, all the tiles bitmaps are filled:
for ( int n = 0; n < nBitmaps; n++ )
{
jobject bitmap = env->GetObjectArrayElement( bitmaps, n );
AndroidBitmap_getInfo(env, bitmap, &info);
AndroidBitmap_lockPixels(env, bitmap, reinterpret_cast<void **>(&pixels));
game->getTileBitmap( n, (unsigned char*)pixels );
AndroidBitmap_unlockPixels(env, bitmap);
env->SetObjectArrayElement( bitmaps, n, bitmap );
}
}
Now, in my custom View:
protected void onDraw(Canvas canvas) {
super.onDraw(canvas);
canvas.drawColor(Color.BLACK);
for ( int tile = 0; tile < board.nTiles; tile++ ){
canvas.drawBitmap( tile_data[tile],
tile_x[tile],
tile_y[tile], paint);
}
}
What i expect is that on my View i see my tiles with transparent areas. what i get instead is a weird behaviour so that on the black background i see the ENTIRE tile like the alpha bytes are all set to opaque, but when i move the tiles one of top of the other, the "transparent" areas get combined in some strange way, like colors are "xor"ed or multiplied in some way! When i move one tile on the other i can see the areas where the alpha bytes are set to transparent but colors gets mangled instead of being transparend!
Basically i expect that pixels having alpha set to 0 are drawn as transparent... i looked on internet but i could not find anything usefull to help me out....
Does somebody have ideas? Anything will be appreciated!
thanks.
Shouldn't you use the index t instead of tile inside the for loop inside onDraw? Like this:
canvas.drawBitmap(tile_data[t], tile_x[t], tile_y[t], paint);