Related
The question comes 1st: I am looking for FAST approach to match images.
Now, the use case: I am developing a detector to detect orb on a 6x5 Match-3 game board for android platform. I have an array of the orb icon with transparent background, but the orb on the screen (screenshot) has different background color, probably different size too. I have to compare each orb on the screen with my array of icons (69 icons specifically) so it's a 69x30=2070 steps. I tried lazy implementation and group almost similar icon together to reduce the steps but still take a long time (10s at most) for computation. I also tried checking the channel and depth of image, resizing the images to have same size and tweaking the threshold value but still no luck.
I have tried Histogram Matching (seperate channel, grayscale), Template Matching (CCOEFF, SQDIFF, CCORR), AKAZE, ORB(unbounded, bounded), PHash all using OpenCV but histogram matching and PHash give me erroneous result (too much false positive), Template Matching consume 10s+ (considered too slow for user to wait) while AKAZE and ORB give better result than all other methods but still needs 6s+ per try. Is there any other method that can helps me cut down the computation time down to somewhere near 1s and can give better result considering the worst case scenario is 2070 steps?
Referrences that I have read that compares the performances of different feature matching algorithms:
A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. It shows that ORB and BRISK should be averagely better than the other approach compared while AKAZE is moderately good for most cases. I deleted my Histogram comparison code as it is not really helpful but you may find the rest of it below.
Mat source = Utils.loadResource(this, R.drawable.orb_icon, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
Mat tmp = new Mat();
Bitmap cropped_img = Bitmap.createBitmap(screenshot, x, y, width, height);
Utils.bitmapToMat(cropped_img, tmp);
//template matching code
int r_rows = source.rows() - tmp.rows() + 1;
int r_cols = source.cols() - tmp.cols() + 1;
Mat result = new Mat();
result.create(r_rows, r_cols, CvType.CV_32F);
Imgproc.matchTemplate(source, tmp, result, Imgproc.TM_CCOEFF_NORMED);
Core.MinMaxLocResult mmr = Core.minMaxLoc(result);
double maxVal = mmr.maxVal;
return maxVal;
//AKAZE
MatOfKeyPoint kp1 = new MatOfKeyPoint();
MatOfKeyPoint kp2 = new MatOfKeyPoint();
Mat desc1 = new Mat();
Mat desc2 = new Mat();
AKAZE akaze = AKAZE.create();
DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
akaze.detectAndCompute(source, new Mat(), kp1, desc1);
akaze.detectAndCompute(tmp, new Mat(), kp2, desc2);
List<MatOfDMatch> knnMatches = new ArrayList<>();
matcher.knnMatch(desc1, desc2, knnMatches, 2);
float threshold = 0.7f;
int count = 0;
for(int i=0; i<knnMatches.size(); i++) {
if(knnMatches.get(i).rows() > 1) {
DMatch[] matches = knnMatches.get(i).toArray();
if(matches[0].distance < threshold * matches[1].distance) {
count++;
}
}
}
//ORB
ORB orb = ORB.create();
DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
MatOfKeyPoint kp1 = new MatOfKeyPoint();
MatOfKeyPoint kp2 = new MatOfKeyPoint();
Mat desc1 = new Mat();
Mat desc2 = new Mat();
orb.detectAndCompute(source, new Mat(), kp1, desc1);
orb.detectAndCompute(tmp, new Mat(), kp2, desc2);
List<MatOfDMatch> knnMatches = new ArrayList<>();
matcher.knnMatch(desc1, desc2, knnMatches, 2);
float threshold = 0.8f;
int count = 0;
for(int i=0; i<knnMatches.size(); i++) {
if(knnMatches.get(i).rows() > 1) {
DMatch[] matches = knnMatches.get(i).toArray();
if(matches[0].distance < threshold * matches[1].distance) {
count++;
}
}
}
//PHash
Mat hash_source = new Mat();
Mat hash_tmp = new Mat();
Img_hash.pHash(tmp, hash_tmp);
Img_hash.pHash(source, hash_source);
Core.norm(source, tmp, Core.NORM_HAMMING);
Edit: As suggested, below is the game board, icon image, and orb screenshot sample.
ICON vs orb screenshot
Also, you may observe the simulation result of each approach by comparing the result(overlay smaller icon) on top of the orb on board:
Histogram Matching
,
Template Matching
and
AKAZE (similar to ORB)
After moving the variable initialization out of my comparison function to base class, detect keypoint and PHash of source icon images on class initialization, run detect and compute function in batch using List to reduce individual function call. It still takes up 4s+ for the image matching process. Time consumption is reduced but accuracy is still a major problem. You may observe my heap stack on below.
I'm a student and I'm working on application that will scann a sudoku and solve it. I'm taking picture, than finding biggest contour. This is what is working.Problem begin when I want to extract that biggest counter on the empty mat (it has white backround), application don't show activity with picture that should (in other images it does) but it return to my mainactivity. I was using this tutorial for the extraction: https://bytefish.de/blog/extracting_contours_with_opencv/.
mat4=mat1; // mat 1 is current frame on camera
transpose(mat4, mat4);
flip(mat4, mat4, +1);
mat5=mat4;
Mat okraje = new Mat();
Mat hiearchy = new Mat();
Imgproc.cvtColor(mat5,mat5,Imgproc.COLOR_BGR2GRAY);
List<MatOfPoint> contourList = new ArrayList<MatOfPoint>();
Imgproc.Canny(mat5,okraje,80,100);
Imgproc.findContours(okraje,contourList,hiearchy,Imgproc.RETR_CCOMP, Imgproc.CHAIN_APPROX_SIMPLE); // TOPKA JEBNE PLNY STVOREC A KRATKO!!
for(int ab=0;ab < contourList.size(); ab++ ){
a = contourArea(contourList.get(ab),false);
if(a>largest_area){
b=ab;
largest_area = a;
largest_contour_index=ab;
bounding_rect=boundingRect(contourList.get(ab));
}
}
Mat len_sudkoku = new Mat();
len_sudkoku.create(mat5.rows(), mat5.cols(),CvType.CV_8UC3);
len_sudkoku.setTo(new Scalar(255,255,255));
Mat lskere = new Mat();
lskere.create(okraje.cols(), okraje.rows(), CvType.CV_8UC1);
Random r = new Random();
Imgproc.drawContours( lskere, contourList,largest_contour_index, new Scalar(r.nextInt(255), r.nextInt(255), r.nextInt(255)), -1);
mat5.copyTo(len_sudkoku,lskere); // pada to!!
Bitmap bm = Bitmap.createBitmap(len_sudkoku.cols(),len_sudkoku.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(len_sudkoku,bm);
setContentView(R.layout.activity_fotka_ukaz);
ImageView IMW = findViewById(R.id.imageView);
IMW.setImageBitmap(bm);
I expected to be like in tutorial that I posted here where a man extracted an apple and place it on another backround. Thing that I notice is that application return to main activity (it should display image), when I'm using a command
mat5.copyTo(len_sudkoku, lskere)
Ok guys i solved it. The problem wasn't in command copy to (obviously) but in declaration size of different mat and bitmap. The problem was that in declaration first input was sometimes row and column so it crashed. Now I add size to Mat hierarchy and okraje, and edit lskere.create(okraje.cols(), okraje.rows(), to lskere.create(okraje.rows(), okraje.cols()..
I need to align different images in my android application, using the OpenCV library. I found a solution in this thread.
public static Bitmap alignImagesHomography(Bitmap A, Bitmap B)
{
final int warp_mode = MOTION_HOMOGRAPHY;
Mat matA = new Mat(A.getHeight(), A.getWidth(), CvType.CV_8UC3);
Mat matAgray = new Mat(A.getHeight(), A.getWidth(), CvType.CV_8U);
Mat matB = new Mat(B.getHeight(), B.getWidth(), CvType.CV_8UC3);
Mat matBgray = new Mat(B.getHeight(), B.getWidth(), CvType.CV_8U);
Mat matBaligned = new Mat(A.getHeight(), A.getWidth(), CvType.CV_8UC3);
Mat warpMatrix = Mat.eye(3, 3, CV_32F);
Utils.bitmapToMat(A, matA);
Utils.bitmapToMat(B, matB);
Imgproc.cvtColor(matA, matAgray, Imgproc.COLOR_BGR2GRAY);
Imgproc.cvtColor(matB, matBgray, Imgproc.COLOR_BGR2GRAY);
int numIter = 5;
double terminationEps = 1e-10;
TermCriteria criteria = new TermCriteria(TermCriteria.COUNT + TermCriteria.EPS, numIter, terminationEps);
findTransformECC(matAgray, matBgray, warpMatrix, warp_mode, criteria, matBgray);
Imgproc.warpPerspective(matA, matBaligned, warpMatrix, matA.size(), Imgproc.INTER_LINEAR + Imgproc.WARP_INVERSE_MAP);
Bitmap alignedBMP = Bitmap.createBitmap(A.getWidth(), A.getHeight(), Bitmap.Config.RGB_565);
Utils.matToBitmap(matBaligned, alignedBMP);
return alignedBMP;
}
public static Bitmap alignImagesEuclidean(Bitmap A, Bitmap B)
{
final int warp_mode = MOTION_EUCLIDEAN;
Mat matA = new Mat(A.getHeight(), A.getWidth(), CvType.CV_8UC3);
Mat matAgray = new Mat(A.getHeight(), A.getWidth(), CvType.CV_8U);
Mat matB = new Mat(B.getHeight(), B.getWidth(), CvType.CV_8UC3);
Mat matBgray = new Mat(B.getHeight(), B.getWidth(), CvType.CV_8U);
Mat matBaligned = new Mat(A.getHeight(), A.getWidth(), CvType.CV_8UC3);
Mat warpMatrix = Mat.eye(2,3,CV_32F);
Utils.bitmapToMat(A, matA);
Utils.bitmapToMat(B, matB);
Imgproc.cvtColor(matA, matAgray, Imgproc.COLOR_BGR2GRAY);
Imgproc.cvtColor(matB, matBgray, Imgproc.COLOR_BGR2GRAY);
int numIter = 5;
double terminationEps = 1e-10;
TermCriteria criteria = new TermCriteria(TermCriteria.COUNT + TermCriteria.EPS, numIter, terminationEps);
findTransformECC(matAgray, matBgray, warpMatrix, warp_mode, criteria, matBgray);
Imgproc.warpAffine(matA, matBaligned, warpMatrix, matA.size(), Imgproc.INTER_LINEAR + Imgproc.WARP_INVERSE_MAP);
Bitmap alignedBMP = Bitmap.createBitmap(A.getWidth(), A.getHeight(), Bitmap.Config.RGB_565);
Utils.matToBitmap(matBaligned, alignedBMP);
return alignedBMP;
}
public static Bitmap alignExposures(Bitmap A, Bitmap B) {
Mat matA = new Mat(A.getHeight(), A.getWidth(), CvType.CV_8UC3);
Mat matB = new Mat(B.getHeight(), B.getWidth(), CvType.CV_8UC3);
Utils.bitmapToMat(A, matA);
Utils.bitmapToMat(B, matB);
List<Mat> src = new ArrayList<>();
src.add(matA);
src.add(matB);
Bitmap output = Bitmap.createBitmap(A.getWidth(),A.getHeight(), Bitmap.Config.RGB_565);
AlignMTB align = createAlignMTB(8, 4, false);
align.process(src,src);
for(int i = 1; i < src.size(); i++) {
add(src.get(0),src.get(i),src.get(0));
}
Utils.matToBitmap(src.get(0),output);
return output;
}
I tried all the three methods written by the user wegenerEDV. Anyway, the first two methods return the same picture as the "Bitmap A" given as input; the third method actually aligns the pictures, but the resulting image is overexposed:
original: https://i.imgur.com/cknHM23.jpg
aligned: https://i.imgur.com/kXCQl6x.jpg
Has anybody found a different solution? Or do these methods actually work and I am doing something wrong?
The best solution to me is to correct the alignImagesHomography method. It actually does something, because it takes around 30 seconds to process the final picture, but then it is exactly equal to the input image.
I've never used findTransformECC(), used by your first and second methods, and I'm not familiar with that algorithm. The only difference between those two methods is the type of transform findTransformECC() is asked to find; homographic transforms are a superset of Euclidean transforms so the first method (using MOTION_HOMOGRAPHY) would be most robust for your use case, though it might also be slower.
the first two methods return the same picture as the "Bitmap A" given as input
If these two methods are working correctly, the result should look almost identical to Image A, even though it is produced from pixels of Image B. Have you checked that the result is bitwise identical to Bitmap A, and does not just look similar?
I think I can see the same bug in both methods: findTransformECC() finds a mapping from matBgray onto matAgray (see the docs), but warpPerspective() and warpAffine(), respectively, are used to apply the resulting transform to matA, storing the result in matBaligned; they should be applied to matB. The only way I can see that you would get a result that is bitwise identical to Image A is if your homography calculation fails, so that the result homography ("warpMatrix") still contains its initial state, which is the identity matrix. The identity homography, incorrectly applied to matA (because of the above bug), will of course give another exact copy of matA in matBaligned. You could check this by printing warpMatrix after calculation and seeing if it is the identity matrix. You should also add some error checking, because at the moment you don't know whether the methods you call are failing at all, or why (eg. bad input parameters, unable to find any correspondences, etc etc).
Your third method, alignExposures(), uses AlignMTB which is intended for HDR imaging, and I don't know how it does alignment. It may only handle 2D translation. The loop in that method is adding the output images back onto one of the source images, so it will saturate to white. If the result you want is the aligned images averaged together (is that what you want?) you should create a new output matrix with a larger datatype (eg. CV_16UC3) to acclumulate the result images in and calculate the average, then use cv::convertTo() to reduce that buffer back to the original datatype (eg. CV_8UC3).
Another algorithm I have used successfully for homographic alignment is:
Create a feature detector, eg. SIFT, from the features2d or xfeatures2d library. I used AKAZE, because SIFT is patented.
Detect features and descriptors in both your images using detectAndCompute().
Create a BFMatcher and perform brute force matching of feature points from both images using match().
If you want to at this point, you can print the matching feature points, draw them onto the images and view them, etc etc.
Organize the found correspondences into two lists (source and destination points) suitable for findHomography().
Call findHomography() using the RANSAC flag.
Apply the homography using warpPerspective().
With this method there are more opportunities to inspect and validate your data, which will help with debugging.
I am new to OpenCV and am trying to count the number of objects in an image. I have done this before using MATLAB Image Processing Toolbox and adapted the same approach in OpenCV (Android) also.
The first step was to convert an image to gray scale. Then to threshold it and then counting the number of blobs. In Matlab there is a command - "bwlabel", which gives the number of blobs. I couldn't find such thing in OpenCV (again, I am a noob in OpenCV as well as Android).
Here is my code,
//JPG to Bitmap to MAT
Bitmap i = BitmapFactory.decodeFile(imgPath + "mms.jpg");
Bitmap bmpImg = i.copy(Bitmap.Config.ARGB_8888, false);
Mat srcMat = new Mat ( bmpImg.getHeight(), bmpImg.getWidth(), CvType.CV_8UC3);
Utils.bitmapToMat(bmpImg, srcMat);
//convert to gray scale and save image
Mat gray = new Mat(srcMat.size(), CvType.CV_8UC1);
Imgproc.cvtColor(srcMat, gray, Imgproc.COLOR_RGB2GRAY,4);
//write bitmap
Boolean bool = Highgui.imwrite(imgPath + "gray.jpg", gray);
//thresholding
Mat threshed = new Mat(bmpImg.getWidth(),bmpImg.getHeight(), CvType.CV_8UC1);
Imgproc.adaptiveThreshold(gray, threshed, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 75, 5);//15, 8 were original tests. Casey was 75,10
Core.bitwise_not(threshed, threshed);
Utils.matToBitmap(threshed, bmpImg);
//write bitmap
bool = Highgui.imwrite(imgPath + "threshed.jpg", threshed);
Toast.makeText(this, "Thresholded image saved!", Toast.LENGTH_SHORT).show();
In the next step, I tried to fill the holes and letters using dilation followed by an erosion but the blobs gets attached to each other which will ultimately give a wrong count. There is a tradeoff between filling holes and getting the blobs attached to each other on tuning the parameters for dilation and erosion.
Here is the code,
//morphological operations
//dilation
Mat dilated = new Mat(bmpImg.getWidth(),bmpImg.getHeight(), CvType.CV_8UC1);
Imgproc.dilate(threshed, dilated, Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new org.opencv.core.Size (16, 16)));
Utils.matToBitmap(dilated, bmpImg);
//write bitmap
bool = Highgui.imwrite(imgPath + "dilated.jpg", dilated);
Toast.makeText(this, "Dilated image saved!", Toast.LENGTH_SHORT).show();
//erosion
Mat eroded = new Mat(bmpImg.getWidth(),bmpImg.getHeight(), CvType.CV_8UC1);
Imgproc.erode(dilated, eroded, Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new org.opencv.core.Size(15, 15)));
Utils.matToBitmap(eroded, bmpImg);
//write bitmap
bool = Highgui.imwrite(imgPath + "eroded.jpg", eroded);
Toast.makeText(this, "Eroded image saved!", Toast.LENGTH_SHORT).show();
Because sometimes my M&Ms might be just next to each other! ;)
I also tried to use Hough Circles but the result is very unreliable (tested with coin images as well as real coins)
Here is the code,
//hough circles
Mat circles = new Mat();
// parameters
int iCannyUpperThreshold = 100;
int iMinRadius = 20;
int iMaxRadius = 400;
int iAccumulator = 100;
Imgproc.HoughCircles(gray, circles, Imgproc.CV_HOUGH_GRADIENT,
1.0, gray.rows() / 8, iCannyUpperThreshold, iAccumulator,
iMinRadius, iMaxRadius);
// draw
if (circles.cols() > 0)
{
Toast.makeText(this, "Coins : " +circles.cols() , Toast.LENGTH_LONG).show();
}
else
{
Toast.makeText(this, "No coins found", Toast.LENGTH_LONG).show();
}
The problem with this approach is that the algorithm is limited to perfect circles only (AFAIK). So, it doesn't work well when I try to scan and count M&Ms or coins lying on my desk (because angle of the device changes). With this approach, sometimes I get less no. of coins detected and sometimes more (I don't get it why more??).
On scanning this image the app sometimes shows 19 coins and sometimes 38 coins counted...I know there are other features which may be detected as circles but I totally don't get it why 38..?
So my questions...
Is there a better way to fill holes without joining adjacent blobs?
How do I count the number of objects accurately? I don't want to limit my app to counting only circles with HoughCircles approach.
FYI : OpenCV-2.4.9-android-sdk. Kindly keep in mind that I am a newbie in OpenCV and Android too.
Any help is much appreciated.
Thanks & Cheers!
Jainam
So to proceed we take your threshold image which you have generated as input and further modify it. The present code is in C++ but I guess you can easily convert it into android platform
Now instead of dilation or blurring you can try flood fill
which results in
Finally now applying the contour detection algorithm algorithm we get
The code for the above is
Mat dst = imread($path to the threshold image); // image should be single channel black and white image
imshow("dst",dst);
cv::Mat mask = cv::Mat::zeros(dst.rows + 2, dst.cols + 2, CV_8U);
// A image with size greater than the present object is created
cv::floodFill(dst, mask, cv::Point(0,0), 255, 0, cv::Scalar(), cv::Scalar(), 4 + (255 << 8) + cv::FLOODFILL_MASK_ONLY);
erode(mask,mask,Mat());
// Now to remove the outer boundary
rectangle(mask,Rect(0,0,mask.cols,mask.rows), Scalar(255,255,255),2,8,0);
imshow("Mask",mask);
Mat copy;
mask.copyTo(copy);
vector<vector<Point> > contours;
vector<Vec4i> hierarchy;
findContours( copy, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, Point(0, 0) );
vector<vector<Point> > contours_poly( contours.size() );
vector<Rect> boundRect( contours.size() );
vector<Point2f>center( contours.size() );
vector<float>Distance( contours.size() );
vector<float>radius( contours.size() );
Mat drawing = cv::Mat::zeros(mask.rows, mask.cols, CV_8U);
int num_object = 0;
for( int i = 0; i < contours.size(); i++ ){
approxPolyDP( Mat(contours[i]), contours_poly[i], 3, true );
// To get rid of the smaller object and the outer rectangle created
//because of the additional mask image we enforce a lower limit on area
//to remove noise and an upper limit to remove the outer border.
if (contourArea(contours_poly[i])>(mask.rows*mask.cols/10000) && contourArea(contours_poly[i])<mask.rows*mask.cols*0.9){
boundRect[i] = boundingRect( Mat(contours_poly[i]) );
minEnclosingCircle( (Mat)contours_poly[i], center[i], radius[i] );
circle(drawing,center[i], (int)radius[i], Scalar(255,255,255), 2, 8, 0);
rectangle(drawing,boundRect[i], Scalar(255,255,255),2,8,0);
num_object++;
}
}
cout <<"No. of object detected =" <<num_object<<endl;
imshow("drawing",drawing);
waitKey(2);
char key = (char) waitKey(20);
if(key == 32){
// You can save your images here using a space
}
I hope this helps you in solving your problem
Just check it out,
Blur source.
Threshold binary inverted on gray.
Find contours, note that you should use CV_RETR_EXTERNAL as contour retrieval mode.
You can take the contours size as your object count.
Code:
Mat tmp,thr;
Mat src=imread("img.jpg",1);
blur(src,src,Size(3,3));
cvtColor(src,tmp,CV_BGR2GRAY);
threshold(tmp,thr,220,255,THRESH_BINARY_INV);
imshow("thr",thr);
vector< vector <Point> > contours; // Vector for storing contour
vector< Vec4i > hierarchy;
findContours( thr, contours, hierarchy,CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE ); // Find the contours in the image
for( int i = 0; i< contours.size(); i=hierarchy[i][0] ) // iterate through each contour.
{
Rect r= boundingRect(contours[i]);
rectangle(src,r, Scalar(0,0,255),2,8,0);
}
cout<<"Numeber of contour = "<<contours.size()<<endl;
imshow("src",src);
waitKey();
I'm pretty new to openCV, and at the moment im tryin to get an object detection running on an android device. What i basically do, is displaying a a camera preview in my app, and when i click on it it captures a picture. Then this picture is given to the runFeatureHomography - method, which first grabs the second image to which the taken image has to be compared. Then the method finds the keypoints in both pictures, computes them and matches them into one Mat called img_matches.
As basic as it can be i guess.
The object im trying to detect here at the moment is some kind of card, just like the format of a credit card. The card is blue and has lots of white and yellow text on it. I can only post one link, thats why i cant show pictures of them.
I dont know why, but when i display the result in the end / or save the result as a bitmap to my phone, it always looks kinda like this:
http://oi44.tinypic.com/oaqel0.jpg <-- result image after everything is done.
This shows me that the object i wanted to detect was indeed detected, but i dont know why there is a black background and not the pictures of the cards. Why doesn't it show my two images the way the are, just with all the lines on them?
In my code im using those three:
FeatureDetector detector = FeatureDetector.create(FeatureDetector.ORB);
DescriptorExtractor extractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
DescriptorMatchermatcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE);
Here is some of my code:
private void runFeatureHomography(Bitmap image)
{
Mat img_object = getObjectImage();
Mat img_scene = newEmptyMat();
Bitmap myimg = image.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(myimg, img_scene);
MatOfKeyPoint keyPoints_object = detectObjectKeyPoints();
MatOfKeyPoint keyPoints_scene = new MatOfKeyPoint();
this.detector.detect(img_scene, keyPoints_scene);
Mat descriptors_object = calculateObjectDescriptor();
Mat descriptors_scene = newEmptyMat();
this.extractor.compute(img_scene, keyPoints_scene, descriptors_scene);
MatOfDMatch matches = new MatOfDMatch();
matcher.match(descriptors_object, descriptors_scene, matches);
double min_dist = Double.MAX_VALUE;
for (int i = -1; ++i < descriptors_object.rows();)
{
double dist = matches.toArray()[i].distance;
if (dist < min_dist)
{
min_dist = dist;
}
}
List<DMatch> good_matches = new ArrayList<DMatch>();
for (int i = -1; ++i < descriptors_object.rows();)
{
if (matches.toArray()[i].distance <= 3 * min_dist)
{
good_matches.add(matches.toArray()[i]);
}
}
System.out.println("4");
Mat img_matches = newEmptyMat();
Features2d.drawMatches(
img_object,
keyPoints_object,
img_scene,
keyPoints_scene,
new MatOfDMatch(good_matches.toArray(new DMatch[good_matches
.size()])), img_matches, Scalar.all(-1),
Scalar.all(-1), new MatOfByte(),
Features2d.NOT_DRAW_SINGLE_POINTS);
List<Point> object = new ArrayList<Point>();
List<Point> scene = new ArrayList<Point>();
for (int i = -1; ++i < good_matches.size();)
{
object.add(keyPoints_object.toArray()[good_matches.get(i).queryIdx].pt);
scene.add(keyPoints_scene.toArray()[good_matches.get(i).trainIdx].pt);
}
Mat H = Calib3d.findHomography(
new MatOfPoint2f(object.toArray(new Point[object.size()])),
new MatOfPoint2f(scene.toArray(new Point[scene.size()])),
Calib3d.RANSAC, 3);
Point[] object_corners = new Point[4];
object_corners[0] = new Point(0, 0);
object_corners[1] = new Point(img_object.cols(), 0);
object_corners[2] = new Point(img_object.cols(), img_object.rows());
object_corners[3] = new Point(0, img_object.rows());
MatOfPoint2f scene_corners2f = new MatOfPoint2f();
Core.perspectiveTransform(new MatOfPoint2f(object_corners),
scene_corners2f, H);
Point[] scene_corners = scene_corners2f.toArray();
Point[] scene_corners_norm = new Point[4];
scene_corners_norm[0] = new Point(scene_corners[0].x
+ img_object.cols(), scene_corners[0].y);
scene_corners_norm[1] = new Point(scene_corners[1].x
+ img_object.cols(), scene_corners[1].y);
scene_corners_norm[2] = new Point(scene_corners[2].x
+ img_object.cols(), scene_corners[2].y);
scene_corners_norm[3] = new Point(scene_corners[3].x
+ img_object.cols(), scene_corners[3].y);
Core.line(img_matches, scene_corners_norm[0], scene_corners_norm[1],
new Scalar(0, 255, 0), 4);
Core.line(img_matches, scene_corners_norm[1], scene_corners_norm[2],
new Scalar(0, 255, 0), 4);
Core.line(img_matches, scene_corners_norm[2], scene_corners_norm[3],
new Scalar(0, 255, 0), 4);
Core.line(img_matches, scene_corners_norm[3], scene_corners_norm[0],
new Scalar(0, 255, 0), 4);
bmp = Bitmap.createBitmap(img_matches.cols(), img_matches.rows(),
Bitmap.Config.ARGB_8888);
Intent resultIntent = new Intent("com.example.capturetest.Result");
startActivity(resultIntent);
}
private volatile Mat cachedObjectDescriptor = null;
private volatile MatOfKeyPoint cachedObjectKeyPoints = null;
private volatile Mat cachedObjectImage = null;
private Mat calculateObjectDescriptor()
{
Mat objectDescriptor = this.cachedObjectDescriptor;
if (objectDescriptor == null)
{
Mat objectImage = getObjectImage();
MatOfKeyPoint objectKeyPoints = detectObjectKeyPoints();
objectDescriptor = newEmptyMat();
this.extractor.compute(objectImage, objectKeyPoints,
objectDescriptor);
this.cachedObjectDescriptor = objectDescriptor;
}
return objectDescriptor;
}
private MatOfKeyPoint detectObjectKeyPoints()
{
MatOfKeyPoint objectKeyPoints = this.cachedObjectKeyPoints;
if (objectKeyPoints == null)
{
Mat objectImage = getObjectImage();
objectKeyPoints = new MatOfKeyPoint();
this.detector.detect(objectImage, objectKeyPoints);
this.cachedObjectKeyPoints = objectKeyPoints;
}
return objectKeyPoints;
}
private Mat getObjectImage()
{
Mat objectImage = this.cachedObjectImage;
if (objectImage == null)
{
objectImage = newEmptyMat();
Bitmap bitmap = ((BitmapDrawable) iv.getDrawable()).getBitmap();
Bitmap img = bitmap.copy(Bitmap.Config.ARGB_8888, false);
Utils.bitmapToMat(img, objectImage);
this.cachedObjectImage = objectImage;
}
return objectImage;
}
private Mat newEmptyMat()
{
return new Mat();
}
After this line matcher.match(descriptors_object, descriptors_scene, matches); i tried to convert the three Mat img_object, img_scene and matches to bitmaps and saved them to my android device just for checking. They all look as they are supposed to look, so until this point everthing is fine.
But after this part...
Mat img_matches = newEmptyMat();
Features2d.drawMatches(
img_object,
keyPoints_object,
img_scene,
keyPoints_scene,
new MatOfDMatch(good_matches.toArray(new DMatch[good_matches
.size()])), img_matches, Scalar.all(-1),
Scalar.all(-1), new MatOfByte(),
Features2d.NOT_DRAW_SINGLE_POINTS);
... i tried to convert the Mat img_matches (which is supposed to have all the information of the two input pictures if i get it right), to a bitmap and save it on my android device, but the picture looks like the picture in the link above (black pictures with lines instead of card-pictures with lines).
Does any of you know what im doing wrong here? I seem to be stuck at the moment.
Thanks in advance guys.
Edit:
Just wanted to let you know that i got the same code running and WORKING as a normal java program on my desktop. The picture is taken from the webcam there. The result image is diplayed absolutely correct in the desktop program, with cards and lines instead of black and lines ;)
Alright, found a working way:
Imgproc.cvtColor(img_object, img_object, Imgproc.COLOR_RGBA2RGB);
Imgproc.cvtColor(img_scene, img_scene, Imgproc.COLOR_RGBA2RGB);
It seems that after i converted my Bitmaps to Mats i have to use the above two lines to convert them from RGBA to RGB. It also works with RGBA to GRAY if you prefer gray pictures.
It seems that RGBA format is not working in this case.
Hope this helps anybody coming here from google.