its may be a simple / stupid question, but I have a conversion problem in opencv (android).
my goal is to calculate the fundamentalMatrix out of corresponding matches from two consecutive images.
i programmed this so far (and working):
detector.detect(actImg, actKP);
detector.detect(prevImg, prevKP);
descExtractor.compute(prevImg, prevKP, descriptorPrev);
descExtractor.compute(actImg, actKP, descriptorAct);
descMatcher.match(descriptorPrev, descriptorAct, matches);
Features2d.drawMatches(prevImg, prevKP, actImg, actKP,matches, mRgba);
matches are of the type MatOfDMatch.
now i would calculate the fundamentalMatrix out of the points that matches against each other. therefor i must know which of the keypoints in the first image (prevKP) were found in the second image (actKP).
Mat fundamental_matrix = Calib3d.findFundamentalMat(nextPts, prevPts, Calib3d.FM_RANSAC,3, 0.99);
first question:
how can i extract / convert MatOfKeyPoints to MatOfPoint2f (that they can be passed to findFundamentalMatrix)
second question:
how to pass only the matched keypoints to the function findFundamentalMatrix.
is this a good way of doing it?
thanks a lot in advace!
EDIT
thanks a lot for your detailed response!
i wrote your code into two functions:
private MatOfPoint2f getMatOfPoint2fFromDMatchesTrain(MatOfDMatch matches2,
MatOfKeyPoint prevKP2) {
DMatch dm[] = matches2.toArray();
List<Point> lp1 = new ArrayList<Point>(dm.length);
KeyPoint tkp[] = prevKP2.toArray();
for (int i = 0; i < dm.length; i++) {
DMatch dmm = dm[i];
if (dmm.trainIdx < tkp.length)
lp1.add(tkp[dmm.trainIdx].pt);
}
return new MatOfPoint2f(lp1.toArray(new Point[0]));
}
private MatOfPoint2f getMatOfPoint2fFromDMatchesQuery(MatOfDMatch matches2,
MatOfKeyPoint actKP2) {
DMatch dm[] = matches2.toArray();
List<Point> lp2 = new ArrayList<Point>(dm.length);
KeyPoint qkp[] = actKP2.toArray();
for (int i = 0; i < dm.length; i++) {
DMatch dmm = dm[i];
if (dmm.queryIdx < qkp.length)
lp2.add(qkp[dmm.queryIdx].pt);
}
return new MatOfPoint2f(lp2.toArray(new Point[0]));
}
but when i am calling
prevPts = getMatOfPoint2fFromDMatchesTrain(matches, prevKP);
nextPts = getMatOfPoint2fFromDMatchesQuery(matches, actKP);
Mat fundamental_matrix = Calib3d.findFundamentalMat(
nextPts, prevPts, Calib3d.FM_RANSAC, 3, 0.99);
the problem is that i get the error -215.
the error:
error: (-215) npoints >= 0 && points2.checkVector(2) == npoints && points1.type() == points2.type() in function cv::Mat
cv::findFundamentalMat(...
i proved that prevPts and nextPts arend below 10 points (for ransac).
so i would guess that the error is that the points arend floating points. but i checked this with the debugger that these points are floating points.
your suggested codeline:
return new MatOfPoint2f(lp2.toArray(new Point[0]));
should convert the points to floating point or am i wrong?
thanks again
Unfortunately there is no better way (even in C++ API) than loop through all matches and copy values to new Mat (or vector).
In Java you can do it as follows:
DMatch dm[] = matches.toArray();
List<Point> lp1 = new ArrayList<Point>(dm.length);
List<Point> lp2 = new ArrayList<Point>(dm.length);
KeyPoint tkp[] = prevKP.toArray();
KeyPoint qkp[] = actKP.toArray();
for (int i = 0; i < dm.length; i++) {
DMatch dm = dm[i];
lp1.add(tkp[dm.trainIdx].pt);
lp2.add(qkp[dm.queryIdx].pt);
}
MatOfPoint2f pointsPrev = new MatOfPoint2f(lp1.toArray(new Point[0]));
MatOfPoint2f pointsAct = new MatOfPoint2f(lp2.toArray(new Point[0]));
Related
I recently started developing an app on Android studio and I just finished writing the code. The accuracy which I get is more than satisfactory but the time taken by the device is a lot. {}I followed some tutorials on how to monitor the performance on android studio and I saw that one small part of my code is taking 6 seconds, which half the time my app takes to display the entire result. I have seen a lot of posts Java OpenCV - extracting good matches from knnMatch , OpenCV filtering ORB matches on OpenCV/JavaCV but haven't come across anyone asking for this problem. The OpenCV link http://docs.opencv.org/2.4/doc/tutorials/features2d/feature_homography/feature_homography.html does provide a good tutorial but the RANSAC function in OpenCV takes different arguments for keypoints as compared to C++.
Here is my code
public Mat ORB_detection (Mat Scene_image, Mat Object_image){
/*This function is used to find the reference card in the captured image with the help of
* the reference card saved in the application
* Inputs - Captured image (Scene_image), Reference Image (Object_image)*/
FeatureDetector orb = FeatureDetector.create(FeatureDetector.DYNAMIC_ORB);
/*1.a Keypoint Detection for Scene Image*/
//convert input to grayscale
channels = new ArrayList<Mat>(3);
Core.split(Scene_image, channels);
Scene_image = channels.get(0);
//Sharpen the image
Scene_image = unsharpMask(Scene_image);
MatOfKeyPoint keypoint_scene = new MatOfKeyPoint();
//Convert image to eight bit, unsigned char
Scene_image.convertTo(Scene_image, CvType.CV_8UC1);
orb.detect(Scene_image, keypoint_scene);
channels.clear();
/*1.b Keypoint Detection for Object image*/
//convert input to grayscale
Core.split(Object_image,channels);
Object_image = channels.get(0);
channels.clear();
MatOfKeyPoint keypoint_object = new MatOfKeyPoint();
Object_image.convertTo(Object_image, CvType.CV_8UC1);
orb.detect(Object_image, keypoint_object);
//2. Calculate the descriptors/feature vectors
//Initialize orb descriptor extractor
DescriptorExtractor orb_descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);
Mat Obj_descriptor = new Mat();
Mat Scene_descriptor = new Mat();
orb_descriptor.compute(Object_image, keypoint_object, Obj_descriptor);
orb_descriptor.compute(Scene_image, keypoint_scene, Scene_descriptor);
//3. Matching the descriptors using Brute-Force
DescriptorMatcher brt_frc = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
MatOfDMatch matches = new MatOfDMatch();
brt_frc.match(Obj_descriptor, Scene_descriptor, matches);
//4. Calculating the max and min distance between Keypoints
float max_dist = 0,min_dist = 100,dist =0;
DMatch[] for_calculating;
for_calculating = matches.toArray();
for( int i = 0; i < Obj_descriptor.rows(); i++ )
{ dist = for_calculating[i].distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
System.out.print("\nInterval min_dist: " + min_dist + ", max_dist:" + max_dist);
//-- Use only "good" matches (i.e. whose distance is less than 2.5*min_dist)
LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
double ratio_dist=2.5;
ratio_dist = ratio_dist*min_dist;
int i, iter = matches.toArray().length;
matches.release();
for(i = 0;i < iter; i++){
if (for_calculating[i].distance <=ratio_dist)
good_matches.addLast(for_calculating[i]);
}
System.out.print("\n done Good Matches");
/*Necessary type conversion for drawing matches
MatOfDMatch goodMatches = new MatOfDMatch();
goodMatches.fromList(good_matches);
Mat matches_scn_obj = new Mat();
Features2d.drawKeypoints(Object_image, keypoint_object, new Mat(Object_image.rows(), keypoint_object.cols(), keypoint_object.type()), new Scalar(0.0D, 0.0D, 255.0D), 4);
Features2d.drawKeypoints(Scene_image, keypoint_scene, new Mat(Scene_image.rows(), Scene_image.cols(), Scene_image.type()), new Scalar(0.0D, 0.0D, 255.0D), 4);
Features2d.drawMatches(Object_image, keypoint_object, Scene_image, keypoint_scene, goodMatches, matches_scn_obj);
SaveImage(matches_scn_obj,"drawing_good_matches.jpg");
*/
if(good_matches.size() <= 6){
ph_value = "7";
System.out.println("Wrong Detection");
return Scene_image;
}
else{
//5. RANSAC thresholding for finding the optimum homography
Mat outputImg = new Mat();
LinkedList<Point> objList = new LinkedList<Point>();
LinkedList<Point> sceneList = new LinkedList<Point>();
List<org.opencv.core.KeyPoint> keypoints_objectList = keypoint_object.toList();
List<org.opencv.core.KeyPoint> keypoints_sceneList = keypoint_scene.toList();
//getting the object and scene points from good matches
for(i = 0; i<good_matches.size(); i++){
objList.addLast(keypoints_objectList.get(good_matches.get(i).queryIdx).pt);
sceneList.addLast(keypoints_sceneList.get(good_matches.get(i).trainIdx).pt);
}
good_matches.clear();
MatOfPoint2f obj = new MatOfPoint2f();
obj.fromList(objList);
objList.clear();
MatOfPoint2f scene = new MatOfPoint2f();
scene.fromList(sceneList);
sceneList.clear();
float RANSAC_dist=(float)2.0;
Mat hg = Calib3d.findHomography(obj, scene, Calib3d.RANSAC, RANSAC_dist);
for(i = 0;i<hg.cols();i++) {
String tmp = "";
for ( int j = 0; j < hg.rows(); j++) {
Point val = new Point(hg.get(j, i));
tmp= tmp + val.x + " ";
}
}
Mat scene_image_transformed_color = new Mat();
Imgproc.warpPerspective(original_image, scene_image_transformed_color, hg, Object_image.size(), Imgproc.WARP_INVERSE_MAP);
processing(scene_image_transformed_color, template_match);
return outputImg;
}
} }
and this part is what is taking 6 seconds to implement on runtime -
LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
double ratio_dist=2.5;
ratio_dist = ratio_dist*min_dist;
int i, iter = matches.toArray().length;
matches.release();
for(i = 0;i < iter; i++){
if (for_calculating[i].distance <=ratio_dist)
good_matches.addLast(for_calculating[i]);
}
System.out.print("\n done Good Matches");}
I was thinking may be I can write this part of the code in C++ using NDK but I just wanted to be sure that the language is the problem and not the code itself.
Please don't be strict, first question! Any criticism is much appreciated!
So the problem was the logcat was giving me false timing results. The lag was due to a Huge Gaussian Blur later on in the code. Instead of System.out.print, I used System.currentTimeMillis, which showed me the bug.
In the following code, I have carried out the following steps:
Loaded an image from sdcard.
Converted it to HSV format.
Used inRange function to mask out the red color.
Used findContours to find the contours.
Find the largest contour from those contours.
Created an ROI around the largest contour using boundingRect and submat functions.
Converted this ROI Mat to HSV format.
Iterated through the ROI Mat, and check for each pixel if it lies within the largest contour. I used the method pointPolygonTest to find this out, but it returns -1 for every pixel, as can be seen from the Log.i output I have pasted here. The question is why? How can I correct this.
private Scalar detectColoredBlob() {
rgbaFrame = Highgui.imread("/mnt/sdcard/DCIM/rgbaMat4Mask.bmp");
Mat hsvImage = new Mat();
Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/hsvImage.bmp", hsvImage);// check
Mat maskedImage = new Mat();
Core.inRange(hsvImage, new Scalar(0, 100, 100), new Scalar(10, 255, 255), maskedImage);
Highgui.imwrite("/mnt/sdcard/DCIM/maskedImage.bmp", maskedImage);// check
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(maskedImage, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
// \/ We will use only the largest contour. Other contours (any other possible blobs of this color range) will be ignored.
MatOfPoint largestContour = contours.get(0);
double largestContourArea = Imgproc.contourArea(largestContour);
for (int i = 1; i < contours.size(); ++i) {// NB Notice the prefix increment.
MatOfPoint currentContour = contours.get(i);
double currentContourArea = Imgproc.contourArea(currentContour);
if (currentContourArea > largestContourArea) {
largestContourArea = currentContourArea;
largestContour = currentContour;
}
}
MatOfPoint2f largestContour2f = new MatOfPoint2f(largestContour.toArray());// Required on Line 289. See http://stackoverflow.com/questions/11273588/how-to-convert-matofpoint-to-matofpoint2f-in-opencv-java-api
Rect detectedBlobRoi = Imgproc.boundingRect(largestContour);
Mat detectedBlobRgba = rgbaFrame.submat(detectedBlobRoi);
Highgui.imwrite("/mnt/sdcard/DCIM/detectedBlobRgba.bmp", detectedBlobRgba);// check
Mat detectedBlobHsv = new Mat();
Imgproc.cvtColor(detectedBlobRgba, detectedBlobHsv, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/roiHsv.bmp", detectedBlobHsv);// check
for (int firstCoordinate = 0; firstCoordinate < detectedBlobHsv.rows(); firstCoordinate++) {
for (int secondCoordinate = 0; secondCoordinate < detectedBlobHsv.cols(); secondCoordinate++) {
Log.i(TAG, "HAPPY " + Arrays.toString(detectedBlobHsv.get(firstCoordinate, secondCoordinate)));
if (Imgproc.pointPolygonTest(largestContour2f, new Point(firstCoordinate, secondCoordinate), false) == -1) {
Log.i(TAG, "HAPPY ....................... OUTSIDE");
}
}
}
Highgui.imwrite("/mnt/sdcard/DCIM/processedcontoured.bmp", detectedBlobHsv);// check
EDIT:
I am doing this because I need to compute the average HSV color of pixels lying inside the contour (i.e. the average HSV color of the biggest red colored blob). If I computed the average color of the ROI detectedBlobHsv by the normal formula, I would do something like
Scalar averageHsvColor= new Scalar(256);
Scalar sumHsvOfPixels = new Scalar(256);
sumHsvOfPixels = Core.sumElems(detectedBlobHsv);
int numOfPixels = detectedBlobHsv.width() * detectedBlobHsv.height();
for (int channel=0; channel<sumHsvOfPixels.val.length; channel++) {
averageHsvColor = sumHsvOfPixels.val[channel]/numOfPixels;
}
So somebody here on SO (probably you?) had suggested me a way to exclude pixels outside my contour a while back. I'd implement that like:
//Giving pixels outside contour of interest an HSV value of `double[]{0,0,0}`, so that they don't affect the computation of `sumHsvOfPixels` while computing average,
//and while keeping track of the number of pixels removed from computation this way, so we can subtract that number from the `$numOfPixels` during computation of average.
int pixelsRemoved = 0;
for (int row=0; row<detectedBlobHsv.rows(); row++) {
for (int col=0; col<detectedBlobHsv.cols(); col++) {
if (Imgproc.pointPolygonTest(largestContour2f, new Point(row, col), false) == -1) {
detectedBlobHsv.put(row, col, new double[]{0,0,0});
pixelsRemoved++;
}
}
}
Then compute the average like
Scalar averageHsvColor= new Scalar(256);
Scalar sumHsvOfPixels = new Scalar(256);
sumHsvOfPixels = Core.sumElems(detectedBlobHsv); //This will now exclude pixels outside the contour
int numOfPixels = ( detectedBlobHsv.width()*detectedBlobHsv.height() )-pixelsRemoved;
for (int channel=0; channel<sumHsvOfPixels.val.length; channel++) {
averageHsvColor = sumHsvOfPixels.val[channel]/numOfPixels;
}
EDIT 1:
Towards the end of the following method, I have created the mask with a list of MatOfPoints which contains the largest contour only. When I wrote it to SDCard, I got
I don't know where I messed up!
private Scalar detectColoredBlob() {
//Highgui.imwrite("/mnt/sdcard/DCIM/rgbaFrame.jpg", rgbaFrame);// check
rgbaFrame = Highgui.imread("/mnt/sdcard/DCIM/rgbaMat4Mask.bmp");
//GIVING A UNIFORM VALUE OF 255 TO THE V CHANNEL OF EACH PIXEL (255 IS THE MAXIMUM VALUE OF V ALLOWED - Simulating a maximum light condition)
for (int firstCoordinate = 0; firstCoordinate < rgbaFrame.rows(); firstCoordinate++) {
for (int secondCoordinate = 0; secondCoordinate < rgbaFrame.cols(); secondCoordinate++) {
double[] pixelChannels = rgbaFrame.get(firstCoordinate, secondCoordinate);
pixelChannels[2] = 255;
rgbaFrame.put(firstCoordinate, secondCoordinate, pixelChannels);
}
}
Mat hsvImage = new Mat();
Imgproc.cvtColor(rgbaFrame, hsvImage, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/hsvImage.bmp", hsvImage);// check
Mat maskedImage = new Mat();
Core.inRange(hsvImage, new Scalar(0, 100, 100), new Scalar(10, 255, 255), maskedImage);
Highgui.imwrite("/mnt/sdcard/DCIM/maskedImage.bmp", maskedImage);// check
// Mat dilatedMat = new Mat();
// Imgproc.dilate(maskedImage, dilatedMat, new Mat());
// Highgui.imwrite("/mnt/sdcard/DCIM/dilatedMat.jpg", dilatedMat);// check
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(maskedImage, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
//FINDING THE BIGGEST CONTOUR
// \/ We will use only the largest contour. Other contours (any other possible blobs of this color range) will be ignored.
MatOfPoint largestContour = contours.get(0);
double largestContourArea = Imgproc.contourArea(largestContour);
for (int i = 1; i < contours.size(); ++i) {// NB Notice the prefix increment.
MatOfPoint currentContour = contours.get(i);
double currentContourArea = Imgproc.contourArea(currentContour);
if (currentContourArea > largestContourArea) {
largestContourArea = currentContourArea;
largestContour = currentContour;
}
}
Rect detectedBlobRoi = Imgproc.boundingRect(largestContour);
Mat detectedBlobRgba = rgbaFrame.submat(detectedBlobRoi);
Highgui.imwrite("/mnt/sdcard/DCIM/detectedBlobRgba.bmp", detectedBlobRgba);// check
Mat detectedBlobHsv = new Mat();
Imgproc.cvtColor(detectedBlobRgba, detectedBlobHsv, Imgproc.COLOR_BGR2HSV);
Highgui.imwrite("/mnt/sdcard/DCIM/roiHsv.bmp", detectedBlobHsv);// check
List<MatOfPoint> largestContourList = new ArrayList<>();
largestContourList.add(largestContour);
Mat roiWithMask = new Mat(detectedBlobHsv.rows(), detectedBlobHsv.cols(), CvType.CV_8UC3);
roiWithMask.setTo(new Scalar(0,0,0));
Imgproc.drawContours(roiWithMask, largestContourList, 0, new Scalar(0, 255, 255), -1);//TODO Using -1 instead of CV_FILLED.
Highgui.imwrite("/mnt/sdcard/DCIM/roiWithMask.bmp", roiWithMask);// check
// CALCULATING THE AVERAGE COLOR OF THE DETECTED BLOB
// STEP 1:
double [] averageHsvColor = new double[]{0,0,0};
int numOfPixels = 0;
for (int firstCoordinate = 0; firstCoordinate < detectedBlobHsv.rows(); ++firstCoordinate) {
for (int secondCoordinate = 0; secondCoordinate < detectedBlobHsv.cols(); ++secondCoordinate) {
double hue = roiWithMask.get(firstCoordinate, secondCoordinate)[0];
double saturation = roiWithMask.get(firstCoordinate, secondCoordinate)[1];
double value = roiWithMask.get(firstCoordinate, secondCoordinate)[2];
averageHsvColor[0] += hue;
averageHsvColor[1] += saturation;
averageHsvColor[2] += value;
numOfPixels++;
}
}
averageHsvColor[0] /= numOfPixels;
averageHsvColor[1] /= numOfPixels;
averageHsvColor[1] /= numOfPixels;
return new Scalar(averageHsvColor);
}
EDIT 2:
I corrected my 3 channel mask and made a single channel mask
Mat roiMask = new Mat(rgbaFrame.rows(), rgbaFrame.cols(), CvType.CV_8UC1);
roiMask.setTo(new Scalar(0));
Imgproc.drawContours(roiMask, largestContourList, 0, new Scalar(255), -1);
and this resulted in the correct roiMask:
Then, before the comment // CALCULATING THE AVERAGE COLOR OF THE DETECTED BLOB, I added:
Mat newImageWithRoi = new Mat(rgbaFrame.rows(), rgbaFrame.cols(), CvType.CV_8UC3);
newImageWithRoi.setTo(new Scalar(0, 0, 0));
rgbaFrame.copyTo(newImageWithRoi, roiMask);
Highgui.imwrite("/mnt/sdcard/DCIM/newImageWithRoi.bmp", newImageWithRoi);//check
This resulted in:
Now again I don't know how to proceed. :s
You don't need to use pointPolygonTest, because you already have the mask.
You can simply sum up the values that lies on the mask. Something along the lines of (not able to test this):
// Initialize at 0!!!
Scalar averageHsvColor= new Scalar(0,0,0);
int numOfPixels = 0;
for(int r=0; r<detectedBlobHsv.height(); ++r)
{
for(int c=0; c<detectedBlobHsv.width(); ++c)
{
if( /* value of mask(r,c) > 0 */)
{
int H = // get H value of pixel at (r, c)
int S = // get S value of pixel at (r, c)
int V = // get V value of pixel at (r, c)
// Sum values
averageHsvColor[0] += H;
averageHsvColor[1] += S;
averageHsvColor[2] += V;
// Increment number of pixels inside mask
numOfPixels ++;
}
}
}
// Compute average
averageHsvColor[0] /= numOfPixels ;
averageHsvColor[1] /= numOfPixels ;
averageHsvColor[2] /= numOfPixels ;
I'm making an app which will match the input image with the images from the database.
I'm using this code anyway:
String path = Environment.getExternalStorageDirectory().getAbsolutePath();
Bitmap objectbmp = BitmapFactory.decodeFile(path+"/Sample/Template.jpg");
Bitmap scenebmp = BitmapFactory.decodeFile(path+"/Sample/Input.jpg");
Mat object = new Mat(); //from the database
Mat scene = new Mat(); //user's input image
// convert bitmap to MAT
Utils.bitmapToMat(objectbmp, object);
Utils.bitmapToMat(scenebmp, scene);
//Feature Detection
FeatureDetector orbDetector = FeatureDetector.create(FeatureDetector.ORB);
DescriptorExtractor orbextractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
MatOfKeyPoint keypoints_object = new MatOfKeyPoint();
MatOfKeyPoint keypoints_scene = new MatOfKeyPoint();
Mat descriptors_object = new Mat();
Mat descriptors_scene = new Mat();
//Getting the keypoints
orbDetector.detect( object, keypoints_object );
orbDetector.detect( scene, keypoints_scene );
//Compute descriptors
orbextractor.compute( object, keypoints_object, descriptors_object );
orbextractor.compute( scene, keypoints_scene, descriptors_scene );
//Match with Brute Force
MatOfDMatch matches = new MatOfDMatch();
DescriptorMatcher matcher;
matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE);
matcher.match( descriptors_object, descriptors_scene, matches );
double max_dist = 0;
double min_dist = 100;
List<DMatch> matchesList = matches.toList();
//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_object.rows(); i++ )
{ double dist = matchesList.get(i).distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
for( int i = 0; i < descriptors_object.rows(); i++ )
{ if( matchesList.get(i).distance <= 3*min_dist )
{ good_matches.addLast( matchesList.get(i));
}
}
I am able to produce and count good matches though, what I want is to know the match rate between two matched images like:
Input - Template1 = 35%
Input - Template2 = 12%
.....................
How to do this?
You could compute a matching rate like goodMatches/totMatches, that is the accuracy of the matching.
Actually there are different ways to do that. The common are:
Cross check: if T1 matches with T2, check if T2 checks with T1
Ratio check: as in SIFT, if the best template that matches with T1 is T2, consider the second best match template T2_2 and accept the first match only if the ratio between the matchings is good enough.
Geometric validation: you should compute the homography between templates and discard the matchings that doesn't agree
I've implemented the first two in Java in an Android application (I used ORB as features).
private List<MatOfDMatch> crossCheck(List<DMatch> matches12, List<DMatch> matches21, List<MatOfDMatch> knn_matches) {
List<MatOfDMatch> good_matches = new ArrayList<MatOfDMatch>();
for(int i=0; i<matches12.size(); i++)
{
DMatch forward = matches12.get(i);
DMatch backward = matches21.get(forward.trainIdx);
if(backward.trainIdx == forward.queryIdx)
good_matches.add(knn_matches.get(i)); //k=2
}
return good_matches;
}
private List<MatOfDMatch> ratioCheck(List<MatOfDMatch> knn_matches, float ratio) {
List<MatOfDMatch> good_matches = new ArrayList<MatOfDMatch>();
for(int i=0; i<knn_matches.size(); i++)
{
List<DMatch> subList = knn_matches.get(i).toList();
if(subList.size()>=2)
{
Float first_distance = subList.get(0).distance;
Float second_distance = subList.get(1).distance;
if((first_distance/second_distance) <= ratio)
good_matches.add(knn_matches.get(i));
}
}
return good_matches;
}
i am working on an android project using opencv , i am creating a histogram for a black and white image (1-0 values on image file) .
i am following some tutorials i found on internet on how to create the histogram . i am doing something like this
ArrayList<Mat> list = new ArrayList<Mat>();
list.add(mRgba);
MatOfInt channels = new MatOfInt(0);
Mat hist= new Mat();
MatOfInt histSize = new MatOfInt(25);
MatOfFloat ranges = new MatOfFloat(0f, 1f);
Imgproc.calcHist(list, channels, new Mat(), hist, histSize, ranges);
Then i want to view the data on hist Mat, if i try something like this...
for(int i = 0;i<25;i++){
Log.e(TAG, "data "+i+" "+hist.get(i, 0));
}
i get
> data 0 [D#2be05908
data 1 [D#2be0a138
data 2 [D#2bdf9f48
data 22 [D#2be06c70
that makes no sense to me. if i try a different approach, like
byte buff[] = new byte[ hist.height()*hist.width() * hist.channels()];
hist.get(0, 0, buff);
i get error about mat compatibility with mat.get function.
Is there any way to directly access the data on hist mat?
i am intrested in getting back all the data, not only man mix
hist.get(i, 0) returns array of doubles.
So you can try this:
for (int i = 0; i< 25; i++) {
double[] histValues = hist.get(i, 0);
for (int j = 0; j < histValues.length; j++) {
Log.d(TAG, "yourData=" + histValues[j]);
}
}
have this problem with PCACompute in Android Opencv2.3.1 because when i call PCACompute my eigenvectors are all 0. So, i take 10 photos for each people and i save it into a Mat of 100X100.
After that, i convert my 100X100 Mat in one Mat 1X10000 with this code:
double [] elem = null;
for(int riga=0;riga<m.rows();riga++)
{
for(int colonna=0;colonna<m.cols();colonna++)
{
elem = m.get(riga, colonna);
mrow.put(0,((riga*100)+colonna), elem[0]);
}//for colonna
}//for riga
After that, when i take 10 photos, i insert all Mat of the photos into one mat with this code:
double b[] = null;
for (int i = 0; i< listafoto.size(); i++)
{
Mat t = listafoto.get(i);
for(int riga = 0;riga<t.rows();riga++)
{
for(int colonna =0;colonna<t.cols();colonna++)
{
b = t.get(riga, colonna);
datiOriginali.put(i, colonna, b[0]);
}//for colonna
}//for riga
}//for lista e contemporaneamente riga datiOriginali
After that, i call PCACompute with this code: `
org.opencv.core.Core.PCACompute(datiOriginali,mean, eigenvectors, 10);`
So, datiOriginali is the input Mat of 10 rows and 10000 cols, mean and eigenvectors are the output matrix. mean matrix give me a result, but eigenvectors give me all 0. Can you help me to resolve this problem?
Thanks in advance.MArco
I based my code on the example at http://www.bytefish.de/blog/pca_in_opencv.
Here's how I did this:
Vector trainingImages = new Vector();;
trainingImages.add(Highgui.imread("/sdcard/facedatabase/s1/1.pgm",0));
trainingImages.add(Highgui.imread("/sdcard/facedatabase/s1/2.pgm",0));
Mat x = (Mat) trainingImages.get(0);
int total = x.rows() * x.cols();
// build matrix (column)
// This matrix will have one col for each image and imagerows x imagecols rows
Mat mat = new Mat(total, trainingImages.size(), CvType.CV_32FC1);
for(int i = 0; i < trainingImages.size(); i++) {
Mat X = mat.col(i);
Mat c = (Mat) trainingImages.get(i);
c.reshape(1,total).convertTo(X, CvType.CV_32FC1);
}
Mat eigenVectors = new Mat();
Mat mean = new Mat();
Core.PCACompute(mat, mean, eigenVectors);