Develop Reference

Android openCV can`t IDFT back to orginal image

enter image description hereHello, i try to transform image from time domain to frequency domain by using opencv DFT() function, it works, but i can`t transform it back by using IDFT() function。
In my getIDFT() function , i get phase from dft complex and use polarToCart to get Re and Im Data, but something wrong with my image , can somebody help me , thanks.
Github : https://github.com/pighaddt/opencv_demo/tree/master/app/src
private Mat getIDFT(Mat DFTimage){
DFTimage.convertTo(DFTimage, CvType.CV_64FC1);
DFTimage = DFTShift(DFTimage);
Mat Re = new Mat(DFTimage.size(), CvType.CV_64FC1); // expand input image to optimal size
Mat Im = new Mat(DFTimage.size(), CvType.CV_64FC1); // expand input image to optimal size
Mat ph = new Mat(DFTimage.size(), CvType.CV_64FC1);
Core.phase(planes.get(0), planes.get(1), ph, false);
Core.polarToCart(DFTData, ph, Re, Im, false);
Mat complexI = Mat.zeros(Re.rows(), Re.cols(), CvType.CV_64FC2);
Mat complexI3 = Mat.zeros(Re.rows(), Re.cols(), CvType.CV_64FC2);
List<Mat> planesIDFT = new ArrayList<Mat>();
planesIDFT.add(Re);
planesIDFT.add(Im);
Core.merge(planesIDFT, complexI); // Add to the expanded another plane with zeros
Mat invDFTcvt = new Mat(Re.size(), CvType.CV_8UC1);
Core.idft(complexI, complexI3, Core.DFT_INVERSE | Core.DFT_COMPLEX_OUTPUT | Core.DFT_SCALE, 0);
Core.split(complexI3, planesIDFT); // planes[0] = Re(DFT(I)), planes[1] = Im(DFT(I))
Core.normalize(planesIDFT.get(0), invDFTcvt, 0, 255, Core.NORM_MINMAX);
invDFTcvt.convertTo(invDFTcvt, CvType.CV_8UC1);
isDFT = false;
return invDFTcvt;
}

Image Matching on Android Game's Board

The question comes 1st: I am looking for FAST approach to match images.
Now, the use case: I am developing a detector to detect orb on a 6x5 Match-3 game board for android platform. I have an array of the orb icon with transparent background, but the orb on the screen (screenshot) has different background color, probably different size too. I have to compare each orb on the screen with my array of icons (69 icons specifically) so it's a 69x30=2070 steps. I tried lazy implementation and group almost similar icon together to reduce the steps but still take a long time (10s at most) for computation. I also tried checking the channel and depth of image, resizing the images to have same size and tweaking the threshold value but still no luck.
I have tried Histogram Matching (seperate channel, grayscale), Template Matching (CCOEFF, SQDIFF, CCORR), AKAZE, ORB(unbounded, bounded), PHash all using OpenCV but histogram matching and PHash give me erroneous result (too much false positive), Template Matching consume 10s+ (considered too slow for user to wait) while AKAZE and ORB give better result than all other methods but still needs 6s+ per try. Is there any other method that can helps me cut down the computation time down to somewhere near 1s and can give better result considering the worst case scenario is 2070 steps?
Referrences that I have read that compares the performances of different feature matching algorithms:
A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. It shows that ORB and BRISK should be averagely better than the other approach compared while AKAZE is moderately good for most cases. I deleted my Histogram comparison code as it is not really helpful but you may find the rest of it below.
Mat source = Utils.loadResource(this, R.drawable.orb_icon, Imgcodecs.CV_LOAD_IMAGE_UNCHANGED);
Mat tmp = new Mat();
Bitmap cropped_img = Bitmap.createBitmap(screenshot, x, y, width, height);
Utils.bitmapToMat(cropped_img, tmp);
//template matching code
int r_rows = source.rows() - tmp.rows() + 1;
int r_cols = source.cols() - tmp.cols() + 1;
Mat result = new Mat();
result.create(r_rows, r_cols, CvType.CV_32F);
Imgproc.matchTemplate(source, tmp, result, Imgproc.TM_CCOEFF_NORMED);
Core.MinMaxLocResult mmr = Core.minMaxLoc(result);
double maxVal = mmr.maxVal;
return maxVal;
//AKAZE
MatOfKeyPoint kp1 = new MatOfKeyPoint();
MatOfKeyPoint kp2 = new MatOfKeyPoint();
Mat desc1 = new Mat();
Mat desc2 = new Mat();
AKAZE akaze = AKAZE.create();
DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
akaze.detectAndCompute(source, new Mat(), kp1, desc1);
akaze.detectAndCompute(tmp, new Mat(), kp2, desc2);
List<MatOfDMatch> knnMatches = new ArrayList<>();
matcher.knnMatch(desc1, desc2, knnMatches, 2);
float threshold = 0.7f;
int count = 0;
for(int i=0; i<knnMatches.size(); i++) {
if(knnMatches.get(i).rows() > 1) {
DMatch[] matches = knnMatches.get(i).toArray();
if(matches[0].distance < threshold * matches[1].distance) {
count++;
}
}
}
//ORB
ORB orb = ORB.create();
DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
MatOfKeyPoint kp1 = new MatOfKeyPoint();
MatOfKeyPoint kp2 = new MatOfKeyPoint();
Mat desc1 = new Mat();
Mat desc2 = new Mat();
orb.detectAndCompute(source, new Mat(), kp1, desc1);
orb.detectAndCompute(tmp, new Mat(), kp2, desc2);
List<MatOfDMatch> knnMatches = new ArrayList<>();
matcher.knnMatch(desc1, desc2, knnMatches, 2);
float threshold = 0.8f;
int count = 0;
for(int i=0; i<knnMatches.size(); i++) {
if(knnMatches.get(i).rows() > 1) {
DMatch[] matches = knnMatches.get(i).toArray();
if(matches[0].distance < threshold * matches[1].distance) {
count++;
}
}
}
//PHash
Mat hash_source = new Mat();
Mat hash_tmp = new Mat();
Img_hash.pHash(tmp, hash_tmp);
Img_hash.pHash(source, hash_source);
Core.norm(source, tmp, Core.NORM_HAMMING);
Edit: As suggested, below is the game board, icon image, and orb screenshot sample.
ICON vs orb screenshot
Also, you may observe the simulation result of each approach by comparing the result(overlay smaller icon) on top of the orb on board:
Histogram Matching
,
Template Matching
and
AKAZE (similar to ORB)
After moving the variable initialization out of my comparison function to base class, detect keypoint and PHash of source icon images on class initialization, run detect and compute function in batch using List to reduce individual function call. It still takes up 4s+ for the image matching process. Time consumption is reduced but accuracy is still a major problem. You may observe my heap stack on below.

Command .copyto is failing

I'm a student and I'm working on application that will scann a sudoku and solve it. I'm taking picture, than finding biggest contour. This is what is working.Problem begin when I want to extract that biggest counter on the empty mat (it has white backround), application don't show activity with picture that should (in other images it does) but it return to my mainactivity. I was using this tutorial for the extraction: https://bytefish.de/blog/extracting_contours_with_opencv/.
mat4=mat1; // mat 1 is current frame on camera
transpose(mat4, mat4);
flip(mat4, mat4, +1);
mat5=mat4;
Mat okraje = new Mat();
Mat hiearchy = new Mat();
Imgproc.cvtColor(mat5,mat5,Imgproc.COLOR_BGR2GRAY);
List<MatOfPoint> contourList = new ArrayList<MatOfPoint>();
Imgproc.Canny(mat5,okraje,80,100);
Imgproc.findContours(okraje,contourList,hiearchy,Imgproc.RETR_CCOMP, Imgproc.CHAIN_APPROX_SIMPLE); // TOPKA JEBNE PLNY STVOREC A KRATKO!!
for(int ab=0;ab < contourList.size(); ab++ ){
a = contourArea(contourList.get(ab),false);
if(a>largest_area){
b=ab;
largest_area = a;
largest_contour_index=ab;
bounding_rect=boundingRect(contourList.get(ab));
}
}
Mat len_sudkoku = new Mat();
len_sudkoku.create(mat5.rows(), mat5.cols(),CvType.CV_8UC3);
len_sudkoku.setTo(new Scalar(255,255,255));
Mat lskere = new Mat();
lskere.create(okraje.cols(), okraje.rows(), CvType.CV_8UC1);
Random r = new Random();
Imgproc.drawContours( lskere, contourList,largest_contour_index, new Scalar(r.nextInt(255), r.nextInt(255), r.nextInt(255)), -1);
mat5.copyTo(len_sudkoku,lskere); // pada to!!
Bitmap bm = Bitmap.createBitmap(len_sudkoku.cols(),len_sudkoku.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(len_sudkoku,bm);
setContentView(R.layout.activity_fotka_ukaz);
ImageView IMW = findViewById(R.id.imageView);
IMW.setImageBitmap(bm);
I expected to be like in tutorial that I posted here where a man extracted an apple and place it on another backround. Thing that I notice is that application return to main activity (it should display image), when I'm using a command
mat5.copyTo(len_sudkoku, lskere)

Ok guys i solved it. The problem wasn't in command copy to (obviously) but in declaration size of different mat and bitmap. The problem was that in declaration first input was sometimes row and column so it crashed. Now I add size to Mat hierarchy and okraje, and edit lskere.create(okraje.cols(), okraje.rows(), to lskere.create(okraje.rows(), okraje.cols()..

How can i improve openCV people detecting algorithm

I am trying to write human detector, it works now, but sometimes it reacts on cats/boxes etc., also i got like 5 fps. So the question is, how can i improve my algorithm for better fps and detection accuracy.
I have tried to use this one:
http://www.pyimagesearch.com/2015/11/09/pedestrian-detection-opencv/
But i couldnt find any way i could use this on android.
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
List<MatOfPoint> list = new ArrayList<>();
Mat frame = new Mat();
Mat gray = new Mat();
Mat hierarchy = new Mat();
Mat originalFrame = inputFrame.rgba();
Imgproc.medianBlur(originalFrame,originalFrame,3);
Imgproc.cvtColor(originalFrame, gray, Imgproc.COLOR_RGB2GRAY, 0);
HOGDescriptor hog = new HOGDescriptor();
//Получаем стандартный определитель людей и устанавливаем его нашему дескриптору
MatOfFloat descriptors = HOGDescriptor.getDefaultPeopleDetector();
hog.setSVMDetector(descriptors);
MatOfRect locations = new MatOfRect();
MatOfDouble weights = new MatOfDouble();
hog.detectMultiScale(gray, locations, weights);
Point rectPoint1 = new Point();
Point rectPoint2 = new Point();
Point fontPoint = new Point();
if (locations.rows() > 0) {
List<Rect> rectangles = locations.toList();
for (Rect rect : rectangles) {
rectPoint1.x = rect.x;
rectPoint1.y = rect.y;
fontPoint.x = rect.x;
fontPoint.y = rect.y - 4;
rectPoint2.x = rect.x + rect.width;
rectPoint2.y = rect.y + rect.height;
final Scalar rectColor = new Scalar( 0 , 0 , 0 );
// Добавляем на изображения найденную информацию
Imgproc.rectangle(originalFrame, rectPoint1, rectPoint2, rectColor, 2);
}
}
frame.release();
gray.release();
hierarchy.release();
list.clear();
return originalFrame;
}

You're using the HOG+SVM approach to detect people; it is inherently going to be quite slow. Never the less, you can use some of the suggestions in this question How to speed up svm.predict?
Depending on your problem, i.e. if the camera is static and the pedestrians are moving you could opt for a background subtraction approach this is probably the most efficient way but bear in mind that this will pick up any objects that are moving in the scene, so you could include thresholds to remove small objects. Some background subtraction algorithms include mixture of gaussian (MOG) or MOG2 or GMG. Also, an important thing to note is that these approaches rely on creating a background model of the scene, i.e. they assume static pixels over time to be part of the background, hence, when a pedestrian stands still for a while in the scene they get embedded into the background resulting in miss detection. There are many papers out there that provide potential solutions to that problem so you might want to have a look at them, here is one that produces decent results: Static and Moving Object Detection Using Flux Tensor with Split Gaussian Models
Additionally, you could opt for a data driven approach, either get a good pre-trained model and do your detection using that or train one yourself using TensorFlow, Caffe or Torch and use the dnn opencv_contrib module to do the detection.

How to normalize OpenCV Android matrices?

tl;dr My KNearest training data and real data don't have the same dimensions and cause my app to crash. I suspect that either my preProces method of the way I instantiate my training data (drawable resource => bitmap => opencv matrix) is the reason for failure. Does any of you know a solution?
I've been trying to get a working demo of a simple OCR app with OpenCV for Android. I use the build in KNearest to recognize the characters. Before a KNearest object is capable of detecting anything, it has to be trained. For the training I use several character outlines.
This is one of them (its a zero).
The training seems to work unsurprisingly it is capable to detect the supposed values of the training images. I wish it did that with other images as well (or at leas not crash my app). This is what I did to train the KNearest model:
Map<Character, Integer> images = new HashMap<>();
images.put('0', R.drawable.training0);
// Prepare two sets of data, the images and their values.
Mat trainingImages = new Mat();
Mat trainingLabels = new Mat();
for (int i = 0; i < 50; i++) {
for (Map.Entry<Character, Integer> entry : images.entrySet()) {
Bitmap bitmapImage = BitmapFactory.decodeResource(
this.getResources(), entry.getValue());
Mat matImage = new Mat();
Utils.bitmapToMat(bitmapImage, matImage);
trainingLabels.push_back(new MatOfInt(entry.getKey() - '0'));
trainingImages.push_back(
preProces(
matImage, new Rect(0, 0, matImage.width(), matImage.height())));
}
}
mKNearest.train(trainingImages, Ml.ROW_SAMPLE, trainingLabels);
The preProces method does nothing more than normalizing a matrix. This is what my preProces method looks like:
private Mat preProces(Mat image, Rect poi) {
Mat cutout = new Mat(image, poi);
Mat resized = new Mat(10, 10, CvType.CV_32F);
Mat converted = new Mat();
Imgproc.resize(cutout, resized, resized.size());
resized.reshape(1, 1).convertTo(converted, CvType.CV_32F);
return converted;
}
Segmenting the image to find (possible) characters was not that difficult, I was able to draw rectangles around the (possible) characters. Once that is done I just pass every point of interest through my preProces method before I pass it into the mKNearest.findNeareset(...) method. This is when the crash happens. The training data and the real data don't seem to have the same dimensions, something the preProces method should solve.
My guess is that either my preProces method fails or that loading drawable resources as bitmap and then converting them to matrices is the reason why it fails. I'd like to know if some of you had similar problems and how you've solved it.
Update: It seems there is quite a bit of noise in the matrices which where created out of a bitmap. Could this be the problem, if so how does one remove the noise?

It seems the answer to this question was pretty simple. I used Imgproc.canny() to detect the edges of the real data but not on the training data. The problem was solved once I passed the training data through Imgproc.canny().
...
Bitmap bitmapImage = BitmapFactory.decodeResource(
this.getResources(), entry.getValue());
Mat matImage = new Mat();
Utils.bitmapToMat(bitmapImage, matImage);
// This was all I had to add to the training data preparation.
Mat cannyImage = new Mat();
Imgproc.Canny(matImage, cannyImage, 1.0, 255.0);
trainingLabels.push_back(new MatOfInt(entry.getKey() - '0'));
trainingImages.push_back(
preProces(
cannyImage, new Rect(0, 0, cannyImage.width(), cannyImage.height())));
}
...