I'm quite new to OpenCV programming, and I'm developing an app that works like this:
Take photo of a shelf from camera
Analyze it with openCV to detect all the rectangles in the photo (i.e. want to find all the products stored on the shelf)
crop all the rectangle elements found.
My problem is that the detection phase works not so well.
The code used to analyze the taken photo:
Bitmap originalPhoto;
byte[] bytes = photo.getByteArray("bitmap");
originalPhoto = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
Mat imgMat=new Mat();
Utils.bitmapToMat(originalPhoto,imgMat);
Mat imgSource=imgMat.clone();
Imgproc.cvtColor( imgMat, imgMat, Imgproc.COLOR_BGR2GRAY);
Bitmap grayscale=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgMat,grayscale);
String root = Environment.getExternalStorageDirectory().toString();
File myDir = new File(root + "/saved_images");
Imgproc.Canny(imgMat,imgMat,0,255);
Bitmap canny=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgMat,canny);
Imgproc.GaussianBlur(imgMat, imgMat, new org.opencv.core.Size(1, 1), 2, 2);
Bitmap blur=Bitmap.createBitmap(imgMat.cols(),imgMat.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgMat,blur);
//find the contours
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Imgproc.findContours(imgMat, contours, new Mat(), Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
MatOfPoint temp_contour = contours.get(0); //the largest is at the index 0 for starting point
for (int idx = 0; idx < contours.size(); idx++) {
temp_contour = contours.get(idx);
//check if this contour is a square
MatOfPoint2f new_mat = new MatOfPoint2f( temp_contour.toArray() );
int contourSize = (int)temp_contour.total();
MatOfPoint2f approxCurve_temp = new MatOfPoint2f();
Imgproc.approxPolyDP(new_mat, approxCurve_temp, contourSize*0.05, true);
if (approxCurve_temp.total() == 4) {
MatOfPoint points = new MatOfPoint( approxCurve_temp.toArray() );
Rect rect = Imgproc.boundingRect(points);
Core.rectangle(imgSource, new Point(rect.x,rect.y), new Point(rect.x+rect.width,rect.y+rect.height), new Scalar(255, 0, 0, 255), 3);
}
}
Bitmap analyzed=Bitmap.createBitmap(imgSource.cols(),imgSource.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(imgSource,analyzed);
if(!myDir.exists()) myDir.mkdirs();
fname = "ImageAnalyzed.png";
file = new File (myDir, fname);
if (file.exists ()) file.delete ();
try {
FileOutputStream out = new FileOutputStream(file);
analyzed.compress(Bitmap.CompressFormat.PNG, 100, out);
out.flush();
out.close();
} catch (Exception e) {
e.printStackTrace();
}
The result photo is not correctly analyzed, it returns to me a photo like this:
Has anyone an idea on how to solve it?
Thanks in advance,
Fabio
edit
I found that using ImgProc.APPROX_CHAIN_NONE instead of ImgProc.APPROX_CHAIN_SIMPLE gives me much better results... Now I have to tune canny thresholds to give the program independency wrt the type of image I analyze. I think that using mean or median values of the image will give me better results as well.
Related
Im detecting patches squares at strip image
I think boundaries are not clear
How can i get this patches clearly
please help me
this is my code and images
Thanks
Code
Bitmap bmp = BitmapFactory.decodeResource(getResources(), R.drawable.src2);
// bmp = changeBitmapContrastBrightness(bmp, (float)1.5, 0);
Mat src = new Mat();
Utils.bitmapToMat(bmp, src);
// Creating an empty matrix to store the result
Mat dst = new Mat();
// Creating kernel matrix
Mat kernel = Mat.ones(1,1, CvType.CV_32F);
for(int i = 0; i<kernel.rows(); i++) {
for(int j = 0; j<kernel.cols(); j++) {
double[] m = kernel.get(i, j);
for(int k = 1; k<m.length; k++) {
m[k] = m[k]/(2 * 2);
}
kernel.put(i,j, m);
}
}
Imgproc.filter2D(src, dst, -1, kernel);
Imgproc.cvtColor(dst, dst, Imgproc.COLOR_BGR2GRAY);
// Preparing the kernel matrix object
Mat kernel1 = Imgproc.getStructuringElement(Imgproc.MORPH_RECT,
new Size((2*2) + 1, (2*2)+1));
Imgproc.dilate(dst, dst, kernel1);
Imgproc.threshold(dst, dst, 160, 255, Imgproc.THRESH_BINARY);
// Creating kernel matrix
Mat kernel2 = Mat.ones(5,5, CvType.CV_32F);
Imgproc.morphologyEx(dst, dst, Imgproc.MORPH_OPEN, kernel2);
}
private static List<MatOfPoint> contourFind(Mat img){
List<MatOfPoint> contours = new ArrayList<>();
Imgproc.findContours(img, contours, new Mat(), Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
List<MatOfPoint> squares = new ArrayList<>();
for(MatOfPoint cnt: contours){
MatOfPoint2f curve = new MatOfPoint2f(cnt.toArray());
MatOfPoint2f approxCurve = new MatOfPoint2f();
Imgproc.approxPolyDP(curve, approxCurve, 0.02 * Imgproc.arcLength(curve, true), true);
int numberVertices = (int) approxCurve.total();
double contourArea = Imgproc.contourArea(cnt);
if (Math.abs(contourArea) < img.size().area() / 10){
squares.add(cnt);
}
}
return squares;
}
Original Image
enter image description here
After process Image
enter image description here
You might want to look at some segmentation algorithm and train your model based on sample images for each category. There are algorithms like the Watershed algorithm for classical Machine learning. Or look at semantic segmentation if you can use deep learning and neural networks.
I want to find the edges of image that I have used in basic image processing methods(blurring,sharpening..etc).I don't get any errors, but it isn't working.The program stops when it comes to on the Imgproc.Canny or Imgproc.findContours method with Debugging.
As a last resort, I wrote native code to find the edges, but I could not run it.
That is my code to call finding contours and execute it:
{
Mat rawImageMatGray = new Mat();
Imgproc.cvtColor(this.rawImageMat, rawImageMatGray, Imgproc.COLOR_BGR2GRAY);
Imgproc.adaptiveThreshold(rawImageMatGray, rawImageMatGray, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY, 55, 15);
Bitmap segmentImage = Bitmap.createBitmap(rawImageMatGray.cols(), rawImageMatGray.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(rawImageMatGray, segmentImage);
segmentImage = Contours(segmentImage);
return segmentImage;
}
private Bitmap Contours(Bitmap cannyImage) {
Mat cannyImageMat = new Mat();
Mat cannyImageMat2 = new Mat();
Utils.bitmapToMat(cannyImage, cannyImageMat);
Mat cannyEdges = new Mat();
Mat hierarchy = new Mat();
List<MatOfPoint> contourList = new ArrayList<MatOfPoint>();
//A list to store all the contours
//Converting the image to grayscale
//Imgproc.cvtColor(originalMat,grayMat,Imgproc.COLOR_BGR2GRAY);
Imgproc.Canny(cannyImageMat, cannyEdges, 80, 100);
//finding contours
Imgproc.findContours(cannyEdges, contourList, hierarchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
//Drawing contours on a new image
Mat contours = new Mat();
contours.create(cannyEdges.rows(), cannyEdges.cols(), CvType.CV_8UC3);
Random r = new Random();
for (int i = 0; i < contourList.size(); i++) {
Imgproc.drawContours(contours, contourList, i, new Scalar(r.nextInt(255), r.nextInt(255), r.nextInt(255)), -1);
}
//Converting Mat back to Bitmap
Bitmap cannyImage2 = Bitmap.createBitmap(cannyImageMat2.cols(),cannyImageMat2.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(cannyImageMat2, cannyImage);
Utils.matToBitmap(contours, cannyImage2);
return cannyImage2;
}
I'm working on Android and OpenCV 3.2, I want to apply the perspective transform but I have some trouble
I referenced to this official document OpenCV (go to perspective transform example ) http://docs.opencv.org/trunk/da/d6e/tutorial_py_geometric_transformations.html
but i get a rotated image result 90° or 180°
intput img = [input img ][2] output img output image 90 ° output 2 [output image 180 °][4]
this is my code
public void prespective(Bitmap img){
Mat imgSrc = new Mat();
Bitmap bmp32 = img.copy(Bitmap.Config.ARGB_8888, true);
Utils.bitmapToMat(bmp32, imgSrc);
Mat gray = new Mat();
cvtColor( imgSrc, gray, COLOR_RGBA2GRAY ); //Convert to gray
Mat thr = new Mat();
threshold( gray, thr, 125, 255, THRESH_BINARY ); //Threshold the gray
List<MatOfPoint> contours = new LinkedList<>(); // Vector for storing contours
findContours( thr, contours,new Mat(), RETR_CCOMP, CHAIN_APPROX_SIMPLE ); // Find the contours in the image
MatOfPoint m = biggestCountousMatOfPoint(contours);
RotatedRect box = Imgproc.minAreaRect(new MatOfPoint2f(m.toArray()));
Rect destImageRect = box.boundingRect();
Mat destImage = new Mat(destImageRect.size(),imgSrc.type());
final Point[] pts = new Point[4];
box.points(pts);
Mat _src = new MatOfPoint2f(pts[0], pts[1], pts[2], pts[3]);
Mat _dst = new MatOfPoint2f(new Point(0, 0), new Point(destImage.width() - 1, 0), new Point(destImage.width() - 1, destImage.height() - 1), new Point(0, destImage.height() - 1));
Mat perspectiveTransform=Imgproc.getPerspectiveTransform(_src, _dst);
Imgproc.warpPerspective(imgSrc, destImage, perspectiveTransform, destImage.size());
if(destImage.height() > 0 && destImage.width() > 0) {
Bitmap cinBitmapRotated = Bitmap.createBitmap(destImage.width(), destImage.height(), Bitmap.Config.ARGB_8888);
if (cinBitmapRotated != null) {
Utils.matToBitmap(destImage, cinBitmapRotated);
prenom2Cin.setImageBitmap(cinBitmapRotated);
}
}
}
what's the problem in my code ?
I am using OpenCV's sample code to detect face with android device. I want to save only detected face area to sd card. I am trying to convert mat to Bitmap and save it. But my problem is it saves whole image rather than just Face. here's my method to convert mat to bitmap
Bitmap bitmap = Bitmap.createBitmap(mGray.cols(), mGray.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mGray, bitmap);
String root = Environment.getExternalStorageDirectory().toString();
File myDir = new File(root + "/saved_images");
myDir.mkdirs();
Random generator = new Random();
int n = 10000;
n = generator.nextInt(n);
String fname = "Image-"+ n +".jpg";
File file = new File (myDir, fname);
if (file.exists ()) file.delete ();
try {
FileOutputStream out = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.JPEG, 90, out);
out.flush();
out.close();
} catch (Exception e) {
e.printStackTrace();
I am a beginner with Opencv. Please help. Thank you in advance
The problem is, you are never trying to get the face pixels. After you detect face, I suggest you to do something such as:
Mat mFaceMatrix = mRgba.submat(facesArray.y, facesArray.y + facesArray.heigth, facesArray.x, facesArray.x + facesArray.width);
Now passing this matrix to createBitmap function should do the trick.
Bitmap bitmap = Bitmap.createBitmap(mFaceMatrix.cols(), mFaceMatrix.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mFaceMatrix, bitmap);
Your code looks fine. I think the problem is with your matrix mGray. It seems that mGray contains the whole image pixels and you are creating bitmap using it. Therefore, my suggestion would be to first check your mGray matrix and take face region and copy the pixels to another matrix and then create bitmap with that new matrix that contains just the face.
Hope it helps.
Let assume that there is only one face. We can crop the result of the face detection and save it as described in this python script:
import cv2
import sys
cascPath = sys.argv[1]
faceCascade = cv2.CascadeClassifier(cascPath)
video_capture = cv2.VideoCapture(0)
while True:
# Capture frame-by-frame
ret, frame = video_capture.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags=cv2.cv.CV_HAAR_SCALE_IMAGE
)
# Draw a rectangle around the faces
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
# Display the resulting frame
cv2.imshow('Video', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
if cv2.waitKey(1) & 0xFF == ord('c'):
crop = frame[y: y + h, x: x + w]
cv2.imwrite("face.jpg", crop)
# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()
I have searched everywhere for a solution but I haven't found any. I know this is a recurrent question, so I'll try to be precise.
I want to perform color segmentation in HSV using Android and OpenCV. I have an array of HSV samples corresponding to the color I want to segment, from which I obtain mean and std in order to obtain a range of values like:
lowerBoundH = new Scalar(meanSamples[0]-stdSamples[0]);
lowerBoundS = new Scalar(meanSamples[1]-stdSamples[1]);
upperBoundH = new Scalar(meanSamples[0]+stdSamples[0]);
upperBoundS = new Scalar(meanSamples[1]+stdSamples[1]);
Once I obtain these ranges for H, S and V I do the following (I only work with channels H & S):
mH = new Mat();
mS = new Mat();
List<Mat> channels = new ArrayList<Mat>(2);
// convert inputFrame to HSV for segmentation
Mat hsvFrame = new Mat();
convertToHSV(inputFrame, hsvFrame);
Core.split(hsvFrame, channels);
// get H&S channels of the HSVimage
mH = channels.get(0);
mS = channels.get(1);
if(mH!=null & mS!=null){
mHinRange = new Mat();
mSinRange = new Mat();
Core.inRange(mH, lowerBoundH, upperBoundH, mHinRange);
Core.inRange(mS, lowerBoundS, upperBoundS, mSinRange);
Mat filtered = new Mat(hsvFrame.height(), hsvFrame.width(), CvType.CV_8UC1, new Scalar(4));
Core.bitwise_and(mHinRange, mSinRange, filtered);
Mat aux = new Mat();
Mat mRgb = new Mat();
inputFrame.copyTo(mRgb);
Imgproc.cvtColor(filtered, mRgb, Imgproc.COLOR_GRAY2BGR, 0);
Imgproc.cvtColor(mRgb, aux, Imgproc.COLOR_BGR2RGBA, 0);
return aux;
}else{
Log.v("NULL","H OR S CHANNELS ARE NULL");
return null;
}
Then, I convert that segmented Mat to Bitmap in order to display it in an ImageView like so:
Bitmap bmp = null;
Mat tmp = new Mat(result.height(),result.width(), CvType.CV_8UC1, new Scalar(4));
try {
bmp = Bitmap.createBitmap(tmp.cols(), tmp.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(tmp, bmp);
}catch (CvException e){Log.d("Exception",e.getMessage());}
qrImage.setImageBitmap(bmp);
But when I display the resulting image I get a completely black image. I know this is wrong because the camera is facing directly to the colored object I want to segment.
What am I doing wrong?
What a stupid mistake. I was creating the Bitmap from the empty tmp Mat.
Instead, I need to create it using the result Mat:
Bitmap bmp = null;
try {
bmp = Bitmap.createBitmap(result.cols(), result.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(result, bmp);
}catch (CvException e){Log.d("Exception",e.getMessage());}
qrImage.setImageBitmap(bmp);