When capturing frames with the camera in my Android app the images contain a lot of noise.
Here are 3 frames i have captured with my device if you look at the on the right of these frames you see that every frame has this dark gray noise. I need the frames to be the same if that is possible.
Does anyone know where this noise is coming from and more important how to get rid of it?
#Override
public void onFocus(byte[] data) {
frame = data;
processFrame();
}
private void processFrame() {
int height = mCameraPreview.getFrameHeight();
int width = mCameraPreview.getFrameWidth();
Mat mYuv = new Mat( height + height/2, width, CvType.CV_8UC1 );
mYuv.put( 0, 0, frame );
Mat mRgba = new Mat();
Imgproc.cvtColor( mYuv, mRgba, Imgproc.COLOR_YUV2RGBA_NV21, 4 );
filter.apply(mRgba);
}
In my filter method I make use of OpenCV template matching to find this specific part of the image. The objectDetected method saves the result as an image.
#Override
public void apply(Mat src) {
Mat match = templateMatch(src, template1, 0.07);
if(match != null){
Mat match2 = templateMatch(match, template2, 0.04);
if(match2 != null){
Mat gray = new Mat(match2.size(), CvType.CV_8U);
Imgproc.cvtColor(match2, gray, Imgproc.COLOR_BGR2GRAY);
objectDetectedListener.objectDetected(gray, "frame");
Related
Im trying to do some image processing using openCV on the array returned from onPreviewFrame(byte[] rawData), i have converted the byte array into int array.
my question is, how to convert this int array into Iplimage or Mat ?
will converting to the new Camera2 be any good?
i have tried this way to convert the array.
void ProcImage(int Width, int Height, int imageArray[]) {
cv::Mat mat = cv::Mat(Width, Height, CV_32SC1 , imageArray);
}
but mat.data contains values like -14545898.
any solutions ?
thanks in advance
This is most likely due to the Mat type CV_32SC1 that you used.
You don't even have to convert the int array, and use the byte array as is
cv::Mat mat = new cv::Mat(Width, Height, CvType.CV_8UC3, byteArray);
where byteArray is your byte array.
If you still want to use the int array trying changing the type to one of Mat: CV_32SC1，CV_32SC2，CV_32SC3，CV_32SC4 as specified in http://ninghang.blogspot.com/2012/11/list-of-mat-type-in-opencv.html
Directly in Java you could do something like this (it is based on the implementation of JavaCameraView):
...
Mat frameData = new Mat(frameHeight + (frameHeight/2), frameWidth, CvType.CV_8UC1);
#Override
public void onPreviewFrame(byte[] frame, Camera arg1) {
Log.d(TAG, "Preview Frame received. Frame size: " + frame.length);
synchronized (this) {
frameData.put(0, 0, frame);
...
this.notify();
}
}
public Mat getRgbaFrame() {
Mat rgba = new Mat();
Imgproc.cvtColor(frameData, rgba, Imgproc.COLOR_YUV2RGBA_NV21, 4);
return mRgba;
}
public Mat getGrayFrame() {
return frameData.submat(0, frameHeight, 0, frameWidth);
}
In my android app, I am trying to take frames in "onPreviewFrame" and convert them to Mat object so i can do on them some image processing, but i don't know how to convert them back to byte[] type and make them be shown on camera preview.
Does any one know how to do that?
Here is my onPreviewFrame function, i am trying to change frame color to grey:
public void onPreviewFrame(byte[] data, Camera arg1)
{
Mat mRgba = new Mat(PreviewSizeWidth, PreviewSizeHeight, CvType.CV_8UC3);
mRgba.put(0, 0, data);
Imgproc.cvtColor(mRgba, mRgba, Imgproc.COLOR_BGR2GRAY);
mRgba.put(0, 0, data);
mCamera.addCallbackBuffer(data);
}
Thanks in advance!
From an older implementation of the OpenCV camera viewer pipeline:
private Mat mYuv, mGraySubmat;
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
if(mYuv == null) {
mYuv = new Mat(height + height / 2, width, CvType.CV_8UC1);
mGraySubmat = mYuv.submat(0, height, 0, width);
}
mYuv.put(0, 0, data);
Imgproc.cvtColor(mGraySubmat, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);
}
I think you can just call the addCallbackBuffer method in the end and it'd work fine!
I'm working with sift using opencv sdk for android
I'm facing difficulties in drawing matches between two images using the
Feature2d.drawMatches()
whenever I run the app it goes through all the steps but stopped when reaching the draw function.
Here's the full code:
public class MainActivity extends AppCompatActivity {
static {
System.loadLibrary("opencv_java");
System.loadLibrary("nonfree");
}
private ImageView imageView;
private FeatureDetector detector = FeatureDetector.create(FeatureDetector.SIFT);
DescriptorExtractor descriptorExtractor=DescriptorExtractor.create(DescriptorExtractor.SIFT);
DescriptorMatcher descriptorMatcher= DescriptorMatcher.create(3);
Bitmap inputImage;
Bitmap inputImage2;
Bitmap out;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
if(OpenCVLoader.initDebug()){
Toast.makeText(this,"Hiii",Toast.LENGTH_SHORT).show();}
inputImage = BitmapFactory.decodeResource(getResources(), R.drawable.object);
inputImage2 = BitmapFactory.decodeResource(getResources(), R.drawable.objecttest);
out = BitmapFactory.decodeResource(getResources(), R.drawable.test);
imageView = (ImageView) this.findViewById(R.id.imageView);
sift();
}
public void onResume() {
super.onResume();
}
public void sift() {
Mat rgba = new Mat();
Mat rgba2 = new Mat();
Mat desc=new Mat();
Mat desc2=new Mat();
MatOfDMatch matches=new MatOfDMatch();
Mat output=new Mat();
Utils.bitmapToMat(inputImage, rgba);
Utils.bitmapToMat(inputImage2, rgba2);
Utils.bitmapToMat(out, output);
MatOfKeyPoint keyPoints = new MatOfKeyPoint();
MatOfKeyPoint keyPoints2 = new MatOfKeyPoint();
Imgproc.cvtColor(rgba, rgba, Imgproc.COLOR_RGBA2GRAY);
Imgproc.cvtColor(rgba2, rgba2, Imgproc.COLOR_RGBA2GRAY);
detector.detect(rgba, keyPoints);
detector.detect(rgba2, keyPoints2);
descriptorExtractor.compute(rgba,keyPoints,desc);
descriptorExtractor.compute(rgba2,keyPoints2,desc2);
descriptorMatcher.match(desc,desc2,matches);
Features2d.drawMatches(rgba,keyPoints,rgba2,keyPoints2,matches,output);
Utils.matToBitmap(output, out);
imageView.setImageBitmap(out);
}
}
I also need to know whether sift is the best choice for the use of detecting complicated objects in real time?
Edit 1: I notice now that I'm getting the following errors
OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows)
Fatal signal 11 (SIGSEGV), code 2, fault addr 0x12ce08e0 in tid 9971
Problem solved:
the error I've mentioned was preventing the image from being displayed for some reason( maybe the difference of properties between the image view and the image we want to display let's say the dimensions in some cases
actually I'm not sure about the previous point but it was mentioned out there, and the solution was to set the image view to be dynamically resized according to the image but that didn't work for me)
My problem was solved by replacing these instructions:
Features2d.drawMatches(rgba,keyPoints,rgba2,keyPoints2,matches,output);
Utils.matToBitmap(output, out);
imageView.setImageBitmap(out);
with the following:
Scalar RED = new Scalar(255,0,0);
Scalar GREEN = new Scalar(0,255,0);
MatOfByte drawnMatches = new MatOfByte();
Features2d.drawMatches(rgba,keyPoints,rgba2,keyPoints2,matches,output,GREEN, RED, drawnMatches, Features2d.NOT_DRAW_SINGLE_POINTS);
Bitmap imageMatched = Bitmap.createBitmap(output.cols(), output.rows(), Bitmap.Config.RGB_565);//need to save bitmap
Utils.matToBitmap(output, imageMatched);
imageView.setImageBitmap(imageMatched);
I didn't actually get the difference, so further help describing this will be so helpful, but hope this can temporarily help people having the same problem
I am building an Android Application and I want to be able to detect black circles. I am using OpenCV3 for Android and I am able to filter out the black colours from my camera feed using the code below. Kindly note that I used the Color-blob-detection example given in the Android SDK and tweaked the code as such:
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
mRgba = inputFrame.rgba();
if (mIsColorSelected) {
Bitmap resultBitmap;
resultBitmap = Bitmap.createBitmap(mRgba.cols(), mRgba.rows(),Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mRgba, resultBitmap);
//TODO - look for circles
Mat mat = new Mat(resultBitmap.getWidth(), resultBitmap.getHeight(), CvType.CV_8UC1);
Utils.bitmapToMat(resultBitmap, mat);
final Bitmap bitmap;
//TODO - filter out the black only
Mat mHSV = mat;
Mat mHSVThreshed = mat;
Mat mRgba2 = mat;
Imgproc.cvtColor(mat, mHSV, Imgproc.COLOR_BGR2HSV, 0);
Core.inRange(mHSV, new Scalar(0, 0, 0), new Scalar(130, 130, 130), mHSVThreshed);
Imgproc.cvtColor(mHSVThreshed, mat, Imgproc.COLOR_GRAY2BGR, 0);
Imgproc.cvtColor(mat, mRgba2, Imgproc.COLOR_BGR2RGBA, 0);
Imgproc.GaussianBlur(mRgba2, mRgba2, new Size(9, 9), 2, 2);
//this is for displaying purposes only.
//At this point, the image would be black and white, where the white spots are the black detected blobs
// Bitmap bmp = Bitmap.createBitmap(mRgba2.cols(), mRgba2.rows(), Bitmap.Config.ARGB_8888);
//Utils.matToBitmap(mRgba2, bmp);
//bitmap = bmp; //resultBitmap;
//TODO - new circle detection code: this uses the colour filtered Mat
mat = mRgba2;
Imgproc.HoughCircles(mat, circles,
Imgproc.CV_HOUGH_GRADIENT, 1, minDist, 100,
20, 0, 0);
/* get the number of circles detected */
int numberOfCircles = (circles.rows() == 0) ? 0 : circles.cols();
/* draw the circles found on the image */
for (int i = 0; i < numberOfCircles; i++) {
/* get the circle details, circleCoordinates[0, 1, 2] = (x,y,r)
* (x,y) are the coordinates of the circle's center
*/
double[] circleCoordinates = circles.get(0, i);
int x = (int) circleCoordinates[0], y = (int) circleCoordinates[1];
Point center = new Point(x, y);
int radius = (int) circleCoordinates[2];
/* circle's outline */
Imgproc.circle(mRgba2, center, radius, new Scalar(0,
200, 255), 4);
/* circle's center outline */
Imgproc.rectangle(mRgba2, new Point(x - 5, y - 5),
new Point(x + 5, y + 5),
new Scalar(0, 200, 255), -1);
}
Utils.matToBitmap(mRgba2, resultBitmap);
bitmap = resultBitmap;
runOnUiThread(new Runnable() {
#Override
public void run() {
mOpenCvCameraView.disableView();
mOpenCvCameraView.setVisibility(SurfaceView.GONE);
imageView.setVisibility(View.VISIBLE);
imageView.setImageBitmap(bitmap);
}
});
}
return mRgba;
}
What my code does is that it takes a snapshot of the camera feed as a Mat, and then using that Mat, I do some image processing on it and to black out everything except the black colour. The resulting Mat is the mRgba2 variable and I converted to Bitmap and displayed on an ImageView. I displayed this Bitmap to confirm that I am getting the result I want.
After I know that I'm able to filter out the colour I wanted, I then run a GaussianBlur on it then proceed to run HoughCircles. However, when I run the
Imgproc.HoughCircles(mat, circles,
Imgproc.CV_HOUGH_GRADIENT, 1, minDist, 100,
20, 0, 0);
line, I get an
The source image must be 8-bit, single-channel in function CvSeq* cvHoughCircles(CvArr*, void*, int, double, double, double, double, int, int)
Error.
I know that the code wherein I run HoughCircles on a given mat variable works, because I tested it before. Now, changing the mat variable I feed onto it does not, and I wonder what I did differently for the code to not work.
HoughCircles runs only on grayscale (CV8U) images.
Replacing
Imgproc.cvtColor(mat, mRgba2, Imgproc.COLOR_BGR2RGBA, 0);
with
Imgproc.cvtColor(mat, mRgba2, Imgproc.COLOR_BGR2GRAY, 0);
should remove the error.
Then the problem is to detect circles of a given color. This won't pose much of a problem in your case, because you are detecting black circles anyway. Instead of black circles, this will detect dark circles.
If you have dark colors other than black in some of the circles, you can filter them out separately by looking at the pixel values in a different color space.
I get a runtime error if I try to convert camera preview YUV byte array
to a RGB(A) byte array with Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV420sp2RGBA, 4 )
in onPreviewFrame(byte[] data, Camera camera):
Preview.java:
mCamera.setPreviewCallback(new PreviewCallback() {
public void onPreviewFrame(byte[] data, Camera camera)
{
// Pass YUV data to draw-on-top companion
System.arraycopy(data, 0, mDrawOnTop.mYUVData, 0, data.length);
mDrawOnTop.invalidate();
}
});
DrawOnTop.java:
public class DrawOnTop extends View {
Bitmap mBitmap;
Mat mYUV_Mat;
protected void onDraw(Canvas canvas) {
if (mBitmap != null)
{
canvasWidth = canvas.getWidth();
canvasHeight = canvas.getHeight();
int newImageWidth = 640;
int newImageHeight = 480;
marginWidth = (canvasWidth - newImageWidth)/2;
if( mYUV_Mat != null ) mYUV_Mat.release();
//mYUV_Mat = new Mat( newImageWidth, newImageHeight, CvType.CV_8UC1 );
mYUV_Mat = new Mat( newImageWidth, newImageHeight, CvType.CV_8UC4 );
mYUV_Mat.put( 0, 0, mYUVData );
//Mat mRgba_Mat = new Mat();
Mat mRgba_Mat = new Mat(newImageWidth,newImageHeight,CvType.CV_8UC4);
//Mat mRgba_Mat = mYUV_Mat;
//Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV2RGBA_NV21, 4 );
//Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV420sp2RGB, 4 );
Imgproc.cvtColor( mYUV_Mat, mRgba_Mat, Imgproc.COLOR_YUV420sp2RGBA, 4 );
// Draw Bitmap New:
Bitmap mBitmap = Bitmap.createBitmap( newImageWidth, newImageHeight, Bitmap.Config.ARGB_8888 );
Utils.matToBitmap( mRgba_Mat, mBitmap );
mRgba_Mat.release();
}
}
The conversion mYUV_Mat.put( 0, 0, mYUVData ) runs correctly.
But the attempts to convert mYUV_Mat to mRgb_Mat using Imgproc.cvtColor
lead all to runtime errors ("Source not found." with Eclipse).
What is the correct Imgproc.cvtColor command for my goal?
(I don't want to use a Java YUV2RGB(A) decode method because it's to slow
for image processing. I want to use the OpenCV Imgproc.cvtColor method
because I can do native calls)
Maybe the Imgproc library isn't properly included in your project, but other OpenCV libraries are? The line that crashes is the first line where you use a method from Imgproc, which would explain why earlier parts of the code run correctly.
Your code looks fine, except you can use the no-argument constructor for mRgba_Mat (since most OpenCV4Android functions, including cvtColor, can infer the required size of the destination matrix), and you're allocating a lot of wasted space for mYUV_Mat. You don't need a full 4 channels if you just allocate YUV matrices 50% more space than their RGB counterparts:
mYUV_Mat = new Mat( newImageHeight + newImageHeight / 2, newImageWidth, CvType.CV_8UC1 );