I am using com.otaliastudios.cameraview.CameraView component from otaliastudios Library. I need to convert some of the frames to Mat objects to process using OpenCV. How can I convert Otaliastudios Frame object into OpenCV Mat object?
Edit: The frame class I am using is located here: github.com/natario1/CameraView/blob/master/cameraview/src/main/java/com/otaliastudios/cameraview/Frame.java
How can I know which Image format this is? Does it make a difference?
You need to know the source format of your phone camera frame.
The Frame object contains a byte[] data field.This field is probably in the same ImageFormat of your camera. The two most common formats are NV21 and YUV_420_888.
The YUV format is composed by a component that is the luminance Y and another component that is called chrominance (U-V).
Typically the relation (and consequently the real bit/byte size) of these two components is defined by methods that reduce the chrominance component because human eyes are more sensible to luminance variations than color variations (see Chroma Subsampling). The reduction is expressed by a set of numbers like 4:2:0.
In this case the part related to chrominance is half the size of luminance.
So the byte size of a Frame has probably a part for the luminance that is equal to width X height bytes and a part of chrominance that is width X (height/2) bytes. This means that the byte size is heavily dependent on the image format that you are acquiring and you have to modify the mat size and choose the CvType according to this.
You have to allocate a mat that has the same size of your frame and put the data into it(from this answer):
mYuv = new Mat(getFrameHeight() + getFrameHeight() / 2,
getFrameWidth(), CvType.CV_8UC1);
....
mYuv.put(0, 0, data);
And then you have your Mat. If you need also to convert to an RGB format check the bottom of this page.
Hope this will help you.
Related
I am trying to convert the YUV frames to JPEG for Mavic 2 Pro. This is done using the newSaveYuvDataToJPEG420P or newSaveYuvDataToJPEG functions in repo.
However, I observe that the incoming data is not complete, yuvFrame.length < width * height the data size is not equal to the product of width and height. So the data is not processed.
What may be the reason for this discrepancy?
There are multiple YUV formats
See https://developer.android.com/reference/android/graphics/ImageFormat#YUV_420_888
Mostly likely the format used is YUV_420_888 as this is more common for cameras which representation, 8 bits for Y and 4 bits for UV (interleaved).
Thus yuvFrame.length = width * height * 12 bits or (6 bytes per 4 pixels)
The frame will have all 3 colour channels in it's length so it's length would never equal width times height in bytes.
Where as YUV444 is 3 bytes per pixel
So that test in the code is logically wrong as would not detect some incomplete YUV frames. The correct test would be yuvFrame.length < width * height * 1.5f
The only reason I can think of why it is failing this incorrect test is that the image format is not YUV, a lot of cameras can output jpeg natively as a jpg frame could fail this test.
Note that you can use the Android YuvImage class to convert YUV images to jpeg
I'm downloading image data over the network and obviously it's compressed. Inside the app I need the data to be 100% accurate to the original. So for example for a 720x720 image I need a 518400 length int array or 2073600 length byte array where the values have not been modified from the original. Since PNG compression is lossless this should be possible.I noticed that BitmapFactory applies some psychovisual effects when decoding that aren't noticeable when looking at the image, however change the byte values ever so slightly (I'm assuming this is for speed and/or better visual look on screen).
What I'm currently doing is using BitmapFactory.decode and then iterating over every pixel to create a new array from approximation and creating a new bitmap with Bitmap.create This is possible because for now the images are binary and in the original images each pixel is either (r, g, b, 255) or (0, 0, 0, 0). I can find the most prevalent colour and set all pixels with alpha values over some threshold to this RGB value. The solution is quite slow though and seems needlessly complicated.
So my question is - is there any combination of options flags for BitmapFactory.decode that returns a 100% true to the original Bitmap or do I have to create a PNG decoder myself to achieve this?
Have a try with this method:
val opts = BitmapFactory.Options()
opts.inPremultiplied = false
var bitmapDecode = BitmapFactory.decodeByteArray(decode, 0, decode.size, opts)
In OpenCV I retrieve a Gabor kernel for image processing which is a 9:9 matrix using:
Imgproc.getGaborKernel(...)
I have a gray matrix of the original image. (i'm not even sure if the kernel is supposed to be the size of the image or just a small segment, I'm fairly certain of the small kernel)
How do I convolve the two and get the output of the convolution?
I'm trying to put together a Gabor wavelet filter for edge detection.
EDIT: as far as convolution of matrices seems to be concerned it looks like the opencv "filter2d" method is what is used to do it and is found in Imgproc class of Android OpenCV api.
However when I do my convolution and put it to the screen its just a black image.
Size size = new Size(9,9);
Mat gaborKernel = Imgproc.getGaborKernel(size, 3.0, -Math.PI/4, Math.PI, 10.0, Math.PI*0.5, CvType.CV_64F);
Imgproc.filter2D(intermediate, output, -1, gaborKernel);
Bitmap temp = Bitmap.createBitmap(intermediate.cols(), intermediate.rows(), Config.ARGB_8888);
Utils.matToBitmap(output, temp);
I did a system output to see the values and all of the values are extremely small as seen below.
You need to normalize your kernel.
Just loop over the kernel matrix, calculate the sum of values. Then loop again to divide each value to the sum. This ensures that your kernel does not change the overal brightness.
I try to visualize the gradiants and angles of an image which computed by the HOGDescriptor of the OpenCV Lib for Android. At the begin i have an 3 channel image Mat() with 8 bit unsigned int (CV_8UC3). The result of the computation is a MAT() (CV_32FC2) of the gradiants and a Mat() (CV_8UC2) of the angles. How can i visualize this results? What represent the values? Why have the angle Mat() 2 channels? Are the 2 channels of the gradiant Mat() the x and y component of the gradiant? I cant find documentation of the computeGradiant-Method.
HOG descriptor is an histogram of oriented gradient: it is an histogram where each bin reprezent the vote for gradient in corresponding orientation.
In order to compute this descriptor, you should first convert you 3 channels color image into a grayscale image
cv::cvtColor(CV_BGR2GRAY);
The result of "ComputeGradient" method is for exemple two images (same size as the original): x-component and y-component.
You should then be able to compute for each pixel the gradient magnitude and orientation.
mag=sqrt(x*x+y*y)
alpha=atan(y/x)
Then you can fill you histogram. Note that HOG descritpor is computed by blocks and cells. See this for more detail.
Please we need help urgently, we are using openCv in Android (Java).
We are facing a lot of problems:
convertTo() doesn't work so we can't convert 3 channel image to 1 channel without passing it on cvtColor().
grayImg.convertTo(grayImg, CvType.CV_8UC1);
cvtColor() gives a weird output:
Imgproc.cvtColor(src, grayImg, Imgproc.COLOR_RGB2GRAY);
Output of this line is the image repeated 4 times!
The only way to get rid of this repetition is to add this line and the output is a white and black image but 3 channel so it crashes any coming function because it needs 1 channel image.
Imgproc.cvtColor(grayImg, grayImg, Imgproc.COLOR_GRAY2RGB,3);
canny() for edge detection:
Imgproc.Canny(grayImg, grayImg, 10, 100,3,true);
findContours() counts a horrible number of contours while number of objects in the image is only 2 input image is 3 channel bmp image and we convert it to Mat.
output image:
https://dl.dropbox.com/u/36214963/canny.jpg
Thanks for your concern
Try BGR2GRAY rather than RGB2GRAY.I had the same problem and I solved it through this.There is also a note in the documentation about this
Converts an image from one color space to another.
The function cvtColor converts an input image from one color space to another. In case of a transformation to-from RGB color space, the order of the channels should be specified explicitly (RGB or BGR). Note that the default color format in OpenCV is often referred to as RGB but it is actually BGR (the bytes are reversed). So the first byte in a standard (24-bit) color image will be an 8-bit Blue component, the second byte will be Green, and the third byte will be Red. The fourth, fifth, and sixth bytes would then be the second pixel (Blue, then Green, then Red), and so on.
If I understand your first question correctly, you have two options to convert RGB images to grayscale ones.
Option 1: Convert the 3 channel image to 1 channel as you are trying to do.
IplImage *RGB_image = cvLoadImage("my_colored_image.jpg");
IplImage *GRAY_IMAGE = cvCreateImage(cvGetSize(RGB_image), IPL_DEPTH_8U, 1);
cvCvtColor(RGB_image, GRAY_IMAGE, CV_RGB2GRAY);
Option 2: Read the colored image as a grayscale image directly.
IplImage* GRAY_IMAGE = cvLoadImage("my_colored_image.jpg", CV_LOAD_IMAGE_GRAYSCALE);
I hope this suits you.
I haven't actually used opencv before, but I don't think convertTo is the answer your looking for.
By looking at the opencv documentation I found this:
cvtColor - Converts an image from one color space to another
Mat color; // the input image
Mat gray(color.rows, color.cols, color.depth());
cvtColor(color, gray, CV_BGR2GRAY);
Or simply (and the function cvtColor will create the image internally):
Mat color;
Mat gray;
cvtColor(color, gray, CV_BGR2GRAY);