I am trying to plot the R channel of an image as a heatmap using the following code:
// Read image as a Mat of 32bit float
Mat imgMain = new Mat(n_height, n_width, CvType.CV_32FC4);
image = BitmapFactory.decodeFile("img.jpg");
Utils.bitmapToMat(image, imgMain);
imgMain.convertTo(imgMain, CvType.CV_32FC4);
// Extract R channel
Mat imgChR = new Mat(n_height, n_width, CvType.CV_32FC1);
extractChannel(imgMain, imgChR, 0);
// imgChR processed with floating point arithmetic
// Convert to HeatMap
normalize(imgChR, imgChR, 0, 1, NORM_MINMAX);
Mat heatOut = new Mat(n_height, n_width, CvType.CV_8UC4);
applyColorMap(imgChR, heatOut, COLORMAP_JET);
// Save as Image
pimage = Bitmap.createBitmap(n_width, n_height, Bitmap.Config.ARGB_8888);
Utils.matToBitmap(heatOut, pimage);
File f = new File("heat.jpeg");
FileOutputStream fos = new FileOutputStream(f);
pimage.compress(Bitmap.CompressFormat.JPEG, 100, fos);
fos.close();
For averaging multiple copies of the image, I am converting the matrix from integers to floats. (The averaging process is abstracted in the following code but please note that the type has to be converted to a 32bit float)
Using the following code, I simply get a black image.
Is this the right approach for plotting a heatmap of a single channel matrix?
If not, is there any other function or approach that can be used?
(When I check the type of heatOut the function returns a 0 as a integer representation of the Mat type.)
Related
> Hi, I am using OpenCV android library grabcut() method to extract an image from background, but the problem is that the output bitmap contains background Same as original image and object become white .I need Object as its same as original image and background transparent
I am using this code
private static Bitmap makeBlackTransparent(Bitmap image) {
// convert image to matrix
Mat src = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
Utils.bitmapToMat(image, src);
// init new matrices
Mat dst = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
Mat tmp = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
Mat alpha = new Mat(image.getWidth(), image.getHeight(), CvType.CV_8UC4);
// convert image to grayscale
Imgproc.cvtColor(src, tmp, Imgproc.COLOR_BGR2GRAY);
// threshold the image to create alpha channel with complete transparency in black background region and zero transparency in foreground object region.
Imgproc.threshold(tmp, alpha, 100, 255, Imgproc.THRESH_BINARY);
// split the original image into three single channel.
List<Mat> rgb = new ArrayList<Mat>(3);
Core.split(src, rgb);
// Create the final result by merging three single channel and alpha(BGRA order)
List<Mat> rgba = new ArrayList<Mat>(4);
rgba.add(rgb.get(0));
rgba.add(rgb.get(1));
rgba.add(rgb.get(2));
rgba.add(alpha);
Core.merge(rgba, dst);
// convert matrix to output bitmap
Bitmap output = Bitmap.createBitmap(image.getWidth(), image.getHeight(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(dst, output);
return output;
}
There are 2 problems in your code:
Firstly you need to segment out the white background, so adjust your thresh to be near 220 - 240 and also use THRESH_BINARY_INV instead of THRESH_BINARY :
Imgproc.threshold(tmp, alpha, 230, 255, Imgproc.THRESH_BINARY_INV);
Secondly, you must pre-multiply the ARGB layers, as Android ImageView behaves weird without premultiplication, for that you need to use cvtColor with COLOR_RGBA2mRGBA flag:
// Create the final result by merging three single channel and alpha(BGRA order)
List<Mat> rgba = new ArrayList<Mat>(4);
rgba.add(rgb.get(0));
rgba.add(rgb.get(1));
rgba.add(rgb.get(2));
rgba.add(alpha);
Core.merge(rgba, dst);
Imgproc.cvtColor(dst, dst, Imgproc.COLOR_RGBA2mRGBA);
i trying convert image from YUV_420_888 to rgb and i have some trouble with output image. In ImageReader i get image in format YUV_420_888 (using camera 2 api for get this image preview).
imageReader = ImageReader.newInstance(1920,1080,ImageFormat.YUV_420_888,10);
In android sdk for YuvImage class writing, that YuvImage using only NV21, YUY2.
as we can see difference between N21 and yuv420 not big and i try convert data to N21
YUV420:
and N21:
in onImageAvailable i get separately each Planes and put them in correct place (as on image)
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ByteBuffer bufferY = image.getPlanes()[0].getBuffer();
byte[] data0 = new byte[bufferY.remaining()];
bufferY.get(data0);
ByteBuffer bufferU = image.getPlanes()[1].getBuffer();
byte[] data1 = new byte[bufferU.remaining()];
bufferU.get(data1);
ByteBuffer bufferV = image.getPlanes()[2].getBuffer();
byte[] data2 = new byte[bufferV.remaining()];
bufferV.get(data2);
...
outputStream.write(data0);
for (int i=0;i<bufferV.remaining();i++) {
outputStream.write(data1[i]);
outputStream.write(data2[i]);
}
after create YuvImage, convert to Bitmap, view and save
final YuvImage yuvImage = new YuvImage(outputStream.toByteArray(), ImageFormat.NV21, 1920,1080, null);
ByteArrayOutputStream outBitmap = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(0, 0,1920, 1080), 95, outBitmap);
byte[] imageBytes = outBitmap.toByteArray();
final Bitmap imageBitmap = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
mImageView.setImageBitmap(imageBitmap);
...
imageBitmap.compress(Bitmap.CompressFormat.JPEG, 95, out);
but my saved image is green and pink:
what did i miss??
I have implemented the YUV_420 logic (exactly as shown in the above diagram) in RenderScript, see full code here:
Conversion YUV_420 _888 to Bitmap, complete code
It produces perfect bimaps for API 22, but for API 21 it shows the "green idyll". From this I can confirm, the results you found. As already mentioned by Silvaren above, the reason seems to be an Android bug in API 21. Looking at my rs code it is clear, that if U and V information is missing (i.e. zero) the G(reen) ARGB component becomes huge during the conversion.
I see similar green pictures on my Galaxy S5 (still API 21) - here even upside down ;-). I suspect that most devices at API 21 currently do not yet use Camera2 for their device-camera apps. There is a free app called "Manual Camera Compatibility" which allows to test this. From this I see that indeed the S5/API21 still not uses Camera2...fortunately not...
There are two main problems on your conversion attempt:
We can not assume that the U and V planes are isolated, they might contain interleaved data (e.g. U-PLANE = {U1, V1, U2, V2, ...} ). In fact, it might even be a NV21 style interleaving already. The key here is looking at the plane's row stride and pixel stride and also check what we can assume about the YUV_420_888 format.
The fact that you've commented that most of the U an V planes data are zeros indicates that you are experiencing an Android bug on the generation of images in YUV_420_888. This means that even if you get the conversion right, the image would still look green if you are affected by the bug, which was only fixed at the Android 5.1.1 and up, so it is worth to check which version you are using besides fixing the code.
bufferV.get(data2) increases the the position of the ByteBuffer. That's why the loop for (int i=0;i<bufferV.remaining();i++) produces 0 iterations. You can easily rewrite it as
for (int i=0; i<data1.length; i++) {
outputStream.write(data1[i]);
outputStream.write(data2[i]);
}
I got an image of ImageFormat.YUV_420_888 and was successful to save to jpeg file, and could view it correctly on windows.
I am sharing here :
private final Image mImage;
private final File mFile;
private final int mImageFormat;
ByteArrayOutputStream outputbytes = new ByteArrayOutputStream();
ByteBuffer bufferY = mImage.getPlanes()[0].getBuffer();
byte[] data0 = new byte[bufferY.remaining()];
bufferY.get(data0);
ByteBuffer bufferU = mImage.getPlanes()[1].getBuffer();
byte[] data1 = new byte[bufferU.remaining()];
bufferU.get(data1);
ByteBuffer bufferV = mImage.getPlanes()[2].getBuffer();
byte[] data2 = new byte[bufferV.remaining()];
bufferV.get(data2);
try
{
outputbytes.write(data0);
outputbytes.write(data2);
outputbytes.write(data1);
final YuvImage yuvImage = new YuvImage(outputbytes.toByteArray(), ImageFormat.NV21, mImage.getWidth(),mImage.getHeight(), null);
ByteArrayOutputStream outBitmap = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(0, 0,mImage.getWidth(), mImage.getHeight()), 95, outBitmap);
FileOutputStream outputfile = null;
outputfile = new FileOutputStream(mFile);
outputfile.write(outBitmap.toByteArray());
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
mImage.close();
}
I am using OpenCV's sample code to detect face with android device. I want to save only detected face area to sd card. I am trying to convert mat to Bitmap and save it. But my problem is it saves whole image rather than just Face. here's my method to convert mat to bitmap
Bitmap bitmap = Bitmap.createBitmap(mGray.cols(), mGray.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mGray, bitmap);
String root = Environment.getExternalStorageDirectory().toString();
File myDir = new File(root + "/saved_images");
myDir.mkdirs();
Random generator = new Random();
int n = 10000;
n = generator.nextInt(n);
String fname = "Image-"+ n +".jpg";
File file = new File (myDir, fname);
if (file.exists ()) file.delete ();
try {
FileOutputStream out = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.JPEG, 90, out);
out.flush();
out.close();
} catch (Exception e) {
e.printStackTrace();
I am a beginner with Opencv. Please help. Thank you in advance
The problem is, you are never trying to get the face pixels. After you detect face, I suggest you to do something such as:
Mat mFaceMatrix = mRgba.submat(facesArray.y, facesArray.y + facesArray.heigth, facesArray.x, facesArray.x + facesArray.width);
Now passing this matrix to createBitmap function should do the trick.
Bitmap bitmap = Bitmap.createBitmap(mFaceMatrix.cols(), mFaceMatrix.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mFaceMatrix, bitmap);
Your code looks fine. I think the problem is with your matrix mGray. It seems that mGray contains the whole image pixels and you are creating bitmap using it. Therefore, my suggestion would be to first check your mGray matrix and take face region and copy the pixels to another matrix and then create bitmap with that new matrix that contains just the face.
Hope it helps.
Let assume that there is only one face. We can crop the result of the face detection and save it as described in this python script:
import cv2
import sys
cascPath = sys.argv[1]
faceCascade = cv2.CascadeClassifier(cascPath)
video_capture = cv2.VideoCapture(0)
while True:
# Capture frame-by-frame
ret, frame = video_capture.read()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.1,
minNeighbors=5,
minSize=(30, 30),
flags=cv2.cv.CV_HAAR_SCALE_IMAGE
)
# Draw a rectangle around the faces
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
# Display the resulting frame
cv2.imshow('Video', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
if cv2.waitKey(1) & 0xFF == ord('c'):
crop = frame[y: y + h, x: x + w]
cv2.imwrite("face.jpg", crop)
# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()
I'm doing a watershed segmentation and the marker image is derived from the source image put through a distance transform. The distance transform returns a floating point image (I have no idea about the bit-depth) and I have trouble putting it through the watershed method since it requires a 32 bit single channel image.
Can I use the mat's convertTo method to set the bit depth to 32?
I also have trouble trying to display the floating point image since the matToBitmap() method doesn't seem to accept them. (in Android)
Mat mImg = new Mat();
Mat mThresh = new Mat();
Mat mDist = new Mat();
ImageView imgView = (ImageView) findViewById(R.id.imageView);
Bitmap bmpIn = BitmapFactory.decodeResource(getResources(),
R.drawable.w1);
Utils.bitmapToMat(bmpIn, mImg);
Imgproc.cvtColor(mImg, mImg, Imgproc.COLOR_BGR2GRAY);
Imgproc.threshold(mImg, mThresh, 0, 255, Imgproc.THRESH_BINARY
| Imgproc.THRESH_OTSU);
//Marker image for watershed
Imgproc.distanceTransform(mThresh, mDist, Imgproc.CV_DIST_L2, Imgproc.CV_DIST_MASK_PRECISE);
//Conversions for watershed
Imgproc.cvtColor(mThresh, mThresh, Imgproc.COLOR_GRAY2BGR, 3);
//Floating-point image -> 32-bit single-channel
mDist.convertTo(...);
Imgproc.watershed(mThresh, mDist); //
Bitmap bmpOut = Bitmap.createBitmap(mThresh.cols(), mThresh.rows(),
Bitmap.Config.ARGB_8888);
Utils.matToBitmap(mThresh, bmpOut);
imgView.setImageBitmap(bmpOut);
Yes, you can use the convertTo function to convert any opencv matrix to another type. The type to convert to should be set in a destination matrix with the same size. convertTo has optional parameters scale and shift, so you can avoid clipping and quantization errors when converting to fixed point depths. So for your code:
Mat mDist32 = Mat(mDist.rows,mDist.cols,CV_32SC1); // 32 bit signed 1 channel, use CV_32UC1 for unsigned
mDist.convertTo(mDist32,CV_32SC1,1,0);
Imgproc.watershed(mThresh,mDist32);
I am reading a raw image from the network. This image has been read by an image sensor, not from a file.
These are the things I know about the image:
~ Height & Width
~ Total size (in bytes)
~ 8-bit grayscale
~ 1 byte/pixel
I'm trying to convert this image to a bitmap to display in an imageview.
Here's what I tried:
BitmapFactory.Options opt = new BitmapFactory.Options();
opt.outHeight = shortHeight; //360
opt.outWidth = shortWidth;//248
imageBitmap = BitmapFactory.decodeByteArray(imageArray, 0, imageSize, opt);
decodeByteArray returns null, since it cannot decode my image.
I also tried reading it directly from the input stream, without converting it to a Byte Array first:
imageBitmap = BitmapFactory.decodeStream(imageInputStream, null, opt);
This returns null as well.
I've searched on this & other forums, but cannot find a way to achieve this.
Any ideas?
EDIT: I should add that the first thing I did was to check if the stream actually contains the raw image. I did this using other applications `(iPhone/Windows MFC) & they are able to read it and display the image correctly. I just need to figure out a way to do this in Java/Android.
Android does not support grayscale bitmaps. So first thing, you have to extend every byte to a 32-bit ARGB int. Alpha is 0xff, and R, G and B bytes are copies of the source image's byte pixel value. Then create the bitmap on top of that array.
Also (see comments), it seems that the device thinks that 0 is white, 1 is black - we have to invert the source bits.
So, let's assume that the source image is in the byte array called Src. Here's the code:
byte [] src; //Comes from somewhere...
byte [] bits = new byte[src.length*4]; //That's where the RGBA array goes.
int i;
for(i=0;i<src.length;i++)
{
bits[i*4] =
bits[i*4+1] =
bits[i*4+2] = ~src[i]; //Invert the source bits
bits[i*4+3] = 0xff; // the alpha.
}
//Now put these nice RGBA pixels into a Bitmap object
Bitmap bm = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
bm.copyPixelsFromBuffer(ByteBuffer.wrap(bits));
Once I did something like this to decode the byte stream obtained from camera preview callback:
Bitmap.createBitmap(imageBytes, previewWidth,　previewHeight,
Bitmap.Config.ARGB_8888);
Give it a try.
for(i=0;i<src.length;i++)
{
bits[i*4] = bits[i*4+1] = bits[i*4+2] = ~src[i]; //Invert the source bits
bits[i*4+3] = 0xff; // the alpha.
}
The conversion loop can take a lot of time to convert the 8 bit image to RGBA, a 640x800 image can take more than 500ms... A quicker solution is to use ALPHA8 format for the bitmap and use a color filter:
//setup color filter to inverse alpha, in my case it was needed
float[] mx = new float[]{
1.0f, 0, 0, 0, 0, //red
0, 1.0f, 0, 0, 0, //green
0, 0, 1.0f, 0, 0, //blue
0, 0, 0, -1.0f, 255 //alpha
};
ColorMatrixColorFilter cf = new ColorMatrixColorFilter(mx);
imageView.setColorFilter(cf);
// after set only the alpha channel of the image, it should be a lot faster without the conversion step
Bitmap bm = Bitmap.createBitmap(width, height, Bitmap.Config.ALPHA_8);
bm.copyPixelsFromBuffer(ByteBuffer.wrap(src)); //src is not modified, it's just an 8bit grayscale array
imageview.setImageBitmap(bm);
Use Drawable create from stream. Here's how to do it with an HttpResponse, but you can get the inputstream anyway you want.
InputStream stream = response.getEntity().getContent();
Drawable drawable = Drawable.createFromStream(stream, "Get Full Image Task");