I have done some google-ing around and couldn't find enough information about this format. It is the default format for camera preview. Can anyone suggest good sources of information about it and how to extract data from a photo/preview image with that format? To be more specific, I need the black and white image extracted.
EDIT: Seems like that format is also called YCbCr 420 Semi Planar
I developed the following code to convert the NV21 to RGB, and it is working.
/**
* Converts YUV420 NV21 to RGB8888
*
* #param data byte array on YUV420 NV21 format.
* #param width pixels width
* #param height pixels height
* #return a RGB8888 pixels int array. Where each int is a pixels ARGB.
*/
public static int[] convertYUV420_NV21toRGB8888(byte [] data, int width, int height) {
int size = width*height;
int offset = size;
int[] pixels = new int[size];
int u, v, y1, y2, y3, y4;
// i percorre os Y and the final pixels
// k percorre os pixles U e V
for(int i=0, k=0; i < size; i+=2, k+=2) {
y1 = data[i ]&0xff;
y2 = data[i+1]&0xff;
y3 = data[width+i ]&0xff;
y4 = data[width+i+1]&0xff;
u = data[offset+k ]&0xff;
v = data[offset+k+1]&0xff;
u = u-128;
v = v-128;
pixels[i ] = convertYUVtoRGB(y1, u, v);
pixels[i+1] = convertYUVtoRGB(y2, u, v);
pixels[width+i ] = convertYUVtoRGB(y3, u, v);
pixels[width+i+1] = convertYUVtoRGB(y4, u, v);
if (i!=0 && (i+2)%width==0)
i+=width;
}
return pixels;
}
private static int convertYUVtoRGB(int y, int u, int v) {
int r,g,b;
r = y + (int)(1.402f*v);
g = y - (int)(0.344f*u +0.714f*v);
b = y + (int)(1.772f*u);
r = r>255? 255 : r<0 ? 0 : r;
g = g>255? 255 : g<0 ? 0 : g;
b = b>255? 255 : b<0 ? 0 : b;
return 0xff000000 | (b<<16) | (g<<8) | r;
}
This image helps to understand.
If you wanna just grayscale image is easer. You can discard all the U and V info, and take just the Y info. The code would can be like this:
/**
* Converts YUV420 NV21 to Y888 (RGB8888). The grayscale image still holds 3 bytes on the pixel.
*
* #param pixels output array with the converted array o grayscale pixels
* #param data byte array on YUV420 NV21 format.
* #param width pixels width
* #param height pixels height
*/
public static void applyGrayScale(int [] pixels, byte [] data, int width, int height) {
int p;
int size = width*height;
for(int i = 0; i < size; i++) {
p = data[i] & 0xFF;
pixels[i] = 0xff000000 | p<<16 | p<<8 | p;
}
}
To create your Bitmap just:
Bitmap bm = Bitmap.createBitmap(pixels, width, height, Bitmap.Config.ARGB_8888);
Where pixels is your int [] array.
NV21 is basically YUV420 but instead of planar format where Y, U and V have independent planes, NV21 has 1 plane for Luma and 2nd plane for Chroma. The format looks like
YYYYYYYYYYYYYYYYYYYYYYYYYYYYY
YYYYYYYYYYYYYYYYYYYYYYYYYYYYY
.
.
.
.
VUVUVUVUVUVUVUVUVUVUVUVUVUVUVU
VUVUVUVUVUVUVUVUVUVUVUVUVUVUVU
.
.
.
.
.
I also had lots of headache because of this preview format.
The best I could find are these:
http://www.fourcc.org/yuv.php#NV21
http://v4l2spec.bytesex.org/spec/r5470.htm
It seems that the Y component is the first width*height bytes int the array you get.
Some more informational links:
http://msdn.microsoft.com/en-us/library/ms867704.aspx#yuvformats_yuvsampling
http://msdn.microsoft.com/en-us/library/ff538197(v=vs.85).aspx
Hope this helps.
The data is in YUV420 format.
If you are only interested in the monochrome channel, i.e. "black and white", then this the first width x height bytes of the data buffer you already have.
The Y channel is the first image plane. It is exactly the grey/intensity/luminosity etc. channel.
Here's code to just extract the greyscale image data:
private int[] decodeGreyscale(byte[] nv21, int width, int height) {
int pixelCount = width * height;
int[] out = new int[pixelCount];
for (int i = 0; i < pixelCount; ++i) {
int luminance = nv21[i] & 0xFF;
out[i] = Color.argb(0xFF, luminance, luminance, luminance);
}
return out;
}
When you only need a grayscale camera preview, you could use a very simple
renderscript:
# pragma version(1)
# pragma rs java_package_name(com.example.name)
# pragma rs_fp_relaxed
rs_allocation gIn; // Allocation filled with camera preview data (byte[])
int previewwidth; // camera preview width (int)
// the parallel executed kernel
void root(uchar4 *v_out, uint32_t x,uint32_t y){
uchar c = rsGetElementAt_uchar(gIn,x+y*previewwidth);
*v_out = (uchar4){c,c,c,255};
}
Note : This is not faster than ScriptIntrinsicYuvToRGB (and a following ScriptIntrinsicColorMatrix to do the RGBA-> gray), but
it runs with API 11+ (where the Intrinsics need Api 17+).
Related
I do not have a background in imaging or graphics, so please bear with me :)
I am using JavaCV in one of my projects. In the examples, a Frame is constructed which has a buffer of a certain size.
When using the public void onPreviewFrame(byte[] data, Camera camera) function in Android, copying this data byte array is no problem if you declare the Frame as new Frame(frameWidth, frameHeight, Frame.DEPTH_UBYTE, 2); where frameWidth and frameHeight are declared as
Camera.Size previewSize = cameraParam.getPreviewSize();
int frameWidth = previewSize.width;
int frameHeight = previewSize.height;
Recently, Android added a method to capture your screen. Naturally, I wanted to grab those images and also covert them to Frames. I modified the example code from Google to use the ImageReader.
This ImageReader is constructed as ImageReader.newInstance(DISPLAY_WIDTH, DISPLAY_HEIGHT, PixelFormat.RGBA_8888, 2);. So currently it uses the RGBA_8888 pixel format. I use the following code to copy the bytes to the Frame, which is instantiated as new Frame(DISPLAY_WIDTH, DISPLAY_HEIGHT, Frame.DEPTH_UBYTE, 2);:
ByteBuffer buffer = mImage.getPlanes()[0].getBuffer();
byte[] bytes = new byte[buffer.remaining()];
buffer.get(bytes);
mImage.close();
((ByteBuffer) frame.image[0].position(0)).put(bytes);
But this gives me a java.nio.BufferOverflowException. I printed the sizes of both buffers and the Frame's buffer size is 691200 whereas the bytes array above is of size 1413056. Figuring out how this latter number is constructed failed because I ran into this native call. So clearly, this won't work out.
After quite a bit of digging I found out that the NV21 image format is "the default format for Camera preview images, when not otherwise set with setPreviewFormat(int)", but the ImageReader class does not support the NV21 format (see the format parameter). So that's tough luck. In the documentation it also reads that
"For the android.hardware.camera2 API, the YUV_420_888 format is recommended for YUV output instead."
So I tried creating an ImageReader like this ImageReader.newInstance(DISPLAY_WIDTH, DISPLAY_HEIGHT, ImageFormat.YUV_420_888, 2);, but this gives me java.lang.UnsupportedOperationException: The producer output buffer format 0x1 doesn't match the ImageReader's configured buffer format 0x23. so that won't work either.
As a last resort, I tried to convert RGBA_8888 to YUV myself using e.g. this post, but I fail to understand how I can obtain an int[] rgba as per the answer.
So, TL;DR how can I obtain NV21 image data like you get in Android's public void onPreviewFrame(byte[] data, Camera camera) camera function to instantiate my Frame and work with it using Android's ImageReader (and Media Projection)?
Edit (25-10-2016)
I have created the following conversion runnable to go from RGBA to NV21 format:
private class updateImage implements Runnable {
private final Image mImage;
public updateImage(Image image) {
mImage = image;
}
#Override
public void run() {
int mWidth = mImage.getWidth();
int mHeight = mImage.getHeight();
// Four bytes per pixel: width * height * 4.
byte[] rgbaBytes = new byte[mWidth * mHeight * 4];
// put the data into the rgbaBytes array.
mImage.getPlanes()[0].getBuffer().get(rgbaBytes);
mImage.close(); // Access to the image is no longer needed, release it.
// Create a yuv byte array: width * height * 1.5 ().
byte[] yuv = new byte[mWidth * mHeight * 3 / 2];
RGBtoNV21(yuv, rgbaBytes, mWidth, mHeight);
((ByteBuffer) yuvImage.image[0].position(0)).put(yuv);
}
void RGBtoNV21(byte[] yuv420sp, byte[] argb, int width, int height) {
final int frameSize = width * height;
int yIndex = 0;
int uvIndex = frameSize;
int A, R, G, B, Y, U, V;
int index = 0;
int rgbIndex = 0;
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
R = argb[rgbIndex++];
G = argb[rgbIndex++];
B = argb[rgbIndex++];
A = argb[rgbIndex++]; // Ignored right now.
// RGB to YUV conversion according to
// https://en.wikipedia.org/wiki/YUV#Y.E2.80.B2UV444_to_RGB888_conversion
Y = ((66 * R + 129 * G + 25 * B + 128) >> 8) + 16;
U = ((-38 * R - 74 * G + 112 * B + 128) >> 8) + 128;
V = ((112 * R - 94 * G - 18 * B + 128) >> 8) + 128;
// NV21 has a plane of Y and interleaved planes of VU each sampled by a factor
// of 2 meaning for every 4 Y pixels there are 1 V and 1 U.
// Note the sampling is every other pixel AND every other scanline.
yuv420sp[yIndex++] = (byte) ((Y < 0) ? 0 : ((Y > 255) ? 255 : Y));
if (i % 2 == 0 && index % 2 == 0) {
yuv420sp[uvIndex++] = (byte) ((V < 0) ? 0 : ((V > 255) ? 255 : V));
yuv420sp[uvIndex++] = (byte) ((U < 0) ? 0 : ((U > 255) ? 255 : U));
}
index++;
}
}
}
}
The yuvImage object is initialized as yuvImage = new Frame(DISPLAY_WIDTH, DISPLAY_HEIGHT, Frame.DEPTH_UBYTE, 2);, the DISPLAY_WIDTH and DISPLAY_HEIGHT are just two integers specifying the display size.
This is the code where a background handler handles the onImageReady:
private final ImageReader.OnImageAvailableListener mOnImageAvailableListener
= new ImageReader.OnImageAvailableListener() {
#Override
public void onImageAvailable(ImageReader reader) {
mBackgroundHandler.post(new updateImage(reader.acquireNextImage()));
}
};
...
mImageReader = ImageReader.newInstance(DISPLAY_WIDTH, DISPLAY_HEIGHT, PixelFormat.RGBA_8888, 2);
mImageReader.setOnImageAvailableListener(mOnImageAvailableListener, mBackgroundHandler);
The methods work and I at least don't get any errors, but the output image is malformed. What is going wrong in my conversion? An example image that is being created:
Edit (15-11-2016)
I have modified the RGBtoNV21 function to be the following:
void RGBtoNV21(byte[] yuv420sp, int width, int height) {
try {
final int frameSize = width * height;
int yIndex = 0;
int uvIndex = frameSize;
int pixelStride = mImage.getPlanes()[0].getPixelStride();
int rowStride = mImage.getPlanes()[0].getRowStride();
int rowPadding = rowStride - pixelStride * width;
ByteBuffer buffer = mImage.getPlanes()[0].getBuffer();
Bitmap bitmap = Bitmap.createBitmap(getResources().getDisplayMetrics(), width, height, Bitmap.Config.ARGB_8888);
int A, R, G, B, Y, U, V;
int offset = 0;
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
// Useful link: https://stackoverflow.com/questions/26673127/android-imagereader-acquirelatestimage-returns-invalid-jpg
R = (buffer.get(offset) & 0xff) << 16; // R
G = (buffer.get(offset + 1) & 0xff) << 8; // G
B = (buffer.get(offset + 2) & 0xff); // B
A = (buffer.get(offset + 3) & 0xff) << 24; // A
offset += pixelStride;
int pixel = 0;
pixel |= R; // R
pixel |= G; // G
pixel |= B; // B
pixel |= A; // A
bitmap.setPixel(j, i, pixel);
// RGB to YUV conversion according to
// https://en.wikipedia.org/wiki/YUV#Y.E2.80.B2UV444_to_RGB888_conversion
// Y = ((66 * R + 129 * G + 25 * B + 128) >> 8) + 16;
// U = ((-38 * R - 74 * G + 112 * B + 128) >> 8) + 128;
// V = ((112 * R - 94 * G - 18 * B + 128) >> 8) + 128;
Y = (int) Math.round(R * .299000 + G * .587000 + B * .114000);
U = (int) Math.round(R * -.168736 + G * -.331264 + B * .500000 + 128);
V = (int) Math.round(R * .500000 + G * -.418688 + B * -.081312 + 128);
// NV21 has a plane of Y and interleaved planes of VU each sampled by a factor
// of 2 meaning for every 4 Y pixels there are 1 V and 1 U.
// Note the sampling is every other pixel AND every other scanline.
yuv420sp[yIndex++] = (byte) ((Y < 0) ? 0 : ((Y > 255) ? 255 : Y));
if (i % 2 == 0 && j % 2 == 0) {
yuv420sp[uvIndex++] = (byte) ((V < 0) ? 0 : ((V > 255) ? 255 : V));
yuv420sp[uvIndex++] = (byte) ((U < 0) ? 0 : ((U > 255) ? 255 : U));
}
}
offset += rowPadding;
}
File file = new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES).getAbsolutePath(), "/Awesomebitmap.png");
FileOutputStream fos = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.PNG, 100, fos);
} catch (Exception e) {
Timber.e(e, "Converting image to NV21 went wrong.");
}
}
Now the image is no longer malformed, but the chroma is off.
The right side is the bitmap that is being created in that loop, the left side is the NV21 saved to an image. So the RGB pixels are processed correctly. Clearly the chroma is off, but the RGB to YUV conversion should be the same one as depicted by wikipedia. What could be wrong here?
Generally speaking, the point of ImageReader is to give you raw access to the pixels sent to the Surface with minimal overhead, so attempting to have it perform color conversions doesn't make sense.
For the Camera you get to pick one of two output formats (NV21 or YV12), so pick YV12. That's your raw YUV data. For screen capture the output will always be RGB, so you need to pick RGBA_8888 (format 0x1) for your ImageReader, rather than YUV_420_888 (format 0x23). If you need YUV for that, you will have to do the conversion yourself. The ImageReader gives you a series of Plane objects, not a byte[], so you will need to adapt to that.
My Problem is: I've set up a camera in Android and receive the preview data by using an onPreviewFrame-listener which passes me an byte[] array containing the image data in the default android YUV-format (device does not support R5G6B5-format). Each pixel consists of 12bits which makes the thing a little tricky. Now what I want to do is converting the YUV-data into ARGB-data in order to do image processing with it. This has to be done with renderscript, in order to maintain a high performance.
My idea was to pass two pixels in one element (which would be 24bits = 3 bytes) and then return two ARGB pixels. The problem is, that in Renderscript a u8_3 (a 3dimensional 8bit vector) is stored in 32bit, which means that the last 8 bits are unused. But when copying the image data into the allocation all of the 32bits are used, so the last 8bit get lost. Even if I used a 32bit input data, the last 8bit are useless, because they're only 2/3 of a pixel. When defining an element consisting a 3-byte-array it actually has a real size of 3 bytes. But then the Allocation.copyFrom()-method doesn't fill the in-Allocation with data, argueing it doesn't has the right data type to be filled with a byte[].
The renderscript documentation states, that there is a ScriptIntrinsicYuvToRGB which should do exactly that in API Level 17. But in fact the class doesn't exist. I've downloaded API Level 17 even though it seems not to be downloadable any more. Does anyone have any information about it? Does anyone have ever tried out a ScriptIntrinsic?
So in conclusion my question is: How to convert the camera data into ARGB data fast, hardwareaccelerated?
That's how to do it in Dalvik VM (found the code somewhere online, it works):
#SuppressWarnings("unused")
private void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {
final int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0)
y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0)
r = 0;
else if (r > 262143)
r = 262143;
if (g < 0)
g = 0;
else if (g > 262143)
g = 262143;
if (b < 0)
b = 0;
else if (b > 262143)
b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
}
}
}
I'm sure you will find the LivePreview test application interesting ... it's part of the Android source code in the latest Jelly Bean (MR1). It implements a camera preview and uses ScriptIntrinsicYuvToRgb to convert the preview data with Renderscript. You can browse the source online here:
LivePreview
I was not able to get running ScriptInstrinsicYuvToRgb, so I decided to write my own RS solution.
Here's ready script (named yuv.rs):
#pragma version(1)
#pragma rs java_package_name(com.package.name)
rs_allocation gIn;
int width;
int height;
int frameSize;
void yuvToRgb(const uchar *v_in, uchar4 *v_out, const void *usrData, uint32_t x, uint32_t y) {
uchar yp = rsGetElementAtYuv_uchar_Y(gIn, x, y) & 0xFF;
int index = frameSize + (x & (~1)) + (( y>>1) * width );
int v = (int)( rsGetElementAt_uchar(gIn, index) & 0xFF ) -128;
int u = (int)( rsGetElementAt_uchar(gIn, index+1) & 0xFF ) -128;
int r = (int) (1.164f * yp + 1.596f * v );
int g = (int) (1.164f * yp - 0.813f * v - 0.391f * u);
int b = (int) (1.164f * yp + 2.018f * u );
r = r>255? 255 : r<0 ? 0 : r;
g = g>255? 255 : g<0 ? 0 : g;
b = b>255? 255 : b<0 ? 0 : b;
uchar4 res4;
res4.r = (uchar)r;
res4.g = (uchar)g;
res4.b = (uchar)b;
res4.a = 0xFF;
*v_out = res4;
}
Don't forget to set camera preview format to NV21:
Parameters cameraParameters = camera.getParameters();
cameraParameters.setPreviewFormat(ImageFormat.NV21);
// Other camera init stuff: preview size, framerate, etc.
camera.setParameters(cameraParameters);
Allocations initialization and script usage:
// Somewhere in initialization section
// w and h are variables for selected camera preview size
rs = RenderScript.create(this);
Type.Builder tbIn = new Type.Builder(rs, Element.U8(rs));
tbIn.setX(w);
tbIn.setY(h);
tbIn.setYuvFormat(ImageFormat.NV21);
Type.Builder tbOut = new Type.Builder(rs, Element.RGBA_8888(rs));
tbOut.setX(w);
tbOut.setY(h);
inData = Allocation.createTyped(rs, tbIn.create(), Allocation.MipmapControl.MIPMAP_NONE, Allocation.USAGE_SCRIPT & Allocation.USAGE_SHARED);
outData = Allocation.createTyped(rs, tbOut.create(), Allocation.MipmapControl.MIPMAP_NONE, Allocation.USAGE_SCRIPT & Allocation.USAGE_SHARED);
outputBitmap = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888);
yuvScript = new ScriptC_yuv(rs);
yuvScript.set_gIn(inData);
yuvScript.set_width(w);
yuvScript.set_height(h);
yuvScript.set_frameSize(previewSize);
//.....
Camera callback method:
public void onPreviewFrame(byte[] data, Camera camera) {
// In your camera callback, data
inData.copyFrom(data);
yuvScript.forEach_yuvToRgb(inData, outData);
outData.copyTo(outputBitmap);
// draw your bitmap where you want to
// .....
}
For anyone who didn't know, RenderScript is now in the Android Support Library, including intrinsics.
http://android-developers.blogspot.com.au/2013/09/renderscript-in-android-support-library.html
http://android-developers.blogspot.com.au/2013/08/renderscript-intrinsics.html
We now have the new renderscript-intrinsics-replacement-toolkit to do it. First, build and import the renderscript module to your project and add it as a dependency to your app module. Then, go to Toolkit.kt and add the following:
fun toNv21(image: Image): ByteArray? {
val nv21 = ByteArray((image.width * image.height * 1.5f).toInt())
return if (!nativeYuv420toNv21(
nativeHandle,
image.width,
image.height,
image.planes[0].buffer, // Y buffer
image.planes[1].buffer, // U buffer
image.planes[2].buffer, // V buffer
image.planes[0].pixelStride, // Y pixel stride
image.planes[1].pixelStride, // U/V pixel stride
image.planes[0].rowStride, // Y row stride
image.planes[1].rowStride, // U/V row stride
nv21
)
) {
null
} else nv21
}
private external fun nativeYuv420toNv21(
nativeHandle: Long,
imageWidth: Int,
imageHeight: Int,
yByteBuffer: ByteBuffer,
uByteBuffer: ByteBuffer,
vByteBuffer: ByteBuffer,
yPixelStride: Int,
uvPixelStride: Int,
yRowStride: Int,
uvRowStride: Int,
nv21Output: ByteArray
): Boolean
Now, go to JniEntryPoints.cpp and add the following:
extern "C" JNIEXPORT jboolean JNICALL Java_com_google_android_renderscript_Toolkit_nativeYuv420toNv21(
JNIEnv *env, jobject/*thiz*/, jlong native_handle,
jint image_width, jint image_height, jobject y_byte_buffer,
jobject u_byte_buffer, jobject v_byte_buffer, jint y_pixel_stride,
jint uv_pixel_stride, jint y_row_stride, jint uv_row_stride,
jbyteArray nv21_array) {
auto y_buffer = static_cast<jbyte*>(env->GetDirectBufferAddress(y_byte_buffer));
auto u_buffer = static_cast<jbyte*>(env->GetDirectBufferAddress(u_byte_buffer));
auto v_buffer = static_cast<jbyte*>(env->GetDirectBufferAddress(v_byte_buffer));
jbyte* nv21 = env->GetByteArrayElements(nv21_array, nullptr);
if (nv21 == nullptr || y_buffer == nullptr || u_buffer == nullptr
|| v_buffer == nullptr) {
// Log this.
return false;
}
RenderScriptToolkit* toolkit = reinterpret_cast<RenderScriptToolkit*>(native_handle);
toolkit->yuv420toNv21(image_width, image_height, y_buffer, u_buffer, v_buffer,
y_pixel_stride, uv_pixel_stride, y_row_stride, uv_row_stride,
nv21);
env->ReleaseByteArrayElements(nv21_array, nv21, 0);
return true;
}
Go to YuvToRgb.cpp and add the following:
void RenderScriptToolkit::yuv420toNv21(int image_width, int image_height, const int8_t* y_buffer,
const int8_t* u_buffer, const int8_t* v_buffer, int y_pixel_stride,
int uv_pixel_stride, int y_row_stride, int uv_row_stride,
int8_t *nv21) {
// Copy Y channel.
for(int y = 0; y < image_height; ++y) {
int destOffset = image_width * y;
int yOffset = y * y_row_stride;
memcpy(nv21 + destOffset, y_buffer + yOffset, image_width);
}
if (v_buffer - u_buffer == sizeof(int8_t)) {
// format = nv21
// TODO: If the format is VUVUVU & pixel stride == 1 we can simply the copy
// with memcpy. In Android Camera2 I have mostly come across UVUVUV packaging
// though.
}
// Copy UV Channel.
int idUV = image_width * image_height;
int uv_width = image_width / 2;
int uv_height = image_height / 2;
for(int y = 0; y < uv_height; ++y) {
int uvOffset = y * uv_row_stride;
for (int x = 0; x < uv_width; ++x) {
int bufferIndex = uvOffset + (x * uv_pixel_stride);
// V channel.
nv21[idUV++] = v_buffer[bufferIndex];
// U channel.
nv21[idUV++] = u_buffer[bufferIndex];
}
}
}
Finally, go to RenderscriptToolkit.h and add the following:
/**
* https://blog.minhazav.dev/how-to-use-renderscript-to-convert-YUV_420_888-yuv-image-to-bitmap/#tobitmapimage-image-method
* #param image_width width of the image you want to convert to byte array
* #param image_height height of the image you want to convert to byte array
* #param y_buffer Y buffer
* #param u_buffer U buffer
* #param v_buffer V buffer
* #param y_pixel_stride Y pixel stride
* #param uv_pixel_stride UV pixel stride
* #param y_row_stride Y row stride
* #param uv_row_stride UV row stride
* #param nv21 the output byte array
*/
void yuv420toNv21(int image_width, int image_height, const int8_t* y_buffer,
const int8_t* u_buffer, const int8_t* v_buffer, int y_pixel_stride,
int uv_pixel_stride, int y_row_stride, int uv_row_stride,
int8_t *nv21);
You are now ready to harness the full power of renderscript. Below, I am providing an example with the ARCore Camera Image object (replace the first line with whatever code gives you your camera image):
val cameraImage = arFrame.frame.acquireCameraImage()
val width = cameraImage.width
val height = cameraImage.height
val byteArray = Toolkit.toNv21(cameraImage)
byteArray?.let {
Toolkit.yuvToRgbBitmap(
byteArray,
width,
height,
YuvFormat.NV21
).let { bitmap ->
saveBitmapToDevice(
name,
session,
bitmap,
context
)}}
I am doing histogram equalization on an image. I first get the RGB image and convert it to YUV. I run the histogram equalization algorithm on Y' of YUV and then convert back to RGB. Is it me, or does the image look weird? I am doing this correctly? this image is pretty bright, other images are a little red.
Here are the before/after images:
The algorithm (the commented values are values that I used previously for conversion. Both yield pretty much the same results) :
public static void createContrast(Bitmap src) {
int width = src.getWidth();
int height = src.getHeight();
Bitmap processedImage = Bitmap.createBitmap(width, height, src.getConfig());
int A = 0,R,G,B;
int pixel;
float[][] Y = new float[width][height];
float[][] U = new float[width][height];
float[][] V = new float [width][height];
int [] histogram = new int[256];
Arrays.fill(histogram, 0);
int [] cdf = new int[256];
Arrays.fill(cdf, 0);
float min = 257;
float max = 0;
for(int x = 0; x < width; ++x) {
for(int y = 0; y < height; ++y) {
pixel = src.getPixel(x, y);
//Log.i("TEST","("+x+","+y+")");
A = Color.alpha(pixel);
R = Color.red(pixel);
G = Color.green(pixel);
B = Color.blue(pixel);
/*Log.i("TESTEST","R: "+R);
Log.i("TESTEST","G: "+G);
Log.i("TESTEST","B: "+B);*/
// convert to YUV
/*Y[x][y] = 0.299f * R + 0.587f * G + 0.114f * B;
U[x][y] = 0.492f * (B-Y[x][y]);
V[x][y] = 0.877f * (R-Y[x][y]);*/
Y[x][y] = 0.299f * R + 0.587f * G + 0.114f * B;
U[x][y] = 0.565f * (B-Y[x][y]);
V[x][y] = 0.713f * (R-Y[x][y]);
// create a histogram
histogram[(int) Y[x][y]]+=1;
// get min and max values
if (Y[x][y] < min){
min = Y[x][y];
}
if (Y[x][y] > max){
max = Y[x][y];
}
}
}
cdf[0] = histogram[0];
for (int i=1;i<=255;i++){
cdf[i] = cdf[i-1] + histogram[i];
//Log.i("TESTEST","cdf of: "+i+" = "+cdf[i]);
}
float minCDF = cdf[(int)min];
float denominator = width*height - minCDF;
//Log.i("TEST","Histeq Histeq Histeq Histeq Histeq Histeq");
for(int x = 0; x < width; ++x) {
for(int y = 0; y < height; ++y) {
//Log.i("TEST","("+x+","+y+")");
pixel = src.getPixel(x, y);
A = Color.alpha(pixel);
Y[x][y] = ((cdf[ (int) Y[x][y]] - minCDF)/(denominator)) * 255;
/*R = minMaxCalc(Y[x][y] + 1.140f * V[x][y]);
G = minMaxCalc (Y[x][y] - 0.395f * U[x][y] - 0.581f * V[x][y]);
B = minMaxCalc (Y[x][y] + 2.032f * U[x][y]);*/
R = minMaxCalc(Y[x][y] + 1.140f * V[x][y]);
G = minMaxCalc (Y[x][y] - 0.344f * U[x][y] - 0.714f * V[x][y]);
B = minMaxCalc (Y[x][y] + 1.77f * U[x][y]);
//Log.i("TESTEST","A: "+A);
/*Log.i("TESTEST","R: "+R);
Log.i("TESTEST","G: "+G);
Log.i("TESTEST","B: "+B);*/
processedImage.setPixel(x, y, Color.argb(A, R, G, B));
}
}
}
My next step is to graph the histograms before and after. I just want to get an opinion here.
The question is a little bit old, but let me answer.
The reason is the way histogram equalization works. The algorithm tries to use all of the 0-255 range instead of given image's range.
So if you give it a dark image, it will change relatively brighter pixels to white colors. And relatively darker colors to black colors.
If you give it a bright image, for the same reason it will get darkened.
I'm looking to change the hue of my background image (PNG) programmatically. How can this be done on Android?
I tested the accepted answer, unfortunately it returns a wrong result. I found and modified this code from here which works fine:
// hue-range: [0, 360] -> Default = 0
public static Bitmap hue(Bitmap bitmap, float hue) {
Bitmap newBitmap = bitmap.copy(bitmap.getConfig(), true);
final int width = newBitmap.getWidth();
final int height = newBitmap.getHeight();
float [] hsv = new float[3];
for(int y = 0; y < height; y++){
for(int x = 0; x < width; x++){
int pixel = newBitmap.getPixel(x,y);
Color.colorToHSV(pixel,hsv);
hsv[0] = hue;
newBitmap.setPixel(x,y,Color.HSVToColor(Color.alpha(pixel),hsv));
}
}
bitmap.recycle();
bitmap = null;
return newBitmap;
}
The linked post has some good ideas, but the matrix math used for ColorFilter may be (a) complex overkill, and (b) introduce perceptible shifts in the resulting colors.
Modifying the solution given by janin here - https://stackoverflow.com/a/6222023/1303595 - I've based this version on Photoshop's 'Color' blend mode. It seems to avoid the image-darkening caused by PorterDuff.Mode.Multiply, and works very well for color-tinting desaturated/artificial-Black & White images without losing much contrast.
/*
* Going for perceptual intent, rather than strict hue-only change.
* This variant based on Photoshop's 'Color' blending mode should look
* better for tinting greyscale images and applying an all-over color
* without tweaking the contrast (much)
* Final color = Target.Hue, Target.Saturation, Source.Luma
* Drawback is that the back-and-forth color conversion introduces some
* error each time.
*/
public void changeHue (Bitmap bitmap, int hue, int width, int height) {
if (bitmap == null) { return; }
if ((hue < 0) || (hue > 360)) { return; }
int size = width * height;
int[] all_pixels = new int [size];
int top = 0;
int left = 0;
int offset = 0;
int stride = width;
bitmap.getPixels (all_pixels, offset, stride, top, left, width, height);
int pixel = 0;
int alpha = 0;
float[] hsv = new float[3];
for (int i=0; i < size; i++) {
pixel = all_pixels [i];
alpha = Color.alpha (pixel);
Color.colorToHSV (pixel, hsv);
// You could specify target color including Saturation for
// more precise results
hsv [0] = hue;
hsv [1] = 1.0f;
all_pixels [i] = Color.HSVToColor (alpha, hsv);
}
bitmap.setPixels (all_pixels, offset, stride, top, left, width, height);
}
If you wrap your Bitmap in an ImageView there is a very simple way:
ImageView circle = new ImageView(this);
circle.setImageBitmap(yourBitmap);
circle.setColorFilter(Color.RED);
My guess is this will be faster than modifying each pixel individually.
I have Black and White picture - RGB 565, 200x50.
as I can calculate the intensity 0..255 of each pixel?
That's what I meant, thanks. ma be this can someone help. I get Intensity 0..255 of each pixel and get the average.
Bitmap cropped = Bitmap.createBitmap(myImage, 503, 270,myImage.getWidth() - 955, myImage.getHeight() - 550);
Bitmap cropped2 = Bitmap.createBitmap(cropped, 0, 0,cropped.getWidth() , cropped.getHeight() / 2 );
final double GS_RED = 0.35;
final double GS_GREEN = 0.55;
final double GS_BLUE = 0.1;
int R, G, B;
int result = 0;
int g = 0;
int ff;
for(int x = 0; x < cropped2.getWidth(); x++)
{
int ff_y = 0;
for(int y = 0; y < cropped2.getHeight(); y++)
{
Pixel = cropped.getPixel(x, y);
R = Color.red(Pixel);
G = Color.green(Pixel);
B = Color.blue(Pixel);
ff = (int)(GS_RED * R + GS_GREEN * G + GS_BLUE * B) ;
ff_y += ff;
}
result += ff_y;
g = result / (cropped2.getWidth()*cropped2.getHeight());
}
Toast.makeText(this, "00" + g, Toast.LENGTH_LONG).show();
You could try to convert it using a color model with a luminance and two chrominance components. The luminance component accounts for the brightness while the two chrominance components represent the colors. You might want to check out http://en.wikipedia.org/wiki/YUV.
Otherwise: If I'm correct, the white over gray to black colors have equal values in a RGB format which has the same number of bits for each channel (e.g. from (0, 0, 0) to (255, 255, 255)). Assuming this is true you could just take one of the channels to represent the intensity as you could determine the other values from that. No guarantee if this works.
Edit:
I wrote a snippet demonstrating the idea described above. I used RGB888 but it should also work with RGB 565 after dropping the assertion and modifying the maximum intensity of a pixel as described in the comments. Mind that there are only 2^5 different intensity levels per pixel. Hence you might want to use a scaled version of the average intensity.
I tested it using images from http://www.smashingmagazine.com/2008/06/09/beautiful-black-and-white-photography/. I hope it will work out porting this to android for you.
// 2^5 for RGB 565
private static final int MAX_INTENSITY = (int) Math.pow(2, 8) - 1;
public static int calculateIntensityAverage(final BufferedImage image) {
long intensitySum = 0;
final int[] colors = image.getRGB(0, 0, image.getWidth(),
image.getHeight(), null, 0, image.getWidth());
for (int i = 0; i < colors.length; i++) {
intensitySum += intensityLevel(colors[i]);
}
final int intensityAverage = (int) (intensitySum / colors.length);
return intensityAverage;
}
public static int intensityLevel(final int color) {
// also see Color#getRed(), #getBlue() and #getGreen()
final int red = (color >> 16) & 0xFF;
final int blue = (color >> 0) & 0xFF;
final int green = (color >> 8) & 0xFF;
assert red == blue && green == blue; // doesn't hold for green in RGB 565
return MAX_INTENSITY - blue;
}