Develop Reference

How do I write rgb2yuv based on the existing yuv2rgb method？

I took over an Android project, which already has a yuv2rgb method. I have to write this method rgb2yuv. Please help me, thank you!
public static int[] yuv2rgb(byte[] pYUV, int width, int height) {
int[] pRGB = new int[width * height];
int i, j, yp;
int hfWidth = width >> 1;
int size = width * height;
int qtrSize = size >> 2;
for (i = 0, yp = 0; i < height; i++) {
int uvp = size + (i >> 1) * hfWidth, u = 0, v = 0;
for (j = 0; j < width; j++, yp++) {
int y = (0xff & pYUV[yp]) - 16;
if ((j & 1) == 0) {
u = (0xff & pYUV[uvp + (j >> 1)]) - 128;
v = (0xff & pYUV[uvp + qtrSize + (j >> 1)]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0;
else if (r > 262143) r = 262143;
if (g < 0) g = 0;
else if (g > 262143) g = 262143;
if (b < 0) b = 0;
else if (b > 262143) b = 262143;
pRGB[i * width + j] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
}
}
return pRGB;
}

My colleague helped me find a way，it's work！
public static byte[] colorconvertRGB_IYUV_I420(int[] aRGB, int width, int height) {
final int frameSize = width * height;
final int chromasize = frameSize / 4;
int yIndex = 0;
int uIndex = frameSize;
int vIndex = frameSize + chromasize;
byte[] yuv = new byte[width * height * 3 / 2];
int a, R, G, B, Y, U, V;
int index = 0;
for (int j = 0; j < height; j++) {
for (int i = 0; i < width; i++) {
//a = (aRGB[index] & 0xff000000) >> 24; //not using it right now
R = (aRGB[index] & 0xff0000) >> 16;
G = (aRGB[index] & 0xff00) >> 8;
B = (aRGB[index] & 0xff) >> 0;
Y = ((66 * R + 129 * G + 25 * B + 128) >> 8) + 16;
U = ((-38 * R - 74 * G + 112 * B + 128) >> 8) + 128;
V = ((112 * R - 94 * G - 18 * B + 128) >> 8) + 128;
yuv[yIndex++] = (byte) ((Y < 0) ? 0 : ((Y > 255) ? 255 : Y));
if (j % 2 == 0 && index % 2 == 0) {
yuv[uIndex++] = (byte) ((U < 0) ? 0 : ((U > 255) ? 255 : U));
yuv[vIndex++] = (byte) ((V < 0) ? 0 : ((V > 255) ? 255 : V));
}
index++;
}
}
return yuv;
}

Convert YV12 (yuv420p) to JPEG / Bitamp

In onPreviewFrame using android.hardware.Camera, if I use the default NV21 format, I can use YuvImage to compress to jpeg format, which works great. If I try to change the format using setPreviewFormat(ImageFormat.YV12), then it does not work anymore as YuvImage does not support YV12 format. I've found only one solution somewhere to convert Bitmap to YV12, but I want to do the opposite and get a jpeg out of these bytes. Is there a library to do this?

If YUV420 to JPEG is what you're looking for then,
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
// TODO Auto-generated method stub
ByteArrayOutputStream out= new ByteArrayOutputStream();
decodeYUV420(pixels, data, previewSize.width, previewSize.height);
mBitmap = Bitmap.createBitmap(pixels, previewSize.width, previewSize.height,Config.ARGB_8888);
mBitmap.compress(CompressFormat.JPEG , 25, out);
.......
where, the decodeYUV420 method goes as follows:
public void decodeYUV420(int[] rgb, byte[] yuv420, int width, int height) {
final int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420[yp])) - 16;
if (y < 0) y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420[uvp++]) - 128;
u = (0xff & yuv420[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0; else if (r > 262143) r = 262143;
if (g < 0) g = 0; else if (g > 262143) g = 262143;
if (b < 0) b = 0; else if (b > 262143) b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
}
}
I've created a similar project to this on github check here
and the code implementation: here
and yes it works!

How to detect average pixel intensity from live camera preview?

I am building a scanner app, and trying to determine the "preview quality" from the preview callback of the camera. I want to customize the camera's AUTO_FLASH_MODE where it will be turned on if the environment is too dark.
How can I detect if there is a high average of dark pixels? This means (in preview) I am getting darkness and therefore need to turn on the camera's flash light.

Either find out how to access pixel values of your image and calculate the average intensity yourself or use any image processing library to do so.
Dark pixels have low values, bright pixels have high values.
You want to calculate the average of all red, green and blue values divided by three times your pixel count.
Define a threshold for when to turn on the flash, but keep in mind that you have to get a new exposure time then.
Prefer flash over exposure time increase as long exposure times yield higher image noise.

I tried this approach but i think it is taking unnecessary time of processing the bitmap and then get an average screen color,
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
Size cameraResolution = resolution;
PreviewCallback callback = this.callback;
if (cameraResolution != null && callback != null)
{
int format = camera.getParameters().getPreviewFormat();
SourceData source = new SourceData(data, cameraResolution.width, cameraResolution.height, format, getCameraRotation());
callback.onPreview(source);
final int[] rgb = decodeYUV420SP(data, cameraResolution.width, cameraResolution.height);
//Bitmap bmp = decodeBitmap(source.getData());
Bitmap bmp = Bitmap.createBitmap(rgb, cameraResolution.width, cameraResolution.height, Bitmap.Config.ARGB_8888);
if (bmp != null)
{
//bmp = decodeBitmap(source.getData());
ByteArrayOutputStream bytes = new ByteArrayOutputStream();
// bmp.compress(Bitmap.CompressFormat.JPEG, 70, bytes);
Bitmap resizebitmap = Bitmap.createBitmap(bmp,
bmp.getWidth() / 2, bmp.getHeight() / 2, 60, 60);
int color = getAverageColor(resizebitmap);
Log.i("Color Int", color + "");
// int color = resizebitmap.getPixel(resizebitmap.getWidth()/2,resizebitmap.getHeight()/2);
String strColor = String.format("#%06X", 0xFFFFFF & color);
//String colorname = sColorNameMap.get(strColor);
Log.d("strColor", strColor);
Log.i("strColor", color + "");
if(!mIsOn)
{
if (color == -16777216 || color < -16777216)//minimum color code (full dark)
{
mIsOn = true;
setTorch(true);
Log.d("Yahooooo", "" + color);
}
}
Log.i("Pixel Value",
"Top Left pixel: " + Integer.toHexString(color));
}
}
else
{
Log.d(TAG, "Got preview callback, but no handler or resolution available");
}
}
}
private int[] decodeYUV420SP(byte[] yuv420sp, int width, int height)
{
final int frameSize = width * height;
int rgb[]=new int[width*height];
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0) y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0) r = 0; else if (r > 262143) r = 262143;
if (g < 0) g = 0; else if (g > 262143) g = 262143;
if (b < 0) b = 0; else if (b > 262143) b = 262143;
rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) &
0xff00) | ((b >> 10) & 0xff);
}
}
return rgb;
}
private int getAverageColor(Bitmap bitmap)
{
int redBucket = 0;
int greenBucket = 0;
int blueBucket = 0;
int pixelCount = 0;
for (int y = 0; y < bitmap.getHeight(); y++) {
for (int x = 0; x < bitmap.getWidth(); x++) {
int c = bitmap.getPixel(x, y);
pixelCount++;
redBucket += Color.red(c);
greenBucket += Color.green(c);
blueBucket += Color.blue(c);
// does alpha matter?
}
}
int averageColor = Color.rgb(redBucket / pixelCount, greenBucket
/ pixelCount, blueBucket / pixelCount);
return averageColor;
}

how to optimize convolution matrix in android

I am using Convolution Matrix for my android app for making image Emboss.
i have defined the class for it as:
public class ConvolutionMatrix {
public static final int SIZE = 3;
public double[][] Matrix;
public double Factor = 1;
public double Offset = 1;
public ConvolutionMatrix(int size) {
Matrix = new double[size][size];
}
public void setAll(double value) {
for (int x = 0; x < SIZE; ++x) {
for (int y = 0; y < SIZE; ++y) {
Matrix[x][y] = value;
}
}
}
public void applyConfig(double[][] config) {
for (int x = 0; x < SIZE; ++x) {
for (int y = 0; y < SIZE; ++y) {
Matrix[x][y] = config[x][y];
}
}
}
public static Bitmap computeConvolution3x3(Bitmap src,
ConvolutionMatrix matrix) {
int width = src.getWidth();
int height = src.getHeight();
Bitmap result = Bitmap.createBitmap(width, height, src.getConfig());
int A, R, G, B;
int sumR, sumG, sumB;
int[][] pixels = new int[SIZE][SIZE];
for (int y = 0; y < height - 2; ++y) {
for (int x = 0; x < width - 2; ++x) {
// get pixel matrix
for (int i = 0; i < SIZE; ++i) {
for (int j = 0; j < SIZE; ++j) {
pixels[i][j] = src.getPixel(x + i, y + j);
}
}
// get alpha of center pixel
A = Color.alpha(pixels[1][1]);
// init color sum
sumR = sumG = sumB = 0;
// get sum of RGB on matrix
for (int i = 0; i < SIZE; ++i) {
for (int j = 0; j < SIZE; ++j) {
sumR += (Color.red(pixels[i][j]) * matrix.Matrix[i][j]);
sumG += (Color.green(pixels[i][j]) * matrix.Matrix[i][j]);
sumB += (Color.blue(pixels[i][j]) * matrix.Matrix[i][j]);
}
}
// get final Red
R = (int) (sumR / matrix.Factor + matrix.Offset);
if (R < 0) {
R = 0;
} else if (R > 255) {
R = 255;
}
// get final Green
G = (int) (sumG / matrix.Factor + matrix.Offset);
if (G < 0) {
G = 0;
} else if (G > 255) {
G = 255;
}
// get final Blue
B = (int) (sumB / matrix.Factor + matrix.Offset);
if (B < 0) {
B = 0;
} else if (B > 255) {
B = 255;
}
// apply new pixel
result.setPixel(x + 1, y + 1, Color.argb(A, R, G, B));
}
}
// final image
return result;
}
}
It is giving me proper result but it takes too much time for calculating the result. Is there any way to make calculation faster and work efficiently?

The core of your slowdown is:
// apply new pixel
result.setPixel(x + 1, y + 1, Color.argb(A, R, G, B));
That's a reasonable amount of work each iteration to set each pixel pixel by pixel, they aren't free in the bitmap class. It's far better to call the getPixels() routine and mess with the raw pixels there then put them back, just one time when you're done.
You could also hardcode the emboss (most of the time you're grabbing a bunch of data and multiplying it by zero with that kernel, you easily cheat and grab like the three pixels you care about.
private static int hardEmboss(int[] pixels, int stride, int index, int[][] matrix, int parts) {
//ignoring the matrix
int p1 = pixels[index];
int p2 = pixels[index + stride + 1];
int p3 = pixels[index + stride + stride + 2];
int r = 2 * ((p1 >> 16) & 0xFF) - ((p2 >> 16) & 0xFF) - ((p3 >> 16) & 0xFF);
int g = 2 * ((p1 >> 8) & 0xFF) - ((p2 >> 8) & 0xFF) - ((p3 >> 8) & 0xFF);
int b = 2 * ((p1) & 0xFF) - ((p2) & 0xFF) - ((p3) & 0xFF);
return 0xFF000000 | ((crimp(r) << 16) | (crimp(g) << 8) | (crimp(b)));
}
Assuming your emboss kernel is:
int[][] matrix = new int[][]{
{2, 0, 0},
{0, -1, 0},
{0, 0, -1}
};
Also, unbeknownst to most everybody there's a critical flaw in the standard convolution algorithm, where the return the results pixel to the center is in error. If you return it to the upper left hand corner you can simply process all the data in the same memory footprint going left to right, top to bottom in a scanline operation.
public static int crimp(int v) { return (v > 255)?255:((v < 0)?0:v); }
public static void applyEmboss(int[] pixels, int stride) {
//stride should be equal to width here, and pixels.length == bitmap.height * bitmap.width;
int pos;
pos = 0;
try {
while (true) {
int p1 = pixels[pos];
int p2 = pixels[pos + stride + 1];
int p3 = pixels[pos + stride + stride + 2];
int r = 2 * ((p1 >> 16) & 0xFF) - ((p2 >> 16) & 0xFF) - ((p3 >> 16) & 0xFF);
int g = 2 * ((p1 >> 8) & 0xFF) - ((p2 >> 8) & 0xFF) - ((p3 >> 8) & 0xFF);
int b = 2 * ((p1) & 0xFF) - ((p2) & 0xFF) - ((p3) & 0xFF);
pixels[pos++] = 0xFF000000 | ((crimp(r) << 16) | (crimp(g) << 8) | (crimp(b)));
}
}
catch (ArrayIndexOutOfBoundsException e) { }
}
The disadvantage is that the pixel appears to shift left and up by 1 pixel. Though, if you do another scanline fill backwards you could shift them back. And all the garbage here will end up as 2 rows on the right and bottom sides (some of this will be filled with embossed nonsense because I didn't have it slow down to check for those places). This means that if you want to cut that off when you readd the pixels reduce the height and width by 2, and leave the stride at the size of the original width. Since all the good data will be in the top bit, you don't have to fiddle at all with the offset.
Also, just use renderscript.

Take a look at: Convolution Demo. It is an App, which compares a convolution implementation done in Java vs in C++.
Needless to say C++ variant runs more than 10x faster.
So if you want speed either implement it via NDK or via Shaders.

How to save video file using canvas on android?

Currently I'm working on video effects like thermal, mono, etc.. for that I'm using preview callback with canvas stuff.
Can anyone please tell me how to save this video?
Below I have my callback code:
mCamera.setPreviewCallback(new PreviewCallback() {
public void onPreviewFrame(byte[] data1, Camera _camera)
{
Camera.Parameters parameters = _camera.getParameters();
parameters.setColorEffect(Camera.Parameters.EFFECT_NONE);
previewSize = parameters.getPreviewSize();
int frameSize = previewSize.width * previewSize.height;
int height = previewSize.height;
int width = previewSize.width;
int pixel;
int A, R, G, B;
final double GS_RED = 0.299;
final double GS_GREEN = 0.587;
final double GS_BLUE = 0.114;
int[] rgba = new int[frameSize+1];
data = data1;
for (int i = 0; i < height; i++)
for(int j = 0; j < width; j++)
{
int y = (0xff & ((int) data[i * previewSize.width + j]));
int u = (0xff & ((int) data[frameSize + (i >> 1) * previewSize.width + (j & ~1) + 0]));
int v = (0xff & ((int) data[frameSize + (i >> 1) * previewSize.width + (j & ~1) + 1]));
y = y < 16 ? 16 : y;
int r = Math.round(1.164f * (y - 16) + 1.596f * (v - 128));
int g = Math.round(1.164f * (y - 16) - 0.813f * (v - 128) - 0.391f * (u - 128));
int b = Math.round(1.164f * (y - 16) + 2.018f * (u - 128));
r = r < 0 ? 0 : (r > 255 ? 255 : r);
g = g < 0 ? 0 : (g > 255 ? 255 : g);
b = b < 0 ? 0 : (b > 255 ? 255 : b);
rgba[i * previewSize.width + j] = 0xff000000 + (b << 16) + (g<< 8) + r;
}
Bitmap bmp = Bitmap.createBitmap(width, height,
Bitmap.Config.RGB_565);
bmp.setPixels(rgba, 0 , width , 0, 0, width, height);
// scan through every single pixel
for(int x = 0; x < width; ++x) {
for(int y = 0; y < height; ++y) {
// get one pixel color
pixel = bmp.getPixel(x, y);
// retrieve color of all channels
A = Color.alpha(pixel);
R = Color.red(pixel);
G = Color.green(pixel);
B = Color.blue(pixel);
// take conversion up to one single value
R = G = B = (int)(GS_RED * R + GS_GREEN * G + GS_BLUE * B);
// set new pixel color to output bitmap
bmp.setPixel(x, y, Color.argb(A, R, G, B));
}
}
canvas = mHolder.lockCanvas();
if (canvas != null)
{
canvas.drawBitmap(bmp, (canvas.getWidth() - width) / 4, (canvas.getHeight() - height) / 4, null);
mHolder.unlockCanvasAndPost(canvas);
}
bmp.recycle();
}
});

You can save each frames and then can merge them with ffmpeg to encode , to create a video file.