Develop Reference

Avoid rotating bitmap twice from Android camera

At the moment I have the code:
onPreviewFrame(byte[] data)
int[] rgbs = new int[480*800];
decodeYUV(rgbs, data, 480, 800);
Bitmap bitmap = Bitmap.createBitmap(rgbs, 800, 480, Bitmap.Config.ARGB_8888);
Matrix matrix = new Matrix();
matrix.postRotate(90);
Bitmap scaledBitmap = Bitmap.createScaledBitmap(bitmap,800,480,true);
Bitmap rotatedBitmap = Bitmap.createBitmap(scaledBitmap , 0, 0, scaledBitmap .getWidth(), scaledBitmap .getHeight(), matrix, true);
sbut.setImageBitmap(rotatedBitmap);
where the decodeYUV function is found here
The preview is set up like this:
param.setPreviewSize(800, 480);
camera.setDisplayOrientation(90);
So the preview is set in portrait mode, Height = 800, Width = 480.
I end up having to go back to landscape to perform the conversion. And then rotate back to portrait again. I can imagine this is quite slow. Is there a more efficient alternative without a double rotation?
I would like to keep the preview in portrait mode. My end result should be that rotatedBitmap above, which is just portrait again. Any lines in the onPreviewFrame method can be changed.

In your case, I would 'freeze' the Activity in Potrait mode and use Gravity Sensor to check the rotations without Activity or Camera recreation.
to freeze use this:
this.setRequestedOrientation(
ActivityInfo.SCREEN_ORIENTATION_PORTRAIT);
to rotate I use this code:
public static void rotateNV21(byte[] input, byte[] output, int width, int height, int rotation) {
try{
boolean swap = (rotation == 90 || rotation == 270);
boolean yflip = (rotation == 90 || rotation == 180);
boolean xflip = (rotation == 270 || rotation == 180);
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
int xo = x, yo = y;
int w = width, h = height;
int xi = xo, yi = yo;
if (swap) {
xi = w * yo / h;
yi = h * xo / w;
}
if (yflip) {
yi = h - yi - 1;
}
if (xflip) {
xi = w - xi - 1;
}
output[w * yo + xo] = input[w * yi + xi];
int fs = w * h;
int qs = (fs >> 2);
xi = (xi >> 1);
yi = (yi >> 1);
xo = (xo >> 1);
yo = (yo >> 1);
w = (w >> 1);
h = (h >> 1);
// adjust for interleave here
int ui = fs + (w * yi + xi) * 2;
int uo = fs + (w * yo + xo) * 2;
// and here
int vi = ui + 1;
int vo = uo + 1;
output[uo] = input[ui];
output[vo] = input[vi];
}
}
}catch (IndexOutOfBoundsException e){
output = input;
}
}

Rotate YUV420/NV21 Image in android

In PreviewCall back of surface we are getting YUV420SP format in camera Preview but due to wrong rotation of that image I want to perform correct rotation of YUV image as I need to send it through network.so correct rotation need to be applied.
I found this link it does correct rotation but image loose the color.
http://www.wordsaretoys.com/2013/10/25/roll-that-camera-zombie-rotation-and-coversion-from-yv12-to-yuv420planar/
also checked Rotate an YUV byte array on Android but it does not show image properly.
I do have checked links on stckoverflow but none of them have satisfactory answer about correctly using the code in android environment.
do any one have idea how to correctly rotate NV21 Image bytes[] with retaining its color information correctly.

If you just want to rotate NV21, following code will be helpful. (I modified the code from here)
public static void rotateNV21(byte[] input, byte[] output, int width, int height, int rotation) {
boolean swap = (rotation == 90 || rotation == 270);
boolean yflip = (rotation == 90 || rotation == 180);
boolean xflip = (rotation == 270 || rotation == 180);
for (int x = 0; x < width; x++) {
for (int y = 0; y < height; y++) {
int xo = x, yo = y;
int w = width, h = height;
int xi = xo, yi = yo;
if (swap) {
xi = w * yo / h;
yi = h * xo / w;
}
if (yflip) {
yi = h - yi - 1;
}
if (xflip) {
xi = w - xi - 1;
}
output[w * yo + xo] = input[w * yi + xi];
int fs = w * h;
int qs = (fs >> 2);
xi = (xi >> 1);
yi = (yi >> 1);
xo = (xo >> 1);
yo = (yo >> 1);
w = (w >> 1);
h = (h >> 1);
// adjust for interleave here
int ui = fs + (w * yi + xi) * 2;
int uo = fs + (w * yo + xo) * 2;
// and here
int vi = ui + 1;
int vo = uo + 1;
output[uo] = input[ui];
output[vo] = input[vi];
}
}
}

Eddy Yong`s code works well with 180 degrees only, but gained some upvotes in here and a lot of copy-pastes in other SO topics and other similar (e.g. github discussions), so I've decided to leave here proper answer for future researchers, and rotation method, which works for 90, 180 and 270 deegres
public static byte[] rotateNV21(final byte[] yuv,
final int width,
final int height,
final int rotation)
{
if (rotation == 0) return yuv;
if (rotation % 90 != 0 || rotation < 0 || rotation > 270) {
throw new IllegalArgumentException("0 <= rotation < 360, rotation % 90 == 0");
}
final byte[] output = new byte[yuv.length];
final int frameSize = width * height;
final boolean swap = rotation % 180 != 0;
final boolean xflip = rotation % 270 != 0;
final boolean yflip = rotation >= 180;
for (int j = 0; j < height; j++) {
for (int i = 0; i < width; i++) {
final int yIn = j * width + i;
final int uIn = frameSize + (j >> 1) * width + (i & ~1);
final int vIn = uIn + 1;
final int wOut = swap ? height : width;
final int hOut = swap ? width : height;
final int iSwapped = swap ? j : i;
final int jSwapped = swap ? i : j;
final int iOut = xflip ? wOut - iSwapped - 1 : iSwapped;
final int jOut = yflip ? hOut - jSwapped - 1 : jSwapped;
final int yOut = jOut * wOut + iOut;
final int uOut = frameSize + (jOut >> 1) * wOut + (iOut & ~1);
final int vOut = uOut + 1;
output[yOut] = (byte)(0xff & yuv[yIn]);
output[uOut] = (byte)(0xff & yuv[uIn]);
output[vOut] = (byte)(0xff & yuv[vIn]);
}
}
return output;
}
originally posted by jake in this SO topic

Android: Adaptive Thresholding

I'm trying to implement adaptive thresholding algorithm by Derek Bradley using Android. But it is returning black pixels all the time. Here is my code snippet. Please suggest me about what should I do. Thanks in advance.
public static Bitmap GrayscaleToBin(Bitmap bm2)
{
Bitmap bm;
bm=bm2.copy(Config.ARGB_8888, true);
final int width = bm.getWidth();
final int height = bm.getHeight();
int[] pixels;
pixels = new int[width*height];
bm.getPixels(pixels,0,width,0,0,width,height);
//Bradley AdaptiveThrsholdging
int []intImg= new int[width*height];
int sum=0;
for(int i=0;i<width;++i){
sum=0;
for(int j=0;j<height;++j)
{
sum=sum+pixels[i+j*width];
if(i==0){intImg[i+j*width]=sum;}
else
{
intImg[i+j*width]= intImg[i-1+j*width]+sum;
}
}
}
int x1,x2,y1,y2=0,count=0;
int s=width >> 3;
int t=15;
for(int i=0;i<width;++i)
{
for(int j=0;j<height;++j)
{
x1=i-s/2;
x2=i+s/2;
y1=j-s/2;
y2=j+s/2;
if (x1 <0) x1 = 0;
if (x2>= width) x2 = width-1;
if (y1 <0) y1 = 0;
if (y2>= height) y2 = height-1;
count = (x2-x1) * (y2-y1);
sum = intImg [y2 * width + x2] -
intImg [y1 * width + x2] -
intImg [y2 * width + x1] +
intImg [y1 * width + x1];
if((pixels[i+j*width]*count)<=(sum*(100-t)/100))
{
pixels[i+j*width]=0;
}
else
{
pixels[i+j*width]=255;
}
}
}
/*---------------------------------------------------------------------------*/
bm.setPixels(pixels,0,width,0,0,width,height);
// Log.d("cdsfss","afterloop");
return bm;
}

After a Long struggle I have solved the issue with the following code.
public static Bitmap GrayscaleToBin(Bitmap bm2)
{
Bitmap bm;
bm=bm2.copy(Config.RGB_565, true);
final int width = bm.getWidth();
final int height = bm.getHeight();
int pixel1,pixel2,pixel3,pixel4,A,R;
int[] pixels;
pixels = new int[width*height];
bm.getPixels(pixels,0,width,0,0,width,height);
int size=width*height;
int s=width/8;
int s2=s>>1;
double t=0.15;
double it=1.0-t;
int []integral= new int[size];
int []threshold=new int[size];
int i,j,diff,x1,y1,x2,y2,ind1,ind2,ind3;
int sum=0;
int ind=0;
while(ind<size)
{
sum+=pixels[ind] & 0xFF;
integral[ind]=sum;
ind+=width;
}
x1=0;
for(i=1;i<width;++i)
{
sum=0;
ind=i;
ind3=ind-s2;
if(i>s)
{
x1=i-s;
}
diff=i-x1;
for(j=0;j<height;++j)
{
sum+=pixels[ind] & 0xFF;
integral[ind]=integral[(int)(ind-1)]+sum;
ind+=width;
if(i<s2)continue;
if(j<s2)continue;
y1=(j<s ? 0 : j-s);
ind1=y1*width;
ind2=j*width;
if (((pixels[ind3]&0xFF)*(diff * (j - y1))) < ((integral[(int)(ind2 + i)] - integral[(int)(ind1 + i)] - integral[(int)(ind2 + x1)] + integral[(int)(ind1 + x1)])*it)) {
threshold[ind3] = 0x00;
} else {
threshold[ind3] = 0xFFFFFF;
}
ind3 += width;
}
}
y1 = 0;
for( j = 0; j < height; ++j )
{
i = 0;
y2 =height- 1;
if( j <height- s2 )
{
i = width - s2;
y2 = j + s2;
}
ind = j * width + i;
if( j > s2 ) y1 = j - s2;
ind1 = y1 * width;
ind2 = y2 * width;
diff = y2 - y1;
for( ; i < width; ++i, ++ind )
{
x1 = ( i < s2 ? 0 : i - s2);
x2 = i + s2;
// check the border
if (x2 >= width) x2 = width - 1;
if (((pixels[ind]&0xFF)*((x2 - x1) * diff)) < ((integral[(int)(ind2 + x2)] - integral[(int)(ind1 + x2)] - integral[(int)(ind2 + x1)] + integral[(int)(ind1 + x1)])*it)) {
threshold[ind] = 0x00;
} else {
threshold[ind] = 0xFFFFFF;
}
}
}
/*-------------------------------
* --------------------------------------------*/
bm.setPixels(threshold,0,width,0,0,width,height);
return bm;
}

You can use Catalano Framework. There's an example using Bradley for Android in samples folder.
FastBitmap fb = new FastBitmap(bitmap);
fb.toGrayscale();
BradleyLocalThreshold bradley = new BradleyLocalThreshold();
bradley.applyInPlace(fb);
bitmap = fb.toBitmap();

Android JavaCV create IplImage from Camera to use with ColorHistogram

I am using JavaCV in Android.
In my code, I have created a ImageComparator(class of OpenCV CookBook
http://code.google.com/p/javacv/source/browse/OpenCV2_Cookbook/src/opencv2_cookbook/chapter04/ImageComparator.scala?repo=examples
http://code.google.com/p/javacv/wiki/OpenCV2_Cookbook_Examples_Chapter_4) Object and use that object to compare images. If I use file from SD card the comparator is working.
File referenceImageFile = new File(absPath1); // Read an image.
IplImage reference = Util.loadOrExit(referenceImageFile,CV_LOAD_IMAGE_COLOR);
comparator = new ImageComparator(reference);
comparator = new ImageComparator(reference);
But from Camera Preview, when I am creating IplImage it is not working. I am getting the following Exception during comparison "score" calculation.
score = referenceComparator.compare(grayImage) / imageSize;
java.lang.RuntimeException: /home/saudet/android/OpenCV-2.4.2/modules/core/src/convert.cpp:1196: error: (-215) i < src.channels() in function void cvSplit(const void*, void*, void*, void*, void*)
For CameraPreview I am using the code from FacePreview to create IplImage.But it create Image in grayScale.
int f = SUBSAMPLING_FACTOR;
if (grayImage == null || grayImage.width() != width / f
|| grayImage.height() != height / f) {
grayImage = IplImage.create(width / f, height / f, IPL_DEPTH_8U, 1);
}
int imageWidth = grayImage.width();
int imageHeight = grayImage.height();
int dataStride = f * width;
int imageStride = grayImage.widthStep();
ByteBuffer imageBuffer = grayImage.getByteBuffer();
for (int y = 0; y < imageHeight; y++) {
int dataLine = y * dataStride;
int imageLine = y * imageStride;
for (int x = 0; x < imageWidth; x++) {
imageBuffer.put(imageLine + x, data[dataLine + f * x]);
}
}
How to create a Color IplImage from Camera to use with ImageComparator?

The below code seems to be working fine.
public void onPreviewFrame(final byte[] data, final Camera camera) {
try {
Camera.Size size = camera.getParameters().getPreviewSize();
processImage(data, size.width, size.height);
camera.addCallbackBuffer(data);
} catch (RuntimeException e) {
// The camera has probably just been released, ignore.
Log.d("Exception", " " + e);
}
}
protected void processImage(byte[] data, int width, int height) {
score.clear();
// First, downsample our image
int f = SUBSAMPLING_FACTOR;
IplImage _4image = IplImage.create(width, height, IPL_DEPTH_8U, f);
int[] _temp = new int[width * height];
if (_4image != null) {
decodeYUV420SP(_temp, data, width, height);
_4image.getIntBuffer().put(_temp);
}
//bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
//bitmap.copyPixelsFromBuffer(_4image.getByteBuffer());
Log.d("CompareAndroid", "processImage");
int imageSize = _4image.width() * _4image.height();
Iterator<ImageComparator> iterator = reference_List.iterator();
// Compute histogram match and normalize by image size.
// 1 means perfect match.
while(iterator.hasNext()){
score.add(((ImageComparator) iterator.next()).compare(_4image) / imageSize);
}
Log.d("CompareImages", "Score Size "+score.size());
postInvalidate();
}
This code seems to be working fine.
private void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width,
int height) {
int frameSize = width * height;
for (int j = 0, yp = 0; j < height; j++) {
int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
for (int i = 0; i < width; i++, yp++) {
int y = (0xff & ((int) yuv420sp[yp])) - 16;
if (y < 0)
y = 0;
if ((i & 1) == 0) {
v = (0xff & yuv420sp[uvp++]) - 128;
u = (0xff & yuv420sp[uvp++]) - 128;
}
int y1192 = 1192 * y;
int r = (y1192 + 1634 * v);
int g = (y1192 - 833 * v - 400 * u);
int b = (y1192 + 2066 * u);
if (r < 0)
r = 0;
else if (r > 262143)
r = 262143;
if (g < 0)
g = 0;
else if (g > 262143)
g = 262143;
if (b < 0)
b = 0;
else if (b > 262143)
b = 262143;
rgb[yp] = 0xff000000 | ((b << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((r >> 10) & 0xff);
}
}
}

I haven't tested it, but something like this should work:
IplImage yuvimage = IplImage.create(width, height * 3 / 2, IPL_DEPTH_8U, 2);
IplImage rgbimage = IplImage.create(width, height, IPL_DEPTH_8U, 3);
cvCvtColor(yuvimage, rgbimage, CV_YUV2BGR_NV21);

How to save video file using canvas on android?

Currently I'm working on video effects like thermal, mono, etc.. for that I'm using preview callback with canvas stuff.
Can anyone please tell me how to save this video?
Below I have my callback code:
mCamera.setPreviewCallback(new PreviewCallback() {
public void onPreviewFrame(byte[] data1, Camera _camera)
{
Camera.Parameters parameters = _camera.getParameters();
parameters.setColorEffect(Camera.Parameters.EFFECT_NONE);
previewSize = parameters.getPreviewSize();
int frameSize = previewSize.width * previewSize.height;
int height = previewSize.height;
int width = previewSize.width;
int pixel;
int A, R, G, B;
final double GS_RED = 0.299;
final double GS_GREEN = 0.587;
final double GS_BLUE = 0.114;
int[] rgba = new int[frameSize+1];
data = data1;
for (int i = 0; i < height; i++)
for(int j = 0; j < width; j++)
{
int y = (0xff & ((int) data[i * previewSize.width + j]));
int u = (0xff & ((int) data[frameSize + (i >> 1) * previewSize.width + (j & ~1) + 0]));
int v = (0xff & ((int) data[frameSize + (i >> 1) * previewSize.width + (j & ~1) + 1]));
y = y < 16 ? 16 : y;
int r = Math.round(1.164f * (y - 16) + 1.596f * (v - 128));
int g = Math.round(1.164f * (y - 16) - 0.813f * (v - 128) - 0.391f * (u - 128));
int b = Math.round(1.164f * (y - 16) + 2.018f * (u - 128));
r = r < 0 ? 0 : (r > 255 ? 255 : r);
g = g < 0 ? 0 : (g > 255 ? 255 : g);
b = b < 0 ? 0 : (b > 255 ? 255 : b);
rgba[i * previewSize.width + j] = 0xff000000 + (b << 16) + (g<< 8) + r;
}
Bitmap bmp = Bitmap.createBitmap(width, height,
Bitmap.Config.RGB_565);
bmp.setPixels(rgba, 0 , width , 0, 0, width, height);
// scan through every single pixel
for(int x = 0; x < width; ++x) {
for(int y = 0; y < height; ++y) {
// get one pixel color
pixel = bmp.getPixel(x, y);
// retrieve color of all channels
A = Color.alpha(pixel);
R = Color.red(pixel);
G = Color.green(pixel);
B = Color.blue(pixel);
// take conversion up to one single value
R = G = B = (int)(GS_RED * R + GS_GREEN * G + GS_BLUE * B);
// set new pixel color to output bitmap
bmp.setPixel(x, y, Color.argb(A, R, G, B));
}
}
canvas = mHolder.lockCanvas();
if (canvas != null)
{
canvas.drawBitmap(bmp, (canvas.getWidth() - width) / 4, (canvas.getHeight() - height) / 4, null);
mHolder.unlockCanvasAndPost(canvas);
}
bmp.recycle();
}
});

You can save each frames and then can merge them with ffmpeg to encode , to create a video file.