Android YUV to grayscale performance optimization - android

I'm trying to convert an YUV image to grayscale, so basically I just need the Y values.
To do so I wrote this little piece of code (with frame being the YUV image):
imageConversionTime = System.currentTimeMillis();
size = frame.getSize();
byte nv21ByteArray[] = frame.getImage();
int lol;
for (int i = 0; i < size.width; i++) {
for (int j = 0; j < size.height; j++) {
lol = size.width*j + i;
yMatrix.put(j, i, nv21ByteArray[lol]);
}
}
bitmap = Bitmap.createBitmap(size.width, size.height, Bitmap.Config.ARGB_8888);
Utils.matToBitmap(yMatrix, bitmap);
imageConversionTime = System.currentTimeMillis() - imageConversionTime;
However, this takes about 13500 ms. I need it to be A LOT faster (on my computer it takes 8.5 ms in python) (I work on a Motorola Moto E 4G 2nd generation, not super powerful but it should be enough for converting images right?).
Any suggestions?
Thanks in advance.

First of all I would assign size.width and size.height to a variable. I don't think the compiler will optimize this by default, but I am not sure about this.
Furthermore Create a byte[] representing the result instead of using a Matrix.
Then you could do something like this:
int[] grayScalePixels = new int[size.width * size.height];
int cntPixels = 0;
In your inner loop set
grayScalePixels[cntPixels] = nv21ByteArray[lol];
cntPixels++;
To get your final image do the following:
Bitmap grayScaleBitmap = Bitmap.createBitmap(grayScalePixels, size.width, size.height, Bitmap.Config.ARGB_8888);
Hope it works properly (I have not tested it, however at least the shown principle should be applicable -> relying on a byte[] instead of Matrix)

Probably 2 years too late but anyways ;)
To convert to gray scale, all you need to do is set the u/v values to 128 and leave the y values as is. Note that this code is for YUY2 format. You can refer to this document for other formats.
private void convertToBW(byte[] ptrIn, String filePath) {
// change all u and v values to 127 (cause 128 will cause byte overflow)
byte[] ptrOut = Arrays.copyOf(ptrIn, ptrIn.length);
for (int i = 0, ptrInLength = ptrOut.length; i < ptrInLength; i++) {
if (i % 2 != 0) {
ptrOut[i] = (byte) 127;
}
}
convertToJpeg(ptrOut, filePath);
}
For NV21/NV12, I think the loop would change to:
for (int i = ptrOut.length/2, ptrInLength = ptrOut.length; i < ptrInLength; i++) {}
Note: (didn't try this myself)
Also I would suggest to profile your utils method and createBitmap functions separately.

Related

Having problems using setPixel() method

Can someone explain me the way setPixel() method works in Android? I am trying to replace some pixels on a bitmap. I extract them by using getPixel() method and their individual colors, eg.green = Color.green(a[i][j]);, but I cannot set them new values back, so as to show the processed image onscreen.
Edit: This is where some of the processing occurs. I try to algorithmically convert to grayscale
for (int i = 0; i < grayWidth; i++) {
for (int j = 0; j < grayHeight; j++) {
a[i][j] = myImage.getPixel(i, j);
red = Color.red(a[i][j]);
green = Color.green(a[i][j]);
blue = Color.blue(a[i][j]);
gray = (red + green + blue) / 3;
a[i][j] = gray;
}
}
and then replace pixels:
for (int m = 0; m < grayHeight; m++) {
for (int n = 0; n < grayWidth; n++) {
grayScale.setPixel(m, n, a[m][n]);
}
}
and finally show it on-screen
imageView.setImageBitmap(grayScale);
Sorry for not explaining it thoroughly in the first place/
Why posts questions without any code? Put yourself in our position, how can we help you, if we do not know what you are trying to do? You're only presenting your problem in English language, which does not equal to programming language in most cases. We want to see the latter, supported with english description of what you are doing and what's going on.
Based on provided information and Android Dev Page for Bitmap, I can assume that your BitMap image might not be mutable. This would throw then IllegalStateException, but without seeing your LogCat / Code, I cannot be sure, whether this is the case.
If the BitMap indeed is immutable, then you can try and look at converting immutable bitmap to mutable and try again.

Crash Android Program On OpenCV Core.DCT() Method

I want to write an android program to compute dct of the input image, and I am using opencv android framework. first of all I convert input image to grayscale and then i want to compute dct using Core.dct() with 16*16 block. this is the dct computation part :
int rownum = 1;
for (int i = 1; i <= M - B + 1; i++) {
for (int j = 1; j <= N - B + 1; j++) {
Mat SubImg = GrayImage.submat(new Range(i, i+B-1), new Range(j, j+B-1));
Mat Block_DCT = Mat.zeros(16, 16, SubImg.type());
Mat Block_DCT_Quantized = Mat.zeros(16, 16, Block_DCT.type());
Core.dct(SubImg, Block_DCT);
Core.divide(Block_DCT, SQ16, Block_DCT_Quantized);
Mat row = Mat.zeros(1, B*B, Block_DCT_Quantized.type());
Block_DCT_Quantized.reshape(1, B*B);
rownum=rownum+1;
}
}
I debug the code and figure out that the application crash on the line Core.dct() !!!
i don't know what is the problem, but I suppose it's because of input type ...
can any body help me ? what should I do ?
Update :
I found the solution, the problem was because of size and type of Mat objects, actually Core.dct() method just work with this type Mat: cv_64fc1. I use SubImg.convertTo() function to change the type and correct the size of block, so it worked...
thanks to Rui Marques for answering...
dct needs float input.
you have to convert your image to CvType.CV_32F or CvType.CV_64F before applying dct / dft

Dealing with Android's texture size limit

I have a requirement to display somewhat big images on an Android app.
Right now I'm using an ImageView with a source Bitmap.
I understand openGL has a certain device-independent limitation as to
how big the image dimensions can be in order for it to process it.
Is there ANY way to display these images (with fixed width, without cropping) regardless of this limit,
other than splitting the image into multiple ImageView elements?
Thank you.
UPDATE 01 Apr 2013
Still no luck so far all suggestions were to reduce image quality. One suggested it might be possible to bypass this limitation by using the CPU to do the processing instead of using the GPU (though might take more time to process).
I don't understand, is there really no way to display long images with a fixed width without reducing image quality? I bet there is, I'd love it if anyone would at least point me to the right direction.
Thanks everyone.
You can use BitmapRegionDecoder to break apart larger bitmaps (requires API level 10). I've wrote a method that will utilize this class and return a single Drawable that can be placed inside an ImageView:
private static final int MAX_SIZE = 1024;
private Drawable createLargeDrawable(int resId) throws IOException {
InputStream is = getResources().openRawResource(resId);
BitmapRegionDecoder brd = BitmapRegionDecoder.newInstance(is, true);
try {
if (brd.getWidth() <= MAX_SIZE && brd.getHeight() <= MAX_SIZE) {
return new BitmapDrawable(getResources(), is);
}
int rowCount = (int) Math.ceil((float) brd.getHeight() / (float) MAX_SIZE);
int colCount = (int) Math.ceil((float) brd.getWidth() / (float) MAX_SIZE);
BitmapDrawable[] drawables = new BitmapDrawable[rowCount * colCount];
for (int i = 0; i < rowCount; i++) {
int top = MAX_SIZE * i;
int bottom = i == rowCount - 1 ? brd.getHeight() : top + MAX_SIZE;
for (int j = 0; j < colCount; j++) {
int left = MAX_SIZE * j;
int right = j == colCount - 1 ? brd.getWidth() : left + MAX_SIZE;
Bitmap b = brd.decodeRegion(new Rect(left, top, right, bottom), null);
BitmapDrawable bd = new BitmapDrawable(getResources(), b);
bd.setGravity(Gravity.TOP | Gravity.LEFT);
drawables[i * colCount + j] = bd;
}
}
LayerDrawable ld = new LayerDrawable(drawables);
for (int i = 0; i < rowCount; i++) {
for (int j = 0; j < colCount; j++) {
ld.setLayerInset(i * colCount + j, MAX_SIZE * j, MAX_SIZE * i, 0, 0);
}
}
return ld;
}
finally {
brd.recycle();
}
}
The method will check to see if the drawable resource is smaller than MAX_SIZE (1024) in both axes. If it is, it just returns the drawable. If it's not, it will break the image apart and decode chunks of the image and place them in a LayerDrawable.
I chose 1024 because I believe most available phones will support images at least that large. If you want to find the actual texture size limit for a phone, you have to do some funky stuff through OpenGL, and it's not something I wanted to dive into.
I wasn't sure how you were accessing your images, so I assumed they were in your drawable folder. If that's not the case, it should be fairly easy to refactor the method to take in whatever parameter you need.
You can use BitmapFactoryOptions to reduce size of picture.You can use somthing like that :
BitmapFactory.Options options = new BitmapFactory.Options();
options.inSampleSize = 3; //reduce size 3 times
Have you seen how your maps working? I had made a renderer for maps once. You can use same trick to display your image.
Divide your image into square tiles (e.g. of 128x128 pixels). Create custom imageView supporting rendering from tiles. Your imageView knows which part of bitmap it should show now and displays only required tiles loading them from your sd card. Using such tile map you can display endless images.
It would help if you gave us the dimensions of your bitmap.
Please understand that OpenGL runs against natural mathematical limits.
For instance, there is a very good reason a texture in OpenGL must be 2 to the power of x. This is really the only way the math of any downscaling can be done cleanly without any remainder.
So if you give us the exact dimensions of the smallest bitmap that's giving you trouble, some of us may be able to tell you what kind of actual limit you're running up against.

Creating a 1bpp (bit per pixel) Bitmap in Android

Ok so been racking my brain on this one all day. Trying to figure out how I can convert a Bitmap from canvas to a 1bpp (bit per pixel) Bitmap file in Android and physically save it as such.
So far I've iterated through the bitmap and created an int[] of the resulting pixel values as 1s or 0s. However, my next question is what do I do with that?
What I tried to do was something like
int[] bits = // populated earlier
byte[] bmp = new byte[bits.length / 8];
int byteindex = 0;
int bitindex = 0;
for (int i=0; i<bits.length; i++) {
if (bits[i] == 1)
// set to 1
else
// set to 0
if (bitindex++ == 8) {
bitindex = 0;
byteindex++;
}
}
OutputStream out = new FileOutputStream("/mnt/sdcard/dynbmp.bmp");
out.write(bmp);
out.close();
I get a file out of it but it's obviously not a valid bmp file. Who knows what it is. You'll have to forgive me for my lack of bit-byte and imaging knowledge, but where am I screwing up? Do I the idea completely wrong? Am I missing some header info or something?
Yes, you are missing several things. It's a little bit more complicated... Look here:
http://en.wikipedia.org/wiki/BMP_file_format

ALPHA_8 bitmaps and getPixel

I am trying to load a movement map from a PNG image. In order to save memory
after I load the bitmap I do something like that.
`Bitmap mapBmp = tempBmp.copy(Bitmap.Config.ALPHA_8, false);`
If I draw the mapBmp I can see the map but when I use getPixel() I get
always 0 (zero).
Is there a way to retrieve ALPHA information from a bitmap other than
with getPixel() ?
Seems to be an Android bug in handling ALPHA_8. I also tried copyPixelsToBuffer, to no avail. Simplest workaround is to waste lots of memory and use ARGB_8888.
Issue 25690
I found this question from Google and I was able to extract the pixels using the copyPixelsToBuffer() method that Mitrescu Catalin ended up using. This is what my code looks like in case anyone else finds this as well:
public byte[] getPixels(Bitmap b) {
int bytes = b.getRowBytes() * b.getHeight();
ByteBuffer buffer = ByteBuffer.allocate(bytes);
b.copyPixelsToBuffer(buffer);
return buffer.array();
}
If you are coding for API level 12 or higher you could use getByteCount() instead to get the total number of bytes to allocate. However if you are coding for API level 19 (KitKat) you should probably use getAllocationByteCount() instead.
I was able to find a nice and sort of clean way to create boundary maps. I create an ALPHA_8 bitmap from the start. I paint my boundry map with paths. Then I use the copyPixelsToBuffer() and transfer the bytes into a ByteBuffer. I use the buffer to "getPixels" from.
I think is a good solution since you can scale down or up the path() and draw the boundary map at the desired screen resolution scale and no IO + decode operations.
Bitmap.getPixel() is useless for ALPHA_8 bitmaps, it always returns 0.
I developed solution with PNGJ library, to read image from assets and then create Bitmap with Config.ALPHA_8.
import ar.com.hjg.pngj.IImageLine;
import ar.com.hjg.pngj.ImageLineHelper;
import ar.com.hjg.pngj.PngReader;
public Bitmap getAlpha8BitmapFromAssets(String file) {
Bitmap result = null;
try {
PngReader pngr = new PngReader(getAssets().open(file));
int channels = pngr.imgInfo.channels;
if (channels < 3 || pngr.imgInfo.bitDepth != 8)
throw new RuntimeException("This method is for RGB8/RGBA8 images");
int bytes = pngr.imgInfo.cols * pngr.imgInfo.rows;
ByteBuffer buffer = ByteBuffer.allocate(bytes);
for (int row = 0; row < pngr.imgInfo.rows; row++) {
IImageLine l1 = pngr.readRow();
for (int j = 0; j < pngr.imgInfo.cols; j++) {
int original_color = ImageLineHelper.getPixelARGB8(l1, j);
byte x = (byte) Color.alpha(original_color);
buffer.put(row * pngr.imgInfo.cols + j, x ^= 0xff);
}
}
pngr.end();
result = Bitmap.createBitmap(pngr.imgInfo.cols,pngr.imgInfo.rows, Bitmap.Config.ALPHA_8);
result.copyPixelsFromBuffer(buffer);
} catch (IOException e) {
Log.e(LOG_TAG, e.getMessage());
}
return result;
}
I also invert alpha values, because of my particular needs. This code is only tested for API 21.

Categories

Resources