Develop Reference

Renderscript allocation from bitmap

I'm trying to get into render script and was confused regarding allocations usage. Almost all examples show next algorithm:
Create In and Out Bitmap
Create In and Out Allocation from Bitmaps In and Out correspondingly
Configure script and perform forEach method
copy result from Out allocation into bitmap using copyTo method
Something like that:
Bitmap srcBitmap = BitmapFactory.decodeResource(getResources(), R.drawable.lena);
Bitmap dstBitmap = Bitmap.createBitmap(srcBitmap.getWidth(), srcBitmap.getHeight(), srcBitmap.getConfig());
Allocation allocationIn = Allocation.createFromBitmap(renderScript, srcBitmap);
Allocation allocationOut = Allocation.createFromBitmap(renderScript, dstBitmap);
scriptColorMatrix.setGreyscale();
scriptColorMatrix.forEach(allocationIn, allocationOut);
//no difference after removing this line
allocationOut.copyTo(dstBitmap);
imagePreview.setImageBitmap(dstBitmap);
This works, but it also works even if I omit step 4 by removing:
allocationOut.copyTo(dstBitmap);
Lets go further and lower brightness after grayscale:
Bitmap srcBitmap = BitmapFactory.decodeResource(getResources(), R.drawable.lena);
Bitmap dstBitmap = Bitmap.createBitmap(srcBitmap.getWidth(), srcBitmap.getHeight(), srcBitmap.getConfig());
Allocation allocationIn = Allocation.createFromBitmap(renderScript, srcBitmap);
Allocation allocationOut = Allocation.createFromBitmap(renderScript, dstBitmap);
scriptColorMatrix.setGreyscale();
scriptColorMatrix.forEach(allocationIn, allocationOut);
//reset color matrix
scriptColorMatrix.setColorMatrix(new Matrix4f());
//adjust brightness
scriptColorMatrix.setAdd(-0.5f, -0.5f, -0.5f, 0f);
//Performing forEach vise versa (from out to in)
scriptColorMatrix.forEach(allocationOut, allocationIn);
imagePreview.setImageBitmap(srcBitmap);
Shortly describing the code above, we performed grayscale collor matrix from In allocation into Out one, and brightness adjustment in backward direction. I never called copyTo method, but at the end I've got result in srcBitmap and it was correct.
That's not the end. Lets go deeper. I'll leave only one bitmap and one Allocation:
Bitmap srcBitmap = BitmapFactory.decodeResource(getResources(), R.drawable.lena);
Allocation allocationIn = Allocation.createFromBitmap(renderScript, srcBitmap);
scriptColorMatrix.setGreyscale();
scriptColorMatrix.forEach(allocationIn, allocationIn);
//reset color matrix
scriptColorMatrix.setColorMatrix(new Matrix4f());
//adjust brightness
scriptColorMatrix.setAdd(-0.5f, -0.5f, -0.5f, 0f);
scriptColorMatrix.forEach(allocationIn, allocationIn);
imagePreview.setImageBitmap(srcBitmap);
The result is the same...
Can anybody explain why does it happen and where to use copyTo and where I can use targeting Bitmap without it?

The Allocation objects are needed in order to provide the proper mapping of Bitmap to something Renderscript understands. If you are targeting API 18 or higher, the Allocation.createFromBitmap() methods you are using are automatically giving in the flag USAGE_SHARED, which tries to have the Allocation use the same backing memory as the Bitmap object. So the two are linked but technically the copyTo() method is still needed as the RS implementation may need to synchronize it back. On some platforms this may have already happened where others may cause this to pause as DMA or other mechanisms are updating the Bitmap's backing memory with whatever changes were made by the RS code.
As for why you can reverse the in/out Allocation order when calling scripts - it's valid and up to you to get the arguments and order correct. To RS they are just objects which point to some type of backing data to be manipulated. Since both were created with the Allocation.createFromBitmap() call, they can be used as either input or output as long as the Bitmap objects are mutable.
Similarly, using the same Allocation for the input and output is not normal, but also not invalid. It just means your input is changing on the fly. As long as your script is not accessing other Elements in the data when the root function is called for a specific Element, then it should work (as you are seeing.)

Has anyone managed to obtain a YUV_420_888 frame using RenderScript and the new Camera API?

I'm using RenderScript and Allocation to obtain YUV_420_888 frames from the Android Camera2 API, but once I copy the byte[] from the Allocation I receive only the Y plane from the 3 planes which compose the frame, while the U and V planes values are set to 0 in the byte[]. I'm trying to mimic the onPreviewframe from the previos camera API in order to perform in app processing of the camera frames. My Allocation is created like:
Type.Builder yuvTypeBuilderIn = new Type.Builder(rs, Element.YUV(rs));
yuvTypeBuilderIn.setX(dimensions.getWidth());
yuvTypeBuilderIn.setY(dimensions.getHeight());
yuvTypeBuilderIn.setYuvFormat(ImageFormat.YUV_420_888);
allocation = Allocation.createTyped(rs, yuvTypeBuilderIn.create(),
Allocation.USAGE_IO_INPUT | Allocation.USAGE_SCRIPT);
while my script looks like:
#pragma version(1)
#pragma rs java_package_name(my_package)
#pragma rs_fp_relaxed
rs_allocation my_frame;

The Android sample app HdrViewfinderDemo uses RenderScript to process YUV data from camera2.
https://github.com/googlesamples/android-HdrViewfinder
Specifically, the ViewfinderProcessor sets up the Allocations, and hdr_merge.rs reads from them.

Yes I did it, since I couldn't find anything useful. But I didn't go the proposed way of defining an allocation to the surface. Instead I just converted the output of the three image planes to RGB. The reason for this approach is that I use the YUV420_888 data twofold. First on a high frequency basis just the intensity values (Y). Second, I need to make some color Bitmaps too. Thus, the following solution. The script takes about 80ms for a 1280x720 YUV_420_888 image, maybe not ultra fast, but ok for my purpose.
UPDATE: I deleted the code here, since I wrote a more general solution here YUV_420_888 -> Bitmap conversion that takes into account pixelStride and rowStride too.

I think that you can use an ImageReader to get the frames of you camera into YUV_420_888
reader = ImageReader.newInstance(previewSize.getWidth(), previewSize.getHeight(), ImageFormat.YUV_420_888, 2);
Then you set an OnImageAvailableListener to the reader :
reader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
#Override
public void onImageAvailable(ImageReader reader) {
int jump = 4; //Le nombre d'image à sauter avant d'en traiter une, pour liberer de la mémoire
Image readImage = reader.acquireNextImage();
readImage.getPlane[0] // The Y plane
readImage.getPlane[1] //The U plane
readImage.getPlane[2] //The V plane
readImage.close();
}
}, null);
Hope that will help you

I'm using almost the same method as widea in their answer.
The exception you keep getting after ~50 frames might be due to the fact that you're processing all the frames by using acquireNextImage. The documentation suggest to:
Warning: Consider using acquireLatestImage() instead, as it will automatically release older images, and allow slower-running processing routines to catch up to the newest frame. [..]
So in case your exception is a IllegalStateException, switching to acquireLatestImage might help.
And make sure you call close() on all images retrieved from ImageReader.

Android Blur with RenderScript

I try to make a gaussian blur on an android Bitmap but I get this error:
rsAssert failed: !mTypes.size() and
rsAssert failed: !mElements.size()
Here is my code :
public Bitmap blurBitmap(Bitmap src) {
Bitmap outBitmap = src.copy(src.getConfig(), true);
final RenderScript rs = RenderScript.create(this);
final Allocation input = Allocation.createFromBitmap(rs, src);
final Allocation output = Allocation.createFromBitmap(rs, outBitmap);
final ScriptIntrinsicBlur script =
ScriptIntrinsicBlur.create(rs, Element.U8_4(rs));
script.setRadius(25f);
script.setInput(input);
script.forEach(output);
output.copyTo(outBitmap);
rs.destroy();
return outBitmap;
}
Note that I used android.support.v8.renderscript to ensure compatibility with android lower versions.
Someone would have an idea to fix it?
Thanks.
Martin

The Element arguments of ScriptIntrinsicBlur should be the same of Allocation's element, so you should use Allocation.getElement(), rather than direct Element.U8_4(rs).
final ScriptIntrinsicBlur script =
ScriptIntrinsicBlur.create(rs, input.getElement());
And you might also want move those final to class private member, since some of them might be different every time your bitmap is different.
And FYI, script.setRadius(25f) is way too high, that will cause slow calculation. If you need such heavy blur, you might consider to scale down the original bitmap at certain level, and blur it, then scale up to your canvas, which will be much faster than heavy blur for a huge image.
One more thing, if you don't care to keep the original bitmap, the allocation of input and output can be the same one, which might save some memory.

While there isn't a good way to increase blur radius, for very large images you can essentially fake it by scaling down (ex .5 original size), blurring, then scaling up to original size. Be careful not to scale too far down as the resulting bitmap will become pixelated.

How to get Android Render Script Group work?

I can get two separate intrinsics working, but not together in a ScriptGroup. I found document on how to use Script Group is extremely sparse.
Here is my code:
mRS = RenderScript.create(getActivity());
mInAllocation = Allocation.createFromBitmap(mRS, mBitmapIn,
Allocation.MipmapControl.MIPMAP_NONE,
Allocation.USAGE_SCRIPT |
Allocation.USAGE_GRAPHICS_TEXTURE |
Allocation.USAGE_SHARED);
mOutAllocation = Allocation.createFromBitmap(mRS, mBitmapOut,
Allocation.MipmapControl.MIPMAP_NONE,
Allocation.USAGE_SCRIPT |
Allocation.USAGE_SHARED);
ScriptIntrinsicColorMatrix gray = ScriptIntrinsicColorMatrix.create(mRS, Element.U8_4(mRS));
gray.setGreyscale();
ScriptIntrinsicBlur blur = ScriptIntrinsicBlur.create(mRS, Element.U8_4(mRS));
blur.setRadius(20.f);
blur.setInput(mInAllocation);
//gray.forEach(mInAllocation, mOutAllocation);
blur.forEach(mOutAllocation);
mOutAllocation.copyTo(mBitmapOut);
Both gray and blur work. And then I tried putting them together, the result is blank. Code:
// gray.forEach(mInAllocation, mOutAllocation);
// blur.forEach(mOutAllocation);
// mOutAllocation.copyTo(mBitmapOut);
ScriptGroup.Builder builder = new ScriptGroup.Builder(mRS);
builder.addKernel(gray.getKernelID());
builder.addKernel(blur.getKernelID());
builder.addConnection(mInAllocation.getType(), gray.getKernelID(), blur.getKernelID());
ScriptGroup group = builder.create();
group.setInput(gray.getKernelID(), mInAllocation);
group.setOutput(blur.getKernelID(), mOutAllocation);
group.execute();
mOutAllocation.copyTo(mBitmapOut);

I was able to reproduce the issue you are seeing and cross checked with notes from my earlier experiments with intrinsics. I think there are a few bugs in renderscript intrinsics code.
-1-
If you want to get a scripgroup working with intrinsics, the below sequence works.
mBlur.setInput(mInAllocation);
sBuilder = new ScriptGroup.Builder(mRS);
sBuilder.addKernel(mBlur.getKernelID());
sBuilder.addKernel(mColor.getKernelID());
sBuilder.addConnection(connect, mBlur.getKernelID(), mColor.getKernelID());
sGroup = sBuilder.create();
// sGroup.setInput(mBlur.getKernelID(), mInAllocation); //See point 2
sGroup.setOutput(mColor.getKernelID(), mOutAllocation);
sGroup.execute();
mOutAllocation.copyTo(outBitmap);
mRS.finish();
-2-
Note the way how the input allocation is passed. The input allocation is passed to mBlur.setInput() and not to sGroup.setInput(). If sGroup.setInput() is used, then the group correctly does not find the input and it results in the following error and ofcourse, I dont see the transformed image on the screen as well.
E/RenderScript(12023): rsAssert failed: !"ScriptGroup:setInput kid not found", in frameworks/rs/rsScriptGroup.cpp at 267
In this specific example from -1-, I was getting the following errors as well the moment sGroup.setInput() is used instead of mBlur.setInput()
E/RenderScript(12023): Blur executed without input, skipping
This appears to be bug(s) in renderscript
-3-
Specifically, in your case, where you want to do ScriptIntrinsicColorMatrix with ScriptIntrinsicBlur in a sequence, there is another issue (not necessarily a bug). While Blur intrinsic has a setInput function, colorMatrix does have a setInput function. So you cannot use -1- as a workaround as well.
-4-
I think the right fixes in renderscript would to be to deprecate
intrinsic.setInput universally just as it was done for ScriptIntrinsicColorMatrix and get ScriptGroup.setInput working while using intrinsics in a script group.
-5-
I did not see any issues using scriptgroup, when I have my own kernels. In other words, scriptGroup.setInput() and scriptGroup.setOutput() work perfectly fine

Resize bitmap using Renderscript

I'm currently working on a project where I have to use RenderScript, so i started learning about it, and it's a great technology, because, just like openGL, it lets you use computational code that goes to a native level, and doesn't have to use the dalvik vm. This part of the code, being processed much faster than if you would use normal android code.
I started working with image processing and what i was wondering is:
Is it possible to resize a bitmap using RenderScript? this should be much faster then resizing an bitmap using android code. Plus, renderscript can process information that is bigger than 48mB (limit on some phones for each process).

While you could use Rendscript to do the bitmap resize, I'm not sure if that's the best choice. A quick look at the Android code base shows that Java API does go into native code to do a bitmap resize, although if the resize algorithm isn't to your needs, you'll have to implement your own.
There are a number of answers on SO for getting the bitmap scaled efficiently. My recommendation is to try those, and if they still aren't doing what your want, either as quickly or how the results appear visually to then investigate into writing your own. If you still want to write your own, do use the performance tools available to see if you really are faster or just reinventing the wheel.

You can use the below function to resize the image.
private Bitmap resize(Bitmap inBmp) {
RenderScript mRs = RenderScript.create(getApplication());
Bitmap outBmp = Bitmap.createBitmap(OUTPUT_IMAGE_WIDTH, inBmp.getHeight() * OUTPUT_IMAGE_WIDTH /inBmp.getWidth(), inBmp.getConfig());
ScriptIntrinsicResize siResize = ScriptIntrinsicResize.create(mRs);
Allocation inAlloc = Allocation.createFromBitmap(mRs, inBmp);
Allocation outAlloc = Allocation.createFromBitmap(mRs, outBmp);
siResize.setInput(inAlloc);
siResize.forEach_bicubic(outAlloc);
outAlloc.copyTo(outBmp);
inAlloc.destroy();
outAlloc.destroy();
siResize.destroy();
return outBmp;
}
OUTPUT_IMAGE is the integer value specifying the width of the output image.
NOTE: While using the RenderScript Allocation you have to be very careful as they lead to memory leakages.