I am looking for a fast way to rotate a Bitmap 180 degrees using OpenCV on Android. Also, the Bitmap may be rotated in place, i.e. without allocation of additional memory.
Here riwnodennyk described different methods of rotating Bitmaps and compared their performances: https://stackoverflow.com/a/29734593/1707617. Here is GitHub repository for this study: https://github.com/riwnodennyk/ImageRotation.
This study doesn't include OpenCV implementation. So, I tried my own implementation of OpenCV rotator (only 180 degrees).
The best rotation methods for my test picture are as follows:
OpenCV: 13 ms
Ndk: 15 ms
Usual: 29 ms
Because OpenCV performance looks very promising and my implementation seems to be unoptimized, I decided to ask you how to implement it better.
My implementation with comments:
#Override
public Bitmap rotate(Bitmap srcBitmap, int angleCcw) {
Mat srcMat = new Mat();
Utils.bitmapToMat(srcBitmap, srcMat); // Possible memory allocation and copying
Mat rotatedMat = new Mat();
Core.flip(srcMat, rotatedMat, -1); // Possible memory allocation
Bitmap dstBitmap = Bitmap.createBitmap(srcBitmap.getWidth(), srcBitmap.getHeight(), Bitmap.Config.ARGB_8888); // Unneeded memory allocation
Utils.matToBitmap(rotatedMat, dstBitmap); // Unneeded copying
return dstBitmap;
}
There are 3 places where, I suppose, unnecessary allocations and copying may take place.
Is it possible to get rid of theese unnecessary operations?
I implemented fast bitmap rotation library.
Performance (approximately):
5 times faster than Matrix rotation method.
3.8 times faster than Ndk rotation method.
3.3 times faster than OpenCV rotation method.
https://github.com/ivankrylatskoe/fast-bitmap-rotation-lib
Related
I am currently working on a project in which one I would like to rotate a bitmap.
The first time, I create my bitmap with the following code :
myBitmap = BitmapFactory.decodeResource(getResources(), drawableResource);
Then I rotate the bitmap using the following code :
final Matrix matrix = new Matrix();
matrix.postRotate(currentRotate);
myBitmap = Bitmap.createBitmap(myBitmap, 0, 0, directionBitmap.getWidth(), directionBitmap.getHeight(), matrix, true);
I works, but after several times, the memory increases and I have the following exception :
java.lang.OutOfMemoryError: Failed to allocate a 119071756 byte
allocation with 16775968 free bytes and 96MB until OOM
It seems that the old bitmaps are still in memory. How to delete/recycle them in order to save the memory ?
Thank your for your help.
I can suggest you to use Glide library from Github.
This library works in background threads. Anyway you can perform your rotation on Background like this:
runOnUiThread(new Runnable(final Matrix matrix = new Matrix();
matrix.postRotate(currentRotate);
myBitmap = Bitmap.createBitmap(myBitmap, 0, 0, directionBitmap.getWidth(), directionBitmap.getHeight(), matrix, true);
)
In my case, the problem was with the size of the bitmaps I was using. The bitmaps I was using were of very high pixels for the given device. In such case, Android system has to descale them to lower pixel density i.e. the one that will suit your device. When you rotate the bitmap, Android system takes up a lot of memory to descale it to lower pixels. Also, Android gets busy and in some cases results in UI thread blocking.
Also, increase the heap size in your manifest file.
I have some code which is loading an image into an OpenGL texture. In the process, I end up loading 3 bitmaps, since I need to load the original bitmap (sized appropriately for the display) and reorient the bitmap based on EXIF data. I'm very quickly calling .recycle() on each bitmap, but I'm noticing that my memory doesn't seem to change.
Here's what the Memory Monitor shows:
As you can see, after loading the image I'm using about 60MB of memory. When I rotate the device that drops off a bit then comes back up. That leads me to think there is no leak, since the memory never goes above that.
When I click the GC button in the memory analyzer, my memory footprint drops dramatically to around 8 MB. This makes sense as the three bitmaps created during the process were recycled, so can be garbage collected. Then you can see that when I rotate again and the activity is rebuilt, the memory jumps right back up.
Here's my code to show you why so many bitmaps are created and when they're recycled.
void layoutImage() {
...
Bitmap bitmap = loadOrientedConstrainedBitmapWithBackouts(...);
imageTexture = new GLTexture(bitmap);
bitmap.recycle(); // recycle bitmap 2
}
Bitmap loadOrientedConstrainedBitmapWithBackouts(Context context, Uri uri, int maxSize) {
...
Bitmap bitmap = loadBitmapWithBackouts(context, uri, sampleSize); // create bitmap 1
...
Bitmap out = orientBitmap(bitmap, orientation); // create bitmap 2
bitmap.recycle(); // recycle bitmap 1
return out;
}
Bitmap orientBitmap(Bitmap source, int orientation) {
...
return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight, matrix, true); // create bitmap 3
}
I'm not really sure that this is a problem, so to speak, since the memory isn't climbing (so no leak), but I'm curious when it stays so high. Since forcing a garbage collection clears it just fine, should I assume that if the system needs that memory it will be collected on the next GC pass? It's been running the whole time I've been writing this and is still sitting comfortably at 60 MB.
Question 1: Can I just trust that the garbage collector will take that memory back if needed?
Also, if we're supposed to be so judiciously recycling our bitmaps, why do so many of the Bitmap methods say things like "The new bitmap may be the same object as source, or a copy may have been made." Do I really have to check the equality every time I use those methods to recycle the bitmap if it's a different object?
Question 2: When using Bitmap creation methods, that may or may not return the same bitmap or a copy, do I need to check source and output equality to recycle the source if it's a copy?
Edit:
I have tried analyzing this with MAT, using a heap dump at peak usage (should be 60 MB), but it only reports 18.2 MB of usage and nothing unusual looking. Could they be reading things differently?
Question 1: Can I just trust that the garbage collector will take that memory back if needed?
Yes. If the incoming references are cleared, the garbage collector will take the memory when it is needed (typically for a new allocation). Calling recycle() doesn't help this process along or make it happen any faster.
The recycle() method exists because Bitmap objects were not counted against the heap until Android 3.0; so the method was helpful to assist the GC since it didn't otherwise have a record of that memory counted against its heap. In 3.0+, the memory is tracked against the heap so this extra bookkeeping isn't necessary anymore.
Question 2: When using Bitmap creation methods, that may or may not return the same bitmap or a copy, do I need to check source and output equality to recycle the source if it's a copy?
The createBitmap() method will return the same object if:
The source is immutable
x and y are both zero
width and height match the source width and height
No transformation matrices have been applied
Since it looks like you are passing in a transformation matrix, you will always get a copy unless the matrix is identity for some reason. But again, no real need to recycle() unless you are still supporting 2.x versions.
I'm capturing an image from the camera which I want to pass through some processing in OpenCV. On older devices, this is failing at the first hurdle though:
public void onPictureTaken(byte[] jpeg, Camera c) {
mImageBitmap = BitmapFactory.decodeByteArray(jpeg, 0, jpeg.length);
org.opencv.android.Utils.bitmapToMat(mImageBitmap, Utils.cameraMat);
...
}
Produces:
04-22 14:23:41.708: I/System.out(7289): Available memory (bytes): 5646168
04-22 14:23:41.718: I/dalvikvm-heap(7289): Forcing collection of SoftReferences for 51121168-byte allocation
04-22 14:23:41.758: E/dalvikvm-heap(7289): Out of memory on a 51121168-byte allocation.
Reading up about this, the advice seems to be to work with a smaller version of the image, but how do I work out roughly what dimensions to resize to given the amount of available memory left on the device?
You need to resize your image before passing it to openCV. The decode instruction consumes a lot of memory because it loads the full image in memory. Even if you don't display it.
This official tutorial will show you how to resize your image without having to load it in memory first
You can find out how much memory your application has available to use, total, with getMemoryClass. You'll have to work out how much of that you're eating up with your application's other data, and the decoded bitmap above.
Some rough estimates:
The S II has an 8 MP camera. Assuming an RGB24 Bitmap, a single decoded bitmap will take up 8 * 3 = 24 MB of memory. The OpenCV matrix will probably take up about the same, possible more depending on what its internal format is (the out-of-memory error suggests it's trying to allocate ~48 MB, which is about twice the expectation here).
That's easily above what a standard app can use for memory - the minimum baseline is only 16 MB. If you need more memory, you probably want to look into using a large heap instead of the normal-sized one. That will let you get roughly double the memory, although it's still device-dependent.
I'm currently working on a project where I have to use RenderScript, so i started learning about it, and it's a great technology, because, just like openGL, it lets you use computational code that goes to a native level, and doesn't have to use the dalvik vm. This part of the code, being processed much faster than if you would use normal android code.
I started working with image processing and what i was wondering is:
Is it possible to resize a bitmap using RenderScript? this should be much faster then resizing an bitmap using android code. Plus, renderscript can process information that is bigger than 48mB (limit on some phones for each process).
While you could use Rendscript to do the bitmap resize, I'm not sure if that's the best choice. A quick look at the Android code base shows that Java API does go into native code to do a bitmap resize, although if the resize algorithm isn't to your needs, you'll have to implement your own.
There are a number of answers on SO for getting the bitmap scaled efficiently. My recommendation is to try those, and if they still aren't doing what your want, either as quickly or how the results appear visually to then investigate into writing your own. If you still want to write your own, do use the performance tools available to see if you really are faster or just reinventing the wheel.
You can use the below function to resize the image.
private Bitmap resize(Bitmap inBmp) {
RenderScript mRs = RenderScript.create(getApplication());
Bitmap outBmp = Bitmap.createBitmap(OUTPUT_IMAGE_WIDTH, inBmp.getHeight() * OUTPUT_IMAGE_WIDTH /inBmp.getWidth(), inBmp.getConfig());
ScriptIntrinsicResize siResize = ScriptIntrinsicResize.create(mRs);
Allocation inAlloc = Allocation.createFromBitmap(mRs, inBmp);
Allocation outAlloc = Allocation.createFromBitmap(mRs, outBmp);
siResize.setInput(inAlloc);
siResize.forEach_bicubic(outAlloc);
outAlloc.copyTo(outBmp);
inAlloc.destroy();
outAlloc.destroy();
siResize.destroy();
return outBmp;
}
OUTPUT_IMAGE is the integer value specifying the width of the output image.
NOTE: While using the RenderScript Allocation you have to be very careful as they lead to memory leakages.
My game uses surfaceview(I know I should use GL).
I draw alot of bitmaps to my game character world controll so on. And i run into this when I opened my LogDog:
08-05 10:17:29.151: ERROR/dalvikvm(24048): Out of memory: Heap Size=5379KB, Allocated=2735KB, Bitmap Size=20576KB, Limit=32768KB
I dont know if it is leak or what.
My Allocation Tracker shows:
like 30:
138 96 char[] 9 android.content.res.AssetManager getCookieName
Then
Tons of:
32 80 android.graphics.BitmapFactory$Options 9 android.graphics.BitmapFactory decodeResource
And last:
Like 30:
141 56 android.graphics.Bitmap 9 android.graphics.BitmapFactory nativeDecodeAsset
And also some more of simular ones.
Here is some code that I think drains my memory:
player = BitmapFactory.decodeResource(getResources(), R.raw.ghostright);
world = BitmapFactory.decodeResource(getResources(), R.raw.lvl2);
thumb = BitmapFactory.decodeResource(getResources(), R.raw.thumb);
resized = Bitmap.createScaledBitmap(player, width/10, width/6, false);
player = resized;
resized = Bitmap.createScaledBitmap(world, height*10, height, false);
world = resized;
resized = Bitmap.createScaledBitmap(thumb, height/6, height/6, false);
thumb = resized;
I heard that I should use resycle but I dont know where because I always use the bitmaps
//Simon
PS: I really need help -.-
I use a lot of bitmaps on SurfaceView too and don't have this problem.
When it comes to animated sprites, you can use a sprite sheet rather than load them individually frame by frame.
You don't need to use the reference "resized" you can just say:
player = Bitmap.createScaledBitmap(player, width/10, width/6, true);
the old bitmap will lose its reference and be collected by GC. Note that I placed TRUE for bitmap filtering when rescaling to make a better quality.
On some devices onSizeChanged can happen two times which may resize bitmaps twice, if that is where you are doing your scaling.
The format of loaded bitmaps does metter whether it is ARGB_4444 or ARGB_8888 etc So you may need to explore this option and if you can use a format which requires less memory yet it has a good enough quality for your game. Of course the rule is not to load images into memory bigger than they are needed and when they are needed.
It dosen't have to be a memory leak, it could just be that you have so large bitmaps that they want to allocate to much memory. Here's a good method to determine how much memory a bitmap is going to take: W*H*8. So if you have a bitmap that's 300*300 px it's 300*300*8 = 720 kb.
Figure out how much allocated heap you have at any given time and see if it increases with time even though you know that you're not allocating new bitmaps. If so, then yes, you have a memory leak. If, however, your app crashes right on startup, then you are probably just exceeding heap limit.