Develop Reference

Android IMAPFolder.fetch OutOfMemoryError: what can I do about that?

EDIT:
I do not see this OOM error anymore. This was possibly related to some memory leak somewhere else in the app: I found one possible Activity leak: this could be the culprit and JavaMail has nothing to do with this activity leak... my mistake.
My simplified mail app is working fine for more than 1 year, but lately, I started to receive some bug report about OutOfMemoryError occurring on Samsung SM-T530 matissewifi 5.0.2 device (i.e. Galaxy Tab 4 (10.1'',Wi-Fi) ).
Code excerpt:
try
{
FlagTerm ft = new FlagTerm(new Flags(Flags.Flag.SEEN), false);
Message[] messages = folder.search(ft);
folder.fetch(messages, createFetchProfile());
return messages;
}
catch (Throwable th)
{
// I got OutOfMemoryError here because of folder.fetch(...):
/*
java.lang.OutOfMemoryError: Failed to allocate a 1036 byte allocation with 8388608 free bytes and 387MB until OOM; failed due to fragmentation (required continguous free 131072 bytes for a new buffer where largest contiguous free 65536 bytes)
or even this (with !0! for largest contiguous free bytes value!!!)
java.lang.OutOfMemoryError: Failed to allocate a 1036 byte allocation with 8388608 free bytes and 385MB until OOM; failed due to fragmentation (required continguous free 131072 bytes for a new buffer where largest contiguous free 0 bytes)
at com.sun.mail.iap.ByteArray.grow(SourceFile:161)
at com.sun.mail.iap.ResponseInputStream.readResponse(SourceFile:125)
at com.sun.mail.iap.Response.(SourceFile:121)
at com.sun.mail.imap.protocol.IMAPResponse.(SourceFile:66)
at com.sun.mail.imap.protocol.IMAPProtocol.readResponse(SourceFile:458)
at com.sun.mail.iap.Protocol.command(SourceFile:414)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(SourceFile:2440)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(SourceFile:2422)
at com.sun.mail.imap.IMAPFolder.fetch(SourceFile:1417)
at <mycode...>
*/
}
fetchprofile is
FetchProfile fetchProfile = new FetchProfile();
fetchProfile.add(UIDFolder.FetchProfileItem.ENVELOPE);
// fetch other info to speed up process
fetchProfile.add(UIDFolder.FetchProfileItem.FLAGS);
fetchProfile.add(UIDFolder.FetchProfileItem.UID);
fetchProfile.add(UIDFolder.FetchProfileItem.CONTENT_INFO);
fetchProfile.add(UIDFolder.FetchProfileItem.SIZE); // not sure about this one
I found grow() method source: here it is (quite simple):
public void grow(int incr)
{
byte[] nbuf = new byte[bytes.length + incr];
System.arraycopy(bytes, 0, nbuf, 0, bytes.length);
bytes = nbuf;
}
Manifest:
I already have
<application android:largeHeap="true">
in AndroidManifest.xml.
Questions:
Could this be related to a mail box containing so many email that mail API crash with OOM exception? I only fetch ENVELOPPE (and a few other things) , so I suppose this is not the correct explanation.
If, as I think, it's NOT related to the amount of email, what can I do about that?
Also, How should I interpret OOM message with value 0 for largest contiguous bytes!?! (sometimes happens, but not always)
java.lang.OutOfMemoryError: Failed to allocate a 1036 byte allocation with 8388608 free bytes and 385MB until OOM; failed due to fragmentation (required continguous free 131072 bytes for a new buffer where largest contiguous free 0 bytes)

Of course you should make sure that you're closing Folders when you're done with them, and not keeping references to Folders or Messages that you're no longer using. And make sure all the other parts of your application aren't causing memory leaks.
The Folder.fetch response needs to be kept in a single byte array. You can see how the byte array grows. Compared to the actual content of messages, the fetch information is usually quite small. But if you have thousands and thousands of messages, it could become large. Since you're only looking for unseen messages, it seems unlikely that that's the case.
I don't know what's possible on Android but if you could get a heap dump when this happens and examine it to determine which objects are using all the memory that would be most helpful.

Possible bug in dalvik of Android 2.x during Bitmap allocation?

The phenomenon: First do allocation some big memory blocks in the Java side until we catche OutOfMemoryError, then free them all. Now, weird things happen: load even a small picture(e.g. width:200, height:200) by BitmapFactory.decodeXXX(decodeResource, decodeFile, ...) will throw an OutOfMemoryError! But its OK to alloc any pure Java big Object(e.g. new byte[2*1024*1024]) now!
Verifying: I wrote some simple codes to verify the problem that can download here, press "Alloc" button many times and you will got an OOF Error, then press "Free All", now the environment is set up. Now you can press "LoadBitmap" and you will see its not work on most of Android 2.x phone.(But in the emulator its just OK, odd)
Digging deeper: I try to dig into some dalvik code to find out why, and find a possible bug in function externalAllocPossible in HeapSource.c which called by dvmTrackExternalAllocation who print the "xxx-byte external allocation too large for this process" messages in LogCat.
In externalAllocPossible it simply wrote:
if (currentHeapSize + hs->externalBytesAllocated + n <=
heap->absoluteMaxSize)
{
return true;
}
return false;
Which means once if the native Bitmap allocation size plus the currentHeapSize(NOT the actually allocated size as shown below, in this case, it's keeping the max size of the heap we bumped up but then freed them all) exceeds the limits, native Bitmap allocation will always fail, but the currentHeapSize in Java seems NOT decrease even when 91.3% Java objects' memory have been freed(set to null and trigger GC)!
Is there anybody else met this problem too?

I think this is correct. Its forcing the entire app (Java+native) take no more than a certain amount of memory from the OS. To do this it has to use the current heap size, because that amount of memory is still allocated to the app (it is not returned to the OS when freed by GC, only returned to the application's memory pool).
At any rate, 2.x is long dead so they're not going to fix it there. They did change how bitmaps store their memory in 3.x and 4.x. Your best bet is to allocate all the bitmaps you use first, then allocate those large structures. Or better yet- throw those large structures into a fixed size LRUCache, and don't use the grow until out of memory idea, instead load new data only when needed.

The class Bitmap has the recycle() method, described as:
Free the native object associated with this bitmap...
The reason behind this method is that there are two heaps: the Java heap and the heap used by native code. The GC only sees the Java heap sizes; for GC, a bitmap may look as a small object because it's size on the Java heap is small, despite the fact that it references a large memory block in the native heap.

How to work with a lot of bitmaps and avoid the error: "bitmap size exceeds VM budget"? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Android: Strange out of memory issue
I'm trying to work with four bitmaps of 1024 x 1024, I need to have four in memory because I am creating a gallery that must show four zoomable bitmaps in landscape mode.
It is working on Motorola Droid 2.2.3, but it is failing on galaxy spica (2.1). This is the exception that i get on spica:
02-28 15:45:25.963: ERROR/dalvikvm-heap(17109): 4194304-byte external allocation too large for this process.
02-28 15:45:25.968: ERROR/(17109): VM won't let us allocate 4194304 bytes
02-28 15:45:25.973: DEBUG/AndroidRuntime(17109): Shutting down VM
02-28 15:45:25.973: WARN/dalvikvm(17109): threadid=3: thread exiting with uncaught exception (group=0x4001b180)
02-28 15:45:25.973: ERROR/AndroidRuntime(17109): Uncaught handler: thread main exiting due to uncaught exception
02-28 15:45:25.988: ERROR/AndroidRuntime(17109): java.lang.OutOfMemoryError: bitmap size exceeds VM budget
how to solve this?
thanks

There is no easy way to solve that - single application has very low memory budget, so there is no way to keep 16MB in memory. You can use OpenGL (textures are not included into that budget), of just use one bitmap at time (or use smaller bitmaps).

You can cut the size in half by using RGB_565 versus the default ARGB_8888.
ARGB_8888 requires 4 bytes per pixel: 4*1024*1024 = 4194304 (4mb)
RGB_565 require 2 bytes per pixel: 2*1024*1024 = 2097152 (2mb)
You can set this with options to the BitmapFactory. See inPreferredConfig
However even with this you'll still have difficulty. Why not cut the size of the images down as well? I doubt you're able to show a 1024x1024 image anyway.

Android Heap Memory size doesn't looks correct

i have an Android app that displays alot of images, it works, the images are gatherd from an url, added to a que and gathered by 4 threads,stored in a cache and then displayed in a listview view 4 images for row, there are abot six rows at each time on the screen. There is a total of usually 90 images.
The rows(and imageviews) are always recycled, so the amount of items is always the same and i'm not initializing anything.
This seems to work quite fine, i have always an average used heap size of 13MB.
The problem i have is that at the beginning mi max heap size is quite small and i get GC messages like:
01-20 16:48:39.191: D/dalvikvm(9743): GC_FOR_ALLOC freed <1K, 31% free 12048K/17351K, paused 25ms
but the more i scroll up down the view the heap size grows more and more untile i get things like
01-20 17:02:05.339: D/dalvikvm(11730): GC_FOR_ALLOC freed 544K, 72% free 13871K/49159K, paused 35ms
as you see even if the used is the same the maximum is increased even if i never got to that limit. and the true problem is that at this point i start to get outofmemory errors.
Can someone explain me what's wrong?
Thanks!

What version of Android are you using? If you're testing on pre 3.0 (ie 2.x), the byte arrays that store most of the information in Bitmaps are allocated and stored in native memory. This means that in heap dumps and in the GC notifications, you only see the small amount of memory used for pointers in Bitmaps, rather than the actual size.
For more information check out this google IO talk on memory management and detecting memory leaks: http://www.youtube.com/watch?v=_CruQY55HOk
Also I've worked on several apps doing similar things. My guess is that either your cache size is way too large, or (more likely) the images you're displaying and storing in the cache are much larger than the size you actually want. If you display a bitmap in an image view, the imageview will store the original bitmap in memory, even if it is significantly larger than what would actually fit in the view. Try resizing the images from disk to at least closer to the appropriate size before trying to display them: How do I scale a streaming bitmap in-place without reading the whole image first?

To cache my Images I use Map<String, Drawable> drawableMap. On a OutOfMemoryError I call this function:
private void cacheLeeren()
{
int size = drawableMap.size();
int del = (int) (size * 0.3);
Set<String> s = drawableMap.keySet();
for (String t : s)
{
if (del >= 0)
{
drawableMap.put(t, null);
del--;
}
}
}
I think it's not the best way...but it works ;-)

My guess is that your app reaches a very high peak of memory usage for a short time. It's true that on average you only use 13MB but if your heap grows to as much as 50MB, it means that momentarily you've consumed much more memory than you're thinking.
Let's try to figure out where this is happening. You've mentioned that you're using an LRU cache. This cache frees memory as soon as it fills up. My guess is that you're starting to free memory too late, and this memory isn't freed immediately - since it depends on the system GC. Whenever you're freeing some items from the cache, try to call System.gc() manually.
You've also mentioned that you're calling Bitmap.recycle(). To the best of my knowledge this is useless on Android 3+ because the native heap is no longer used for bitmaps. Since all bitmaps are on the dalvik heap, they will be freed by the GC.. You can't rush this like before unless you call System.GC() yourself.
Another idea for your source of problems is heap fragmentation. See my previous SO answer to a similar issue in this question.

How to measure VRAM consumption on Android?

I want to acquire　Android Device VRAM size.
Is there a method for acquisition from the program?

Let's do some calculation using Nexus One:
Screen resolution is 480x800. So minimum required video memory size would be:
400 * 800 * 4 bytes = 1536000 bytes
Assuming that driver may (and normally should) use several buffers, we should also expect values like:
1536000 * 2 bytes = 3072000 bytes
1536000 * 3 bytes = 4608000 bytes
etc...
It would be weird to have values that are not multiple of 1536000 (or W x H x 4 in general).
After some searches on Android internals I've found this documentation :
...Android makes two requirements of the driver: a linear address space of mappable memory that it can write to directly...accessing the driver by calling open on /dev/fb0...
So I tried and take size of /dev/graphics/fb0 file (on my device there is no /dev/fb0).
But a direct approach doesn't work:
File file = new File("/dev/graphics/fb0");
file.length(); // ==0, doesn't work, no read access
Using next trick you can get actual size of fb0:
>adb pull /dev/graphics/fb0
1659 KB/s (4608000 bytes in 2.712s)
Video memory is ~4mb (Nexus One). Let's check if this is multiple of Nexus screen size:
4608000/1536000 = 3
It looks like a right value. And we also can say that driver uses three screen buffers.
So, as a conclusion, you can detect video memory size using adb, but you can't use this approach from your android application in runtime due to file access restrictions.

You typically do not have a dedicated "VRAM" on mobile devices. At least you don't have it with PowerVR architectures (wich totally dominate the market with their MBX and SGX cores).
That is, the OpenGL driver allocates normal RAM until you run out of it, and the more you allocate the less you have left for your application.

The Android/OpenGL APIs don't offer explicit methods to read the VRAM size from a given device.
Poor man solution:
You could try to infer the VRAM size in an empiric way adding 1MB texture until you get an out of memory error from gl.glGetError().

From your "dmesg" output u can read off the VRAM, so for my Tablet:
> [ 0.000000] Machine: TDM3730 [ 0.000000] Reserving 12582912
> bytes SDRAM for VRAM
>
> 7>[ 3.929962] VRAM: checking region 9f400000 3072
> <4>[ 3.929992] Failed. Allocating 4194304 bytes for fb 0
> <7>[ 3.935333] VRAM: alloc mem type 0 size 4194304 paddr dec2bd4c
> <7>[ 3.935485] VRAM: checking region 9f400000 3072
> <7>[ 3.935485] VRAM: found 9f400000, end a0000000
> <6>[ 3.936584] android_usb gadget: high speed config #1: android
> <4>[ 3.960113] allocating 4194304 bytes for fb 1
or details at:
http://pastebin.com/jQSXQqHh

Is simple just count how many Mb ram that from usable to real capacity of the ram, example for my lenovo a369i has 512 RAM Module, but in setting app only showing 471 Mb usable so the 41Mb left is reserved for the GPU, so the conclusion is my a369i has 41Mb vram
This method is based from shared graphics memory (wiki)

I suspect that android.os.StatFs is what you're looking for:
http://developer.android.com/reference/android/os/StatFs.html