I am instrumenting the Dalvik VM and would like to know if there are any tools to analyze garbage collection in dalvik. I know about allocation tracker but I'm looking for something more elaborate.
Get a log of all GC operations over time:
Every time a GC takes place, you get a line in your LogCat.
08-08 16:42:21.998: D/dalvikvm(26942): GC_CONCURRENT freed 773K, 26% free 4739K/6368K, paused 4ms+3ms, total 92ms
08-08 16:42:21.998: D/dalvikvm(26942): WAIT_FOR_CONCURRENT_GC blocked 11ms
It seems I'm getting those for all apps on my device.
This line includes lots of interesting stats about the GC like the amount of memory freed, how long the GC took, when exactly did it happen and the size of your heap (used/total).
Since all of those log lines have the tag dalvikvm, you should be able to collect and filter them over a long period of time and analyze to learn about GC behavior.
Analyzing a specific run of the GC:
If you want to analyze what happens in one specific GC operation, the best tool for the job is Eclipse MAT. Eclipse MAT can parse heap dumps. Take a heap snapshot, wait for the GC (or trigger it yourself using DDMS), and then take another snapshot.
Eclipse MAT can show you the delta between the two snapshots. Notice you'll see both new allocations and GC-caused deallocations. More info about comparing snapshots is available here.
Some other thoughts:
I'm not sure how much you would be able to learn from analyzing the GC process. The inner workings of the GC are an implementation detail. It can change without notice between OS versions / devices / configurations.
I'm trying to think of ways to improve the GC latency you're experiencing.. It seems to me the GC usually runs when memory conditions are low. This probably happens during new allocations, therefore the GC might be running while your service is active. Maybe, if you use the time your service is inactive to GC manually, you would be able to reduce the number of GC's happening in the critical path of responding to a web request. To try that, I would add a simple background timer, and reset it whenever my service becomes active (new request). When the timer ticks (inactivity for some period of time), I would run System.gc() manually.
Related
I'm getting this log in my logcat
I/zygote: Background concurrent copying GC freed 148531(5MB) AllocSpace objects, 18(936KB) LOS objects, 32% free, 12MB/18MB, paused 3.758ms total 217.199ms
is this something that needs to be addressed or it is can be ignored
This is not an error, it's just an Android log message notifying you about when garbage collection takes place. Everything's normal. The log messages don't harm your app. It's only a problem if you go out of memory, or you see performance hiccups due to garbage collection.
If you are seeing this frequently (or consistently), then you are likely allocating too many objects. A common cause is allocating many (or a few large) objects within a loop
Every time we hit this loop, we allocate one hundred new Bitmap objects.
The best way to prevent GC sweeps is to not allocate objects. Of course you have to allocate objects in Java, so you need to ensure that you are not allocating unnecessarily.
Finally, you can always filter or ignore the logs. :D
This page may help you with further research: https://blog.gceasy.io/2017/05/09/understanding-android-gc-logs/
Can anyone explain the below Logcat message :
D/dalvikvm(4440): GC_EXTERNAL_ALLOC freed 338K, 47% free 6427K/11911K, external 20418K/22446K, paused 53ms
E/dalvikvm-heap(4440): 2519424-byte external allocation too large for this process.
D/dalvikvm(4440): GC_FOR_MALLOC freed <1K, 47% free 6427K/11911K, external 20398K/22446K, paused 40ms
E/GraphicsJNI(4440): VM won't let us allocate 2519424 bytes
In older versions of Android, certain bits of native framework code would tell the VM about native allocations. This "external allocation" mechanism was an ugly hack introduced so that native allocations would cause the Dalvik VM to do a garbage collection pass.
The basic problem was that the Java-language Bitmap object used native memory for the pixel storage. Because the managed-heap objects were tiny, and the native-heap objects were large, you could allocate tons of Bitmaps without causing a GC. This was causing apps to bloat up and the system to slow down.
So, "external allocations" were introduced. Whenever the pixel storage for a Bitmap was allocated on the native heap, an equal amount of memory was deducted from the managed heap. The idea is that, if your heap is filling up with no-longer-reference bitmaps, you'll run out of managed heap space and the GC will fire.
(Unfortunately the GC can't actually release the native storage -- you need to run a finalizer to do that, and finalizers run in a separate pass after the GC completes. For a while the native objects were also holding on to some additional managed-heap objects, so you'd have to GC + finalize + GC to actually clean everything up.)
My "favorite" part about external allocations is that the API was a simple "increase by N" / "decrease by N", which meant there was no way to associate the native heap with a managed heap object, or check for leaks. Because all of the information about the Bitmap was kept in the native object, you couldn't even guess at how much native storage was needed, so it was impossible to look at an hprof dump and figure out how much memory a Bitmap was actually using.
In Android 3.0 ("Honeycomb") the pixel storage was moved onto the managed heap, and the external allocation mechanism was removed.
So, what the log message in your question means is: some code, probably Bitmap, wanted to allocate 2.5MB of native heap, but that would exceed the VM's external allocation heap limit. You need to figure out what's eating up 20MB of external allocation storage and release some of it.
The only way to get information about external allocations is by watching the event log. A few years back I threw a script together (gclog.py -- was in AOSP dalvik/tools for a while). I have no idea if it will still do anything useful. I talk about how to use it in this old thread.
I would like to know that why does the below message shows up everytime while running any application.
12-11 17:18:37.141: D/dalvikvm(3155): GC_CONCURRENT freed 485K, 9% free 6696K/7303K, paused 9ms+335ms
paused 9ms+335ms due to this pause my audio which i supposed to play is missing, because as per my code it receives audio data every 40ms so here it paused for 9ms+335ms which is 10 times data loss
I know its performing some kind of Garbage Collection but my question is why does it frequently comes in the logcat.
Thanks for any kind of help!!!!!!
Garbage Collection is used by Java to prevent you from going Out of memory by removing Objects that are no longer referenced by any class and cannot be reached from your code.
If you have too many of these objects, you will get GC calls alot, which may at some times affect your performance. On the other hand, having Objects that are referenced all the time, may prevent GC from being called (memory leak) and your memory may fill up and you get an OutOfMemoryException.
Your goal is not to eliminate GC, but reduce them as much as possible in methods that are delay sensitive and are running on the UI thread (like onDraw(), getView(), ... for e.g.).
That is normal behaviour for Android. The phone just does garbage collection.
For more information checkout the great Google I/O video on this:
Google I/O 2011: Memory management for Android Apps
Folks, check out these doc/links,its related to memory management:
http://static.googleusercontent.com/media/www.google.com/en//events/io/2011/static/presofiles/memory_management_for_android_apps.pdf
http://eclipse.org/mat/
http://therockncoder.blogspot.in/2012/09/fixing-android-memory-leak.html
Update -- Pardon my verbosity below. As concisely as possible, this behaves itself:
try {
mArrowBitmap = BitmapFactory.decodeFile(nam);
} catch(OutOfMemoryError e) {
Log.e(TAG, "OUT OF MEMORY ERROR!!! ["+e+"]");
}
meaning that it throws an OOME when the external heap is exhausted. And I can catch that, and use it in unit testing.
This, however, misbehaves:
try {
mArrowDrawable = new BitmapDrawable(mContext.getResources(), mArrowBitmap);
} catch(OutOfMemoryError e) {
Log.e(TAG, "OUT OF MEMORY ERROR!!! ["+e+"]");
}
meaning that it performs a really impressive suppuku. All I can do then is go through its pockets looking for loose change.
And for those with more time on their hands . . .
Problem: I have lots of bitmaps, some from Resources, some loaded externally. I do some unavoidable manipulation on the device (i.e. stuff I can't do ahead of time, or do server-side). Someplace during loading, I have at least one memory leak. I'm chasing that, and pretending to be competent (and spending lots of time staring at DDMS/MAT). Thanks to lots of comments on SO, I have done at least the basic research on bitmaps, especially on where they live pre-Honeycomb. My app runs fine Honeycomb and forward. It runs less fine on Gingerbread. I can either
(a) ignore the memory leak since Honeycomb and forward will eventually reclaim the heap memory and, instead, try to get everybody on the planet to stop using Gingerbread, or
(b) isolate and squish the memory leak(s).
Option (b) seems better.
On Gingerbread, I understand that the native "backing" memory for my bitmaps (including the ones that the system mysteriously creates on my behalf when I load .xml or Drawables), all lives outside the Dalvik heap. I further apprehend that the only thing on the Dalvik heap is a (relatively) small pointer to that native backing memory. When I attempt to release resources, I must locate all the bitmaps I have created, and do a synchronous recycle on them. My scheme for this, fwiw, is to "register" all my bitmaps into a simple vector when I create them, and then iterate the vector to recycle anything resembling a bitmap in native backing memory.
For this reason, I am untroubled by the various reports from the Dalvik heap that there is plenty of memory on hand, yet my decode calls fail for lack of memory. In fact, I have dedicated some of my unit-test activities to pushing the boundary right to the edge to verify I'm not misbehaving (and, hopefully, do not do so in the future). I do these unit tests using the "standard" Android unit test classes, since Robolectric can't "see" the heap to which I refer above. Confession: I really like Robolectric much better, because it's about a zillion times faster.
That is a long way of getting to here:
When I intentionally run the native 'backing' memory off the end, by creating too many bitmaps, most of the time, I get what I expect: OutOfMemoryError. I catch that Error and act responsibly (for very small values of 'responsibly').
When I prevail upon the Drawable calls (instead of the BitmapFactory calls, but sometimes both do this behavior), instead of an Error or an Exception, I get this in the logcat:
11-17 16:22:39.372: I/dalvikvm-heap(15069): Clamp target GC heap from 33.916MB to 32.000MB
11-17 16:22:39.372: D/dalvikvm(15069): GC_CONCURRENT freed 5K, 39% free 3701K/6023K, external 26854K/27965K, paused 2ms+2ms
11-17 16:22:39.402: D/dalvikvm(15069): GC_EXTERNAL_ALLOC freed 5K, 39% free 3696K/6023K, external 26854K/27965K, paused 36ms
11-17 16:22:39.442: E/dalvikvm-heap(15069): 81796-byte external allocation too large for this process.
11-17 16:22:39.442: E/dalvikvm(15069): Out of memory: Heap Size=6023KB, Allocated=3696KB, Bitmap Size=26854KB, Limit=32768KB
11-17 16:22:39.442: E/dalvikvm(15069): Trim info: Footprint=6023KB, Allowed Footprint=6023KB, Trimmed=404KB
11-17 16:22:39.442: E/GraphicsJNI(15069): VM won't let us allocate 81796 bytes
11-17 16:22:39.462: I/dalvikvm-heap(15069): Clamp target GC heap from 33.912MB to 32.000MB
11-17 16:22:39.462: D/dalvikvm(15069): GC_CONCURRENT freed <1K, 39% free 3696K/6023K, external 26854K/27965K, paused 3ms+2ms
I grok the clamping parts. What I can't seem to untangle is how to detect the part in the middle:
11-17 16:22:39.442: E/dalvikvm(15069): Out of memory: Heap Size=6023KB, Allocated=3696KB, Bitmap Size=26854KB, Limit=32768KB
It does not appear to be either an Error or an Exception. I can, of course, see this bad behavior manually by watching DDMS. But I can't figure out how to isolate it programmatically, so that I can incorporate it into my unit tests. It would, of course, also be nice to catch this behavior in my app, so that if a mistake does slip past my gauntlet of unit tests (it happens(way too often)) I might avoid the app unceremoniously barfing on its shoes.
Many thanks for the dedicated folks hereabouts who have explained the mysteries of Android memory with aplomb. For anybody still reading, and puzzled by as I was by Android's entertaining memory structure, I heartily recommend starting here, with Mr. Dubroy's excellent presentation -- Well worth watching, especially to the end of the audience questions. Right at the end is a person off-camera who asks some exceedingly well-informed questions about these same baffling memory bits.
Any guidance here would be gratefully accepted.
If you see the Android logs, you may see a lot of those things.
What do they mean, knowing those may help us doing better memory allocations.
Example:
28470 dalvikvm D GC_FOR_MALLOC freed 665 objects / 239992 bytes in 71ms
28470 dalvikvm D GC_FOR_MALLOC freed 673 objects / 240288 bytes in 87ms
21940 dalvikvm D GC_EXPLICIT freed 4802 objects / 185320 bytes in 78ms
28470 dalvikvm D GC_FOR_MALLOC freed 666 objects / 240536 bytes in 63ms
GC_FOR_MALLOC means that the GC was triggered because there wasn't enough memory left on the heap to perform an allocation. Might be triggered when new objects are being created.
GC_EXPLICIT means that the garbage collector has been explicitly asked to collect, instead of being triggered by high water marks in the heap. Happens all over the place, but most likely when a thread is being killed or when a binder communication is taken down.
There are a few others as well:
GC_CONCURRENT Triggered when the heap has reached a certain amount of objects to collect.
GC_EXTERNAL_ALLOC means that the the VM is trying to reduce the amount of memory used for collectable objects, to make room for more non-collectable.
Update: There has been a name-change of the first event in later versions of Android. It's now called "GC_FOR_ALLOC".
There is also a new event available, although very rare in modern phones:
GC_BEFORE_OOM means that the system is running really low on memory, and that there is a final GC performed, in order to avoid calling the low memory killer.
Another place where the Dalvik garbage collector messages are explained is in this video: Google I/O 2011: Memory management for Android Apps
At about 14 minutes into the presentation, he breaks down the message format. (BTW, that video has really good info on debugging memory leaks)
Roughly speaking, the format is [Reason] [Amount Freed], [Heap Statistics], [External Memory Statistics], [Pause Time]
Reason
Robert/yuku already gave info on the meaning of these.
Amount Freed
E.g. freed 2125K
Self explanatory
Heap Statistics
E.g. 47% free 6214K/11719K
These numbers reflect conditions after the GC ran. The "47% free" and 6214K reflect the current heap usage. The 11719K represents the total heap size. From what I can tell, the heap can grow/shrink, so you will not necessarily have an OutOfMemoryError if you hit this limit.
External Memory Statistics
E.g external 7142K/8400K
Note: This might only exist in pre-Honeycomb versions of Android (pre 3.0).
Before Honeycomb, bitmaps are allocated external to your VM (e.g. Bitmap.createBitmap() allocates the bitmap externally and only allocates a few dozen bytes on your local heap). Other examples of external allocations are for java.nio.ByteBuffers.
Pause Time
If it's a concurrent GC event, there will be two times listed. One is for a pause before the GC, one is for a pause when the GC is mostly done.
E.g. paused 3ms+5ms
For non-concurrent GC events, there is only one pause time and it's typically much bigger.
E.g. paused 87ms
I also found this in the Android sources, dalvik/vm/alloc/Heap.h. May this be useful.
typedef enum {
/* Not enough space for an "ordinary" Object to be allocated. */
GC_FOR_MALLOC,
/* Automatic GC triggered by exceeding a heap occupancy threshold. */
GC_CONCURRENT,
/* Explicit GC via Runtime.gc(), VMRuntime.gc(), or SIGUSR1. */
GC_EXPLICIT,
/* GC to try to reduce heap footprint to allow more non-GC'ed memory. */
GC_EXTERNAL_ALLOC,
/* GC to dump heap contents to a file, only used under WITH_HPROF */
GC_HPROF_DUMP_HEAP
} GcReason;