This is happening on an embedded system that is using a custom build of Android 4.0.2 platform. I see one of our android activity apps growing to around 400MB (rss size when "ps" is invoked) and getting killed by Linux OOM killer.
The android platform was configured with max heap size set to 62M. I am clueless how Dalvik VM let the activity grow to 400MB.
Shouldn't the app get Java out of memory exceptions when heap reaches around 60MB?
We don't see those Java exceptions in the logcat logs or in anr traces.
We implemented a sample activity that allocates byte arrays in sequence and set each byte to a dummy value. We do see Outofmemory exceptions when the activity allocated around 60MB.
Are there allocation paths in android that don't get counted towards heap budget?
The activity renders bitmap pngs downloaded from a web site.
Below are "getprop" results on our platform.
$ adb shell getprop | grep -i heap
I appreciate any pointers.
Thanks
Edited:
Note:
Below is ps output. The Pss and Uss are around 316M which is way above.
PID Vss Rss Pss Uss cmdline
logcat: hd[0]: pexecd(65): 982 351512K 351316K 326300K 316632K mytest.home^M
logcat: hd[0]: pexecd(65): 660 679916K 61044K 57200K 56952K ./videngine^M
RAM: 741764K total, 20320K free, 2148K buffers, 80104K cached, 24964K shmem, 10368K slab
Direct allocations in native code don't count against that Java heap total. There may be other possibilities as well (perhaps pages mapped and populated from files?).
If you have a custom android build, you may be able to set OOM killer values to preserve your own application.
I see one of our android activity apps growing to around 400MB (rss size when "ps" is invoked)
To quote Dianne Hackborn, regarding the output of ps on Android: "the Vss and Rss columns are basically noise (these are the straight-forward address space and RAM usage of a process, where if you add up the RAM usage across processes you get an ridiculously large number)"
I would heartily encourage you to read her epic SO answer on measuring an app's memory footprint. Notably, Rss plays no role in her analysis, beyond the quote I cited above. Hence, I would suggest not worrying about Rss and focus on other metrics.
Here is what you can do to get an idea of what memory your app is using.
Go to DDMS, and create a heap dump by clicking the icon that looks like this:
Next convert the HPROF in android format to regular HPROF format using the hprof-conv tool in the androir-sdk/tools folder.
Next, open the heap dump with Eclipse Memory Analyser (MAT), and look at the dominator tree, there you will see a list of variables that your app is forcing the Dalvik Garbage Collector (GC) to keep. Right click on them and go to "Path the GC root", and "Exclude Weak References", will show you the references thats keeping those objects alive. Look and see if you have any expired references that's been kept as a memory leak.
You can watch this video for much detailed way of find memory leak in Android application.
Related
This problem may be related to Android or Linux. If you are not familiar with Android, let me introduce init and bootanimation in brief first. Android init is the same as Linux init. It is the first process in user space. The pid is 1. Init process will do some system initializations defined in init.rc. One of the initialization is the boot animation. It is played by the process named bootanimation. Bootanimation is a native process. It plays the animation using OpenGL ES. Boot animation is usually described by a zip file with dozens or hundreds of static images. Play them in certain rate becomes animation.
Now the strange problem comes. It's about memory occupation of init process. In a clean android 8.1.0 system, the memory occupied by init exceeds 40M+. We can find out allocation detail by adb shell command 'dumpsys meminfo init'.
Pss
Total
------
Native Heap 1076
Dalvik Heap 0
Stack 16
Other dev 363
Other mmap 333
EGL mtrack 10040
GL mtrack 29170
Unknown 32
TOTAL 41030
We can see that graphics related EGL/GL consumes most of the memory. Normally, init is a backend process. It has nothing to do with graphics. Then why graphics related memory takes the most. The only thing related to graphics I can figure out is boot animation. As described above, it is started/forked by init process. So I did some tests. If I disabled boot animation, memory occupation of init did decrease a lot to about 20M. I also did other tests. I tried to modify the source code of bootanimation to load and render a single image instead of hundreds of images. Memory occupation of init also decreased a lot. It means memory occupation of init has something to do with numbers of images loaded by bootanimation. I did further test and found that memory occupation of init will decrease a lot after forking bootanimation process by init. It seems that memory is transferred from init to its child process bootanimation.
Then comes my final question: as a child process, why will bootanimation affect the memory occupation of its parent process? When bootanimation process exits, shouldn't all resources occupied be release?
Extra comments: Although in my sense, when a process exits, all resources should be released. And the android source codes written by Google should be seasoned and have no bug. I still doubted about leak of resources in bootanimation. Bootanimation uses SkBitmap, which is in skia library, to load images and use OpenGL ES 1.0 textures to render them. I confirmed if I didn't confirm wrongly, that SkBitmap and OpenGL textures would be destroyed correctly. You guys may have interests in related android source codes.
Core code for playing animation in bootanimation:
http://androidxref.com/8.1.0_r33/xref/frameworks/base/cmds/bootanimation/BootAnimation.cpp#862
Code for loading image:
http://androidxref.com/8.1.0_r33/xref/frameworks/base/cmds/bootanimation/BootAnimation.cpp#190
Code for starting bootanimation in init:
http://androidxref.com/8.1.0_r33/xref/system/core/init/service.cpp#688
Thanks for all your answers, suggestions and comments.
After a new series of tests and code reading, I found something new. I will add some new comments. This problem has nothing to do with init. The problem is related to implementation of dumpsys meminfo. Process memory displayed by dumpsys meminfo consists of two parts. One part is directly allocated by this process. Such as Dalvik Heap and Native Heap. While the other part is not directly allocated by this process, it is allocated by system just caused by this process. EGL/GL mtrack part shows above belong to this part. Tracking this part of memory is implemented by different hardware platform. Is is counted by display driver. What I am using is a MTK platform. It will track GL memory for those processes that do real GL operations. I believe these statistics is accurate. But there is total GL memory consumed by the system, that is the total GL mtrack value we can see when execute dumpsys meminfo without extra parameters. It may be not equal to the sum of counted GL values of all accounted processes. Then it accounts the difference of the total GL memory and sum of GL memory to init process. Because init is the ancestor process of all other processes. So the value to GL mtrack of init looks large, it is not allocated by init nor caused by init actually.
Now we can explain the very strange problem above and know it has nothing to do with process init. But my question still remain unresolved. That is, why event bootanimation is terminated, GL memory allocated by it is not freed? Does it mean leak of resource? Or maybe can I think it as a bug of mtk display driver?
I was looking at the output given below for the command
adb shell dumpsys meminfo com.imangi.templerun
However, I am not able to understand this properly.
Can anybody help me understand this?
Since columns and rows presented may vary for different versions of 'dumpsys', I'll try to provide some generic overview here...
Every application in Android runs in different process that is running instance of its own Dalvik VM.
Native Heap row represents memory used by the process itself (Ex: Native C mallocs).
Dalvik Heap is memory allocated by Dalvik VM (Ex: Variables in your Java Android code).
Dalvik Other is memory used for JIT and GC.
Android may share pages of memory among several processes (Think code of common frameworks). Clean memory is one that hasn't changed since it was allocated or loaded from storage (Code of your application). Dirty memory is space used for computations. Android does not have swap mechanism so Dirty memory is also RAM that will be freed when app exits.
Private Dirty is unshared dirty memory (guess you figured that).
Private Clean is unshared clean memory (and that).
PSS Proportional Set Size: Is a tricky measurement where all private pages contribute 100% of their size and shared memory contribute 'size/(num of processes shared)'. This way if you sum up all PSS for all processes you'll get total memory used.
Swapped Dirty No idea. I have question regarding this:Android dumpsys meminfo "Swapped Dirty" coloumn meaning?
Further reading:
https://developer.android.com/tools/debugging/debugging-memory.html
I'm working on an app and I have memory issues.
I started to study this thing and I have met Eclipse's debugging system.
I use DDMS's Heap tester to see how much memory my app allocated.
I saw it's about 90%.
Now I made a simple new project, a blank empty activity without any functions or variables. Just a splendid new project.
I ran this heap tester and I saw the results:
Heap size: 10,629 MB
Allocated: 9,189 MB
Free: 1,440 MB
Used: 86.45 %
Objects: 44,565
Well, is it normal?
I have a very simple blank activity, and nothing else, and this app is used 86% of memory?
Allocated 9 MB of 10? Really? Is that normal? How this works?
Please instruct me about this, because I would like to know how these memory allocations work.
Dalvik will initially allocate a certain heap size to your app. In your case, this is around 10 MB. As your app needs more memory, Dalvik will increase the heap size upto the maximum configured size (which is different for different devices). If your app still needs more memory after the maximum is reached, then it will cause a OutOfMemoryException.
To learn more about analyzing memory allocations in Android, check out this excellent article from the Android developers blog:
http://android-developers.blogspot.in/2011/03/memory-analysis-for-android.html
Examining Heap Usage is somewhat tricky but is equally easy. Let's find out how.
So consider a small application. You have Android debugging tools to determine the heap usage and to examine them.
You can check this- memory-analysis-for-android, which have more details of how to analize the application effectively in android.
Let's have a short description here too:
There are two ways to start DDMS-
1) Using Eclipse: click Window > Open Perspective > Other... > DDMS
2) or from the command line: run ddms (or ./ddms on Mac/Linux) in the tools/ directory
Then select your application process from Devices and click "Update Heap".
Now switch to the Heap tab in DDMS.
To see the first update, click the Cause GC button.
You will see something like this:
We can see that our set (the Allocated column) is a little over 20MB. If you do some little flip flop, that number can go up. In small applications, the amount of memory we leak is bounded. In some ways, this can be the worst kind of leak to have, because we never get an OutOfMemoryError indicating that we are leaking.
You can use Heap Dump to identify the problem. Click the Dump HPROF file button in the DDMS toolbar and save the file wherever you want. Then run hprof-conv on it.
Using MAT which is a powerful Memory Analyzer tool-
You can install MAT from SITE which is a stand-alone Memory Analyzer tool and analyze the Heap dumps using it.
NOTE:
If you're running ADT (which includes a plug-in version of DDMS) and have MAT installed in Eclipse as well, clicking the "dump HPROF" button will automatically do the conversion (using hprof-conv) and open the converted hprof file into Eclipse (which will be opened by MAT).
Start the MAT and load the converted HPROF file. Navigate to the Histogram view which shows a list of classes sortable by the number of instances, the shallow heap (total amount of memory used by all instances), or the retained heap (total amount of memory kept alive by all instances, including other objects that they have references to).
If we sort by shallow heap, we can see that instances of byte[] are at the top.
Next, Right-click on the byte[] class and select List Objects > with incoming references. This produces a list of all byte arrays in the heap, which we can sort based on Shallow Heap usage.
Pick one of the big objects, and drill down on it. This will show you the path from the root set to the object - the chain of references that keeps this object alive. Lo and behold, there's our bitmap cache!
MAT can't tell us for sure that this is a leak, because it doesn't know whether these objects are needed or not -- only the programmer can do that. However, looking at the stats it is predictable to know that the cache is using a large amount of memory relative to the rest of the application, so we might consider limiting the size of the cache.
Go this way all along for all, and you will see a tremendous amount of performance optimization.
What you see here is allocated memory and not maximum memory which can be allocated, maximum memory which can be allocated depends upon android version and device to device.
In this case, your apps does not have any high memory requirement, all the files,system and object being used to run the app is very small hence initially android has allocated your app a common initial space,now this space goes on increasing as demand from app increases until its met, or it exceeds maximum heap size defines per app by android, in this scenario your app will crash stating running out of memory as reason.
To read more about memory allocation in android go through below developer link
http://developer.android.com/training/articles/memory.html
I am trying to understand how much memory is allocated for each process in Android. As I understand, for each app process there is a seperate instance of Zygote process and Zygote will fork Dalvik VM.
I wrote a small sample java application which just print's "hello world" in a loop. Later invoked Dalvik VM through ADB shell. Now, Procrank command shows that,
PID Vss Rss Pss Uss cmdline
9374 7556K 7556K 3600K 3384K dalvikvm
USS which is shown here is 3384k.
Later, I wrote a small android program with one activity and TextView. Now procrank shows
PID Vss Rss Pss Uss cmdline
6540 37256K 35124K 4387K 3392K com.example.helloworld
My questions are:
If each app process contains Dalvik VM, won't it take more memory?
How do we know, the size of memory occupied by VM and the size occupied by the Java process?
My understanding might be wrong with respect to Dalvik and how it is invoked and the memory occupied by VM in each process.
Any insights are welcome.
Thanks.
Okay I waited for quite some time expecting an expert answer for this question . Let me share you what i know .
Answer to first question : "If each app process contains Dalvik VM, won't it take more memory?"
The Dalvik heap is preloaded with classes and data by zygote (loading over 1900 classes as of Android version 2.2). When zygote forks to start an android application, the new application gets a copy-on-write mapping of this heap. As Dan Borstein says below, this helps with memory reduction as well as application startup time.
It's used in Android to amortize the RAM footprint of the large amount of effectively-read-only data (technically writable but rarely actually written) associated with common library classes across all active VM processes. 1000+ classes get preloaded by the system at boot time, and each class consumes at least a little heap for itself, including often pointing off to a constellation of other objects. The heap created by the preloading process gets shared copy-on-write with each spawned VM process (but again doesn't in practice get written much). This saves hundreds of kB of dirty unpageable RAM per process and also helps speed up process startup.
To your 2 question: "How do we know, the size of memory occupied by VM and the size occupied by the Java process?"
I didnt understand what you mean by 'size occupied by the Java process' check link for clarity about memory in dalvik.
In addition to procrank we can also use adb shell dumpsys meminfo 'your apps package name' check this link for more info on dumpsys.
The size of the VM heap cannot exceed 16mb, 24mb, 32mb depending on the phone.
But what is the maximum size of the native heap? How much native memory can be allocated to the app when it is in foreground.
Thanks.
Technically there's no restriction in the NDK. Someone asked this a while back and was referred to this android-ndk Groups thread. A relevent quote:
"Also given that this is the NDK list, the limit is actually
not imposed on you, because it is only on the Java heap. There is no limit on
allocations in the native heap..."
Dianne Hackborn
She does go on to say that it shouldn't be abused and if it is than applications could be killed.
There's no simple answer to this; you can use as much memory as the device has, minus what it's using for other programs. When Android thinks it's low on memory, it'll start killing background tasks, so it's a soft limit. Most devices do not have swap space. You can get some statistics about the device's memory from inside Dalvik with android.app.ActivityManager.MemoryInfo (I assume there's an NDK equivalent).
adb shell dumpsys meminfo PACKAGENAME will give you native and dalvik memory usage of your app.