Develop Reference

Android 32-bit ARM OOM killer triggered unexpectedly

I am running a relatively simple program that calls mmap() repeatedly to obtain 100MB+ memory (and touch every single page of them). The problem is that this program always triggers the OOM killer in the kernel (based on 3.4) even though there are still plenty of physical pages (2G RAM in total and 1.6G+ is free).
I do see that these two lines of output seem to indicate that there are not many small free blocks left in HighMem (but many big ones are available still):
Normal: 1220*4kB 1034*8kB 791*16kB 547*32kB 289*64kB 130*128kB 39*256kB 9*512kB 5*1024kB 3*2048kB 149*4096kB = 714608kB
HighMem: 1*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 2*2048kB 254*4096kB = 1046484kB
But I just can't wrap my head around this when the system also says that there are 524288 pages of RAM and 440840 free pages (most are free).
Below is the detailed output. Any help is appreciated!
test_program invoked oom-killer: gfp_mask=0x0, order=0, oom_adj=-17, oom_score_adj=-1000
[<c0011074>] (unwind_backtrace+0x0/0xe0) from [<c00813f0>] (dump_header.isra.16+0x74/0x18c)
[<c00813f0>] (dump_header.isra.16+0x74/0x18c) from [<c0081674>] (oom_kill_process.part.19.constprop.20+0x44/0x268)
[<c0081674>] (oom_kill_process.part.19.constprop.20+0x44/0x268) from [<c0081d64>] (out_of_memory+0x328/0x3f4)
[<c0081d64>] (out_of_memory+0x328/0x3f4) from [<c0081ef0>] (pagefault_out_of_memory+0xc0/0x154)
[<c0081ef0>] (pagefault_out_of_memory+0xc0/0x154) from [<c00120e0>] (do_page_fault+0x1bc/0x244)
[<c00120e0>] (do_page_fault+0x1bc/0x244) from [<c0008364>] (do_DataAbort+0x34/0x98)
[<c0008364>] (do_DataAbort+0x34/0x98) from [<c000d3f4>] (__dabt_usr+0x34/0x40)
Exception stack(0xed479fb0 to 0xed479ff8)
9fa0: aee1cffc aeec2000 0db4da5f aee1d000
9fc0: 00000000 0001b184 00000040 00000059 00000000 00000000 00000000 be9ccacc
9fe0: 00000064 be9cc9f0 0000b037 0000830c 80000030 ffffffff
Mem-info:
Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 157
HighMem per-cpu:
CPU 0: hi: 186, btch: 31 usd: 39
active_anon:37727 inactive_anon:936 isolated_anon:0
active_file:4108 inactive_file:25830 isolated_file:0
unevictable:1 dirty:12 writeback:0 unstable:0
free:440273 slab_reclaimable:1457 slab_unreclaimable:2172
mapped:12414 shmem:944 pagetables:752 bounce:0
Normal free:714608kB min:3512kB low:5052kB high:5932kB active_anon:0kB inactive_anon:0kB active_file:788kB inactive_file:1632kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:772160kB mlocked:0kB dirty:48kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:5828kB slab_unreclaimable:8688kB kernel_stack:2032kB pagetables:3008kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 10223 10223
HighMem free:1046484kB min:512kB low:3128kB high:4616kB active_anon:150908kB inactive_anon:3744kB active_file:15644kB inactive_file:101688kB unevictable:4kB isolated(anon):0kB isolated(file):0kB present:1308608kB mlocked:4kB dirty:0kB writeback:0kB mapped:49656kB shmem:3776kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Normal: 1220*4kB 1034*8kB 791*16kB 547*32kB 289*64kB 130*128kB 39*256kB 9*512kB 5*1024kB 3*2048kB 149*4096kB = 714608kB
HighMem: 1*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 2*2048kB 254*4096kB = 1046484kB
30879 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/1
Free swap = 0kB
Total swap = 0kB
524288 pages of RAM
440840 free pages
9650 reserved pages
3629 slab pages
59279 pages shared
0 pages swap cached

What's the data in dalvik-LinearAlloc, dalvik-aux-structure, dalvik-bitmap-1, dalvik-bitmap-2, dalvik-card-table, dalvik-mark-stack and dalvik-zygote?

I use showmap command on a PID, and I can't understand this part in the report:
16384 3752 689 0 3132 0 620 4 /dev/ashmem/dalvik-LinearAlloc (deleted)
2460 1748 934 0 828 0 920 18 /dev/ashmem/dalvik-aux-structure (deleted)
8192 572 572 0 0 0 572 1 /dev/ashmem/dalvik-bitmap-1 (deleted)
8192 0 0 0 0 0 0 1 /dev/ashmem/dalvik-bitmap-2 (deleted)
4100 312 312 0 0 0 312 1 /dev/ashmem/dalvik-card-table (deleted)
502140 14860 14860 0 0 0 14860 3 /dev/ashmem/dalvik-heap (deleted)
1500 280 280 0 0 0 280 1 /dev/ashmem/dalvik-jit-code-cache (deleted)
174764 0 0 0 0 0 0 1 /dev/ashmem/dalvik-mark-stack (deleted)
22148 22148 2141 0 20452 0 1696 1 /dev/ashmem/dalvik-zygote (deleted)
I want to know what data is in the dalvik-LinearAlloc, dalvik-aux-structure, dalvik-bitmap-1, dalvik-bitmap-2, dalvik-card-table, dalvik-mark-stack and dalvik-zygote.
These ashmem cost millions bytes memory, and I want to find a measure to shrink the size of these ashmem.

showmap is dumpping the smap data from one process. The smap is describing the process's memory area's detail. In virtual memory manage system, the memory can be gained by the system API such as mmap, brk. After gaining virtual memory address by these APIs, the address and length will be recorded in the smap.
And let's list each section of the dalvik relative memory usage:
Dalvik Heap section(Heap Management, GC)
dalvik-bitmap-1, dalvik-bitmap-2 is the Dalvik Heap management data stucture. In Dalvik, the GC is marksweep, and 8 bytes memory will be marked(Used or free) as one bit in the bitmap. These two bitmaps will be used as active map(used for marking # runtime) and the other will be used as marked map(used # GC time).
dalvik-mark-stack: For GC mark step use. The mark step will iterate the bitmap, so this is a Breadth-first search which will need a stack.
dalvik-card-table: is used for Dalvik Concurrent GC, in bitmap marking steps, the process will do other tasks which will lead using memory. These card tables is recording the memory dirty after first marking step. You can see the detail by searching mark sweep GC.
dalvik-heap is used for process memory usage
dalvik-zygote is one part of the hole heap, which will not be used # GC. All processes will share these memories such as framework resources.
dalvik-jit is The jit memory used in Dalvik. JIT: just in time, which will convert dex bytecode to machine code which can be executed by CPU.
dalvik-LinearAlloc: is the dalvik's perm memory such as: Method, Class definition datas, thread stack datas. These memory can be setted READONLY after parsing the class definition.
dalvik-aux-structure: auxillary data structures, which will compress the method/class/string const reference. These references will be used # each dex file, but sum of these memory will cost a large memory. So Dalvik create a tmp memory to shared these references.
If you want to analysis your program's memory, I suggest you to use MAT in eclipse. And the native heap usage, you can use mmap to manage.

Android lsmod output

I'm having a hard time trying to figure out the lsmod output from Android 4.0.3. What all those "Live ADDR" entries mean?
option 14926 0 - Live 0xbf12b000
usb_wwan 9419 1 option, Live 0xbf124000
cdc_acm 15150 0 - Live 0xbf11b000
g_crypto 11918 0 - Live 0xbf0cc000
g_wmt_cmoscam 1421 0 - Live 0xbf0c8000
g_wmt_v4l2_cmos 68606 1 g_wmt_cmoscam, Live 0xbf0b1000
g_wmt_v4l2 5135 1 g_wmt_v4l2_cmos, Live 0xbf0ab000
g_wmt_ve_h264 11644 0 - Live 0xbf0a5000
g_wmt_ve_com 9021 1 g_wmt_ve_h264, Live 0xbf09e000
g_wmt_vd_vpx 65936 0 - Live 0xbf088000
g_wmt_vd_rv 735 0 - Live 0xbf084000
g_wmt_vd_vc1 713 0 - Live 0xbf080000
g_wmt_vd_h264 715 0 - Live 0xbf07c000
g_wmt_vd_h263 715 0 - Live 0xbf078000
g_wmt_vd_divx 715 0 - Live 0xbf074000
g_wmt_vd_mpeg4 717 0 - Live 0xbf070000
g_wmt_vd_mpeg2 717 0 - Live 0xbf06c000
g_wmt_vd_jdec 18829 0 - Live 0xbf063000
g_wmt_lock 2881 5 - Live 0xbf05f000
g_wmt_vd_com 158558 12 g_wmt_vd_vpx,g_wmt_vd_rv,g_wmt_vd_vc1,g_wmt_vd_h264,g_wmt_vd_h263,g_wmt_vd_divx,g_wmt_vd_mpeg4,g_wmt_vd_mpeg2,g_wmt_vd_jdec, Live 0xbf032000
mali 102375 6 - Live 0xbf00d000
ump 27968 11 mali, Live 0xbf000000
I'm asking this because I use the exact same kernel to boot Ubuntu, and this is the output for lsmod on Ubuntu
option 14926 0
g_wmt_lock 2881 0
g_wmt_vd_jdec 18829 0
g_wmt_vd_mpeg2 717 0
g_wmt_vd_mpeg4 717 0
g_wmt_vd_divx 715 0
g_wmt_vd_h263 715 0
g_wmt_vd_h264 715 0
g_wmt_vd_vc1 713 0
g_wmt_vd_rv 735 0
g_wmt_vd_vpx 65936 0
g_wmt_ve_h264 11644 0
g_wmt_ve_com 9021 1 g_wmt_ve_h264
g_wmt_cmoscam 1421 0
g_wmt_v4l2_cmos 68606 1 g_wmt_cmoscam
g_wmt_v4l2 5135 1 g_wmt_v4l2_cmos
g_wmt_vd_com 158558 9 g_wmt_vd_jdec,g_wmt_vd_mpeg2,g_wmt_vd_mpeg4,g_wmt_vd_divx,g_wmt_vd_h263,g_wmt_vd_h264,g_wmt_vd_vc1,g_wmt_vd_rv,g_wmt_vd_vpx
g_crypto 11918 0
mali 102375 0
ump 27968 1 mali
cdc_acm 15150 0
usb_wwan 9419 1 option
rt3070sta_rtl8188 442205 0
HDMI, Camera, etc. are not working on Ubuntu, although the same modules are loaded on both OS's. That's why I asked about the Live and ADDR columns. I'd like to know if this has anything to do with the lack of functionality on Ubuntu.

lsmod just reads from the file /proc/modules (it actually calls cat /proc/modules)
This is the same for many UNIX like OSes:
The first column contains the name of the module.
The second column refers to the memory size of the module, in bytes.
The third column lists how many instances of the module are currently loaded. A value of zero represents an unloaded module.
The fourth column states if the module depends upon another module to be present in order to function, and lists those other modules.
The fifth column lists what load state the module is in: Live, Loading, or Unloading are the only possible values.
The sixth column lists the current kernel memory offset for the loaded module. This information can be useful for debugging purposes, or for profiling tools such as oprofile.
Source

Android Memory Usage and Objects in Memory not being released in 1.6 and 2.1

I'm checking my app for Memory Leaks/Usage and came across something weird that I've only seen so far in Android 1.6 and 2.1. After clicking around in the app a bit and I run "adb shell dumpsys meminfo" for my application, I see the following:
DUMP OF SERVICE meminfo:
Applications Memory Usage (kB):
Uptime: 34639912 Realtime: 153524709
** MEMINFO in pid 5778 [com.app.myapp] **
native dalvik other total
size: 14336 4679 N/A 19015
allocated: 13971 4139 N/A 18110
free: 280 540 N/A 820
(Pss): 2986 4181 13491 20658
(shared dirty): 972 3948 620 5540
(priv dirty): 2876 3224 10976 17076
Objects
Views: 545 ViewRoots: 4
AppContexts: 32 Activities: 31
Assets: 2 AssetManagers: 2
Local Binders: 43 Proxy Binders: 79
Death Recipients: 2
OpenSSL Sockets: 1
SQL
heap: 91 dbFiles: 0
numPagers: 4 inactivePageKB: 0
activePageKB: 0
Asset Allocations
zip:/data/app/com.app.myapp.apk:/resources.arsc: 119K
As you can see, nothing is getting deallocated/GC'd, the Activities are piling up, the AppContexts, etc. until the app just crashes with an OutOfMemoryError. This doesn't happen on 2.2+.
Can anybody give me some insight into why this is happening? I have a feeling it's either something simple, or it's just something weird with my app, but I'm at a loss as to why this is happening.
FYI, I've reproduced this in a 1.6 and 2.1 emulator, as well as my G1 running 1.6. A recent crash report from a user also shows this, which they were running 2.1 on a Droid Eris. Let me know if any more details/code is needed to help with this.
##UPDATE##
Thanks to the info from momo, I was able to track down some memory leak issues, which drastically cut down on the amount of Activities/AppContexts that would show in the Objects list of meminfo.
The number is now down to around the number of actual activities that are in my application, so it seems that on older versions of Android, it will show the total amount of objects your app is consuming. On newer versions it won't, though that could just be only the case on my test devices.

To get a clear picture on why Activities are held up, I normally use MAT and then look at Path to GC root from the Activity that get stuck.
I've a created a simple project which load simple TestActivity in order to illustrate the process. Below is the code for it:
package com.so;
import android.app.Activity;
import android.os.Bundle;
public class TestActivity extends Activity {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
}
}
Here are the steps:
Dump the hprof on the running process via DDMS "Dump HPROF File" function
Assuming you have MAT installed, this should bring up the MAT screen
Now filter based on your activity package, for the sample above, it is com.so. Screenshot for this process is below:
Now you want to see if this has a clear path to GC. You do that by right clicking the Activity and show all references as shown below:
You should see that your Activity is held by com.android.internal.policy.impl.PhoneWindow$DecorView and no one else. If this is the a case, you are ok and this Activity will be eventually reclaimed by GC.
Now I will do change my class to include a static variable that will hold its own instance:
package com.so;
import java.util.ArrayList;
import android.app.Activity;
import android.content.Context;
import android.os.Bundle;
public class TestActivity extends Activity {
static ArrayList memoryLeakList;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// create a deliberate static list to cause the leak
TestActivity.memoryLeakList = new ArrayList();
TestActivity.memoryLeakList.add(this);
}
}
And if I run the code hprof with the same steps, I now get the reference of the Activity is held by the ArrayList and not the com.android.internal.policy.impl.PhoneWindow$DecorView signifying that there is possibility of a leak if I don't clean up the array
Now, you don't have to do that for every Activity, what I would do just briefly run the app and then dump the HPROF. You would then again filter by package to get the snapshot of your application. In the initial Histogram, you should be suspicious for any Activity that has number of instances more than one after hitting GC button in DDMS and start investigating from there.
One more note, on my 2.1 phones, I couldn't get the HPROF via DDMS, so I did it through the emulator following these steps:
Go to ./adb shell
Type ps to get the pid of your app process
Type kill -10 , you should see in your logcat that it is dumping the memory to /data/misc
If you get permission denied, make sure you read/write on that folder by doing chmod 777 data/misc
Pull the hprof generated by either using DDMS File Explorer in Eclipse or pull command
Since the hprof is dalvik based, in order to use it with memory profiling tools you need to convert it first via hprof-conv available in the tools directory of your Android SDK installation
Run ./hprof-conv [source dump] [target dump]

Use Memory Analyzer (MAT) to view your process Heap.
http://www.eclipse.org/mat/

How to measure VRAM consumption on Android?

I want to acquire　Android Device VRAM size.
Is there a method for acquisition from the program?

Let's do some calculation using Nexus One:
Screen resolution is 480x800. So minimum required video memory size would be:
400 * 800 * 4 bytes = 1536000 bytes
Assuming that driver may (and normally should) use several buffers, we should also expect values like:
1536000 * 2 bytes = 3072000 bytes
1536000 * 3 bytes = 4608000 bytes
etc...
It would be weird to have values that are not multiple of 1536000 (or W x H x 4 in general).
After some searches on Android internals I've found this documentation :
...Android makes two requirements of the driver: a linear address space of mappable memory that it can write to directly...accessing the driver by calling open on /dev/fb0...
So I tried and take size of /dev/graphics/fb0 file (on my device there is no /dev/fb0).
But a direct approach doesn't work:
File file = new File("/dev/graphics/fb0");
file.length(); // ==0, doesn't work, no read access
Using next trick you can get actual size of fb0:
>adb pull /dev/graphics/fb0
1659 KB/s (4608000 bytes in 2.712s)
Video memory is ~4mb (Nexus One). Let's check if this is multiple of Nexus screen size:
4608000/1536000 = 3
It looks like a right value. And we also can say that driver uses three screen buffers.
So, as a conclusion, you can detect video memory size using adb, but you can't use this approach from your android application in runtime due to file access restrictions.

You typically do not have a dedicated "VRAM" on mobile devices. At least you don't have it with PowerVR architectures (wich totally dominate the market with their MBX and SGX cores).
That is, the OpenGL driver allocates normal RAM until you run out of it, and the more you allocate the less you have left for your application.

The Android/OpenGL APIs don't offer explicit methods to read the VRAM size from a given device.
Poor man solution:
You could try to infer the VRAM size in an empiric way adding 1MB texture until you get an out of memory error from gl.glGetError().

From your "dmesg" output u can read off the VRAM, so for my Tablet:
> [ 0.000000] Machine: TDM3730 [ 0.000000] Reserving 12582912
> bytes SDRAM for VRAM
>
> 7>[ 3.929962] VRAM: checking region 9f400000 3072
> <4>[ 3.929992] Failed. Allocating 4194304 bytes for fb 0
> <7>[ 3.935333] VRAM: alloc mem type 0 size 4194304 paddr dec2bd4c
> <7>[ 3.935485] VRAM: checking region 9f400000 3072
> <7>[ 3.935485] VRAM: found 9f400000, end a0000000
> <6>[ 3.936584] android_usb gadget: high speed config #1: android
> <4>[ 3.960113] allocating 4194304 bytes for fb 1
or details at:
http://pastebin.com/jQSXQqHh

Is simple just count how many Mb ram that from usable to real capacity of the ram, example for my lenovo a369i has 512 RAM Module, but in setting app only showing 471 Mb usable so the 41Mb left is reserved for the GPU, so the conclusion is my a369i has 41Mb vram
This method is based from shared graphics memory (wiki)

I suspect that android.os.StatFs is what you're looking for:
http://developer.android.com/reference/android/os/StatFs.html