Develop Reference

Can low memory cause seg faults in native code?

I have a group of crashes in native code that are rare but happen consistently inolving SEGV_MAPERR or SEGV_ACCERR. These crashes are almost always reported by Crashlytics with very low RAM free (1-5% typically). 'Normal' crashes (ie, ones I have debugged) have no pattern in RAM free.
Is it possible these crashes are caused by a low memory condition? What would be the mechanism for this? Is there any way to tell if these are low memory related crashes or programming errors (using pointers wrongly, etc)? In many cases, the crash is happening in a library which I can't debug and I can't replicate the crashes on my devices.
Here's some of these crashes pulled from the Developer Console because it provides a little more detail than Crashlytics in the trace in these cases:
********** Crash dump: **********
Build fingerprint: 'htc/a32eul_metropcs_us/htc_a32eul:5.1/LMY47O/637541.3:user/release-keys'
pid: 10902, tid: 10989, name: .xxx.xxxx >>> com.xxx.xxxxx <<<
signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x97f78000
Stack frame #00 pc 0004cd80 /data/app/xxx.xxx.xxxxx-1/lib/arm/libxxx.so: Routine xxxxxMixerInterleavedFloatOutput at libgcc2.c:?
********** Crash dump: **********
Build fingerprint: 'Xiaomi/land/land:6.0.1/MMB29M/V8.1.1.0.MALMIDI:user/release-keys'
pid: 2661, tid: 2746, name: .xxx.xxxx >>> com.xxx.xxxx <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
Stack frame #00 pc 00016954 /system/lib/libc.so (__memcpy_base+36)
Stack frame #01 pc 0000b14c /data/app/com.xxx.xxxx-2/lib/arm/libswresample-2.so: Routine ??
??:0

There are two general possibilities:
A low memory condition in of itself is not going to somehow trigger a segfault in a running application. What can happen is that when the application asks for additional memory to be allocated to it, the memory allocation request fails. This is a well defined memory condition. It is documented that the relevant system calls can fail in allocating memory. But what often happens is that the application are not coded properly to check for a failed memory allocation request, and they crash for that reason. In that case, it is not true that a low memory condition is responsible for an application segfault, it is an application bug.
The Linux kernel overcommits the available memory. As a result of that it is possible that the kernel will have no option but to select a process to be killed, when all available RAM has been exhausted.
However, in the case of the OOM killer kicking in, the chosen victims are terminated with a SIGKILL. A SEGFAULT indicates an application bug.

What is EGL and GL mtrack in Android Memory Dump

I'm currently working on a hybrid application that is exhibiting some peculiar memory usage that I'm trying to debug. As soon as the application starts, it appears to be immediately using close to 250MB of memory, which seems excessively given that all we have loaded at that point is the login screen. I've been looking at a few memory profiling tools for Android (the Xamarin profiler and the android profiler shipped with the SDK), but all of them show relatively low heap usage (~10-15MB, which I'm trying to get down). I ran "adb shell dumpsys meminfo APPNAME -d' and got the following trace:
** MEMINFO in pid 24925 [APPNAME] **
Pss Private Private Swapped Heap Heap Heap
Total Dirty Clean Dirty Size Alloc Free
------ ------ ------ ------ ------ ------ ------
Native Heap 19439 19396 0 0 28672 22915 5756
Dalvik Heap 15441 14992 0 0 37319 36837 482
Dalvik Other 542 368 0 0
Stack 432 432 0 0
Ashmem 17388 16508 880 0
Gfx dev 40538 34504 0 0
Other dev 4 0 4 0
.so mmap 6211 224 3080 0
.apk mmap 10531 0 10232 0
.ttf mmap 453 0 260 0
.dex mmap 1263 0 980 0
.oat mmap 635 0 152 0
.art mmap 707 516 24 0
Other mmap 452 4 52 0
EGL mtrack 63508 63508 0 0
GL mtrack 79116 79116 0 0
Unknown 21756 21756 0 0
TOTAL 278416 251324 15664 0 65991 59752 6238
I've been trying to understand what this means by using the documentation provided at: https://developer.android.com/tools/debugging/debugging-memory.html, but that page doesn't seem to have any info on the biggest culprits: Gfx dev, EGL mtrack, GL mtrack, and Unknown. Is there some documentation on what these categories are or why they would grow to be so big?
Thanks

EGL and GL shows here memory consumed by Graphics layer basically. I am sure you are running adb shell dumpsys meminfo command on Android lollipop device. Actually dumpsys meminfo tool/command has been modified in lollipop to calculate and display graphics memory.
In old version (KitKat or older) you can not find EGL and GL information although graphics consumed memory in KitKat or older versions also.
This is we can request Google to update their documentation to explain new memory components also. In short you can say it is bug in documentation of Android. They should update it as per latest implementation of adb shell dumpsys meminfo tool/command.
References:
EGL - http://en.wikipedia.org/wiki/EGL_(API)
GL - http://en.wikipedia.org/wiki/OpenGL

GL mtrack is driver-reported GL memory usage. It's primarily the sum of GL texture sizes, GL command buffers, fixed global driver RAM overheads, etc.
EGL mtrack is gralloc memory usage. It's primarily the sum of the SurfaceView/TextureView.

Heap Corruption - SEGV_MAPERR in Android Native code

I am trying to create a small library for stream AES encryption, I started my work based on Facebook Conceal project (https://github.com/facebook/conceal), just changing some things and improving the wrapper around the native to support ciphers with padding.
It is working and it can decipher files without problems but I get random Heap Memory Corruptions when I work with large streams, and after a lot of time debugging I have been unable to find the error.
Here is my code:
https://gist.github.com/frisco82/9782725
I have tried to find memory allocation or free problems but there are almost no malloc or free, and jni call should be safe, the same goes for openssl ones (I have compiled my own but conceal provided ones also fail)
CheckJni does not warn about anything and while the context handling is a bit out of the box it doesn't seem broken (indeed Android conscrypt seems to use something similar).
Also if someone can point me to a Android native AES multistep (multiple update calls) library I will switch to that and forget this.
The error varies from time to time but it is usually similar to his:
03-26 10:33:02.065: A/dalvikvm(2475): ### ABORTING: DALVIK: HEAP MEMORY CORRUPTION IN mspace_malloc addr=0x0
03-26 10:33:02.065: A/libc(2475): Fatal signal 11 (SIGSEGV) at 0xdeadbaad (code=1), thread 2494 (AsyncTask #1)
03-26 10:33:02.205: I/DEBUG(933): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
03-26 10:33:02.205: I/DEBUG(933): Build fingerprint: 'generic_x86/google_sdk_x86/generic_x86:4.4.2/KK/999428:eng/test-keys'
03-26 10:33:02.205: I/DEBUG(933): Revision: '0'
03-26 10:33:02.205: I/DEBUG(933): pid: 2475, tid: 2494, name: AsyncTask #1 >>> com.proton <<<
03-26 10:33:02.205: I/DEBUG(933): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr deadbaad
Full stack traces:
http://pastebin.com/f6mDuQEj

It is working and it can decipher files without problems but I get
random Heap Memory Corruptions when I work with large streams.
From above line it looks to me that your program is clearly overwriting the memory which was allocated implicitly or explicitly by your code. I was trying to understand your code however it was not clear to me. But I tried to look from memory corruption scenario and found that your program does have malloc/free call which might lead to memory overrun.
EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX*) malloc(sizeof(EVP_CIPHER_CTX));
EVP_CIPHER_CTX_init(ctx);
EVP_CIPHER_CTX *ctx = (EVP_CIPHER_CTX*) malloc(sizeof(EVP_CIPHER_CTX));
EVP_CIPHER_CTX_init(ctx);
I tried to check the layout of the EVP_CIPHER_CTX structure but it was not available in your code. But I saw that these pointers are getting used in various context within your program. Now you should check that under which scenario your buffer can be overwritten as some places you have used different keyLength and depending on this your program is executing different function. I think you may want to review these codes and see whether overflow is possible!!!....
As your application would be running on android based system where we can not run any dynamic tool(Valgrind/WinDBG/Pageheap..) so I guess you need to review your code by putting some log at important place and see where you are overwriting.
Hope above information would be useful for you to understand your problem.

After all I was able to work around this problem, EVP_CipherUpdate (or jni ReleaseByteArrayElements) sometimes overflow the output buffer causing the heap corruption, nothing in my code was wrong and also it was not a problem with the caller as replacing EVP_CipherUpdate with a memcpy call with the same parameters worked as expected and there was no heap corruption.
So the solution was adding some extra length to the output buffer sent to nativeUpdate and the error was gone.
I have made the full working version of the library for others to use at:
https://github.com/frisco82/conceal

query about "dumpsys meminfo" in android

When I run "adb shell dumpsys meminfo" on android 4.2, I get the result like
Shared Private Heap Heap Heap
Pss Dirty Dirty Size Alloc Free
------ ------ ------ ------ ------ ------
Native 28 8 28 16404 12256 3663
Dalvik 14062 10060 13736 20032 15254 4778
Cursor 0 0 0
Ashmem 0 0 0
Other dev 4762 9556 0
.so mmap 11699 1824 1500
.jar mmap 0 0 0
.apk mmap 368 0 0
.ttf mmap 811 0 0
.dex mmap 3736 0 0
Other mmap 114 16 32
Unknown 12064 544 12052
TOTAL 47644 22008 27348 36436 27510 8441
I have read the page of How do I discover memory usage of my application in Android?, but still have several questions:
Why the native Pss, shared dirty private dirty is very small?
the heap size should be smaller than Pss?
What does Unknown mean? Seems very big.
If I want to know how much memory my app uses, which data should I use? The Total Pss? But it doesn't include the native Pss which is nealy zero?

Please refer to the question How do I discover memory usage of my application in Android?.
I would also like you to refer to Detail VSS,RSS,PSS,USS link. The most appropriate data to use for a particular app is
USS (Unique Set Size ) as
USS is the total private memory for a process, i.e. that memory that is completely unique to that process . USS is an extremely useful number because it indicates the true incremental cost of running a particular process. When a process is killed, the USS is the total memory that is actually returned to the system. USS is the best number to watch when initially suspicious of memory leaks in a process.
which you can get by adb shell procrank | grep <your.package.name>

Troubleshooting android app memory usage problems

I would like to know how I can troubleshoot high memory usage problems of my app on Android. I've search the internet and found out that the DDMS plugin is useful in taking a memory dump of the heap for my app. This however has been useless.
The app "Usage Timelines Free" is showing 94 MBs of memory used, while the DDMS heap dump shows me a total of 8.4 MBs, with the suspected leaks being the resource files.
When I generate a dump from adb (dumpsys meminfo), I get:
Shared Private Heap Heap Heap
Pss Dirty Dirty Size Alloc Free
------ ------ ------ ------ ------ ------
Native 20 8 20 54588 39431 1892
Dalvik 6732 9952 6396 10756 10028 728
Cursor 0 0 0
Ashmem 184 0 184
Other dev 11462 1172 11384
.so mmap 2467 2072 1156
.jar mmap 0 0 0
.apk mmap 48 0 0
.ttf mmap 2 0 0
.dex mmap 1037 0 0
Other mmap 41 16 32
Unknown 46352 292 46348
TOTAL 68345 13512 65520 65344 49459 2620
How can I know what is behind this huge memory consumption. My app is a foreground monitor service which runs forever, with a few activities which are accessed a few times per day.
Thanks.

Minimize your android data usage. Check the link :http://engineroots.games4punk.com/minimize-your-android-data-usage/

DDMS and got a heap memory dump. It only showed 8.4 MBs being used!
Then you are only using 8.4MB of heap space after a complete garbage collection (a net effect of creating the heap dump).
Note that if you ran this on Android 1.x/2.x, MAT will not report space consumed by bitmaps. Always try to dump the heap from an Android 3.0+ device or emulator.
When I generate a dump from adb (dumpsys meminfo), I get:
My guess is that you are running this on an Android 1.x/2.x device or emulator, and you have a lot of bitmap memory. Try running your heap dump and other tests on an Android 3.0+ environment.
Also, bear in mind that adb dumpsys meminfo does not perform a complete garbage collection, whereas dumping the HPROF file does. The Android garbage collector is optimized to minimize CPU utilization and therefore does not attempt to perform a complete garbage collection. Hence, at runtime, the heap is usually filled with garbage that will be reclaimed, as needed, by the GC engine.
You might also wish to read Dianne Hackborn's essay on this subject.