I would like to understand what is rlimit and pipe limit for an executable?
I am debugging an issue where dynamically built executables are crashing while loaded but statically built executables are fine.
WHen a dynamically built executable is loaded,
I see the following prints after the register dump
Process 2748(ls) has RLIMIT_CORE set to 1
Aborting Core
First of all some one please explain what is rlimit with perspective to exec. Also is there some thing called pipe limit? I see this proc entry.
Please point me to some resource that explains these attributes of an executable.
Question 2 :
When an executable is loaded, what exactly exec does? linker will load the shared libraries that are needed by my executable. So for this to happen linker(a binary inside /system/bin) should execute first.
Hence I understand that linker(system/bin/linker) and executable(ls) both are loaded by exec. Is my understanding correct? This is all theory. If someone, probably who is good at both fs and proc management, could point me to the relevant source, it would be great help. Thank you.
The RLIMIT_CORE is used to place a limit on the amount of info that coredump is allowed to produce, before it is aborted. Once this limit is hit, no more info is logged and the message Aborting Core is logged to the console.
From the man page of core :
A process can set its soft RLIMIT_CORE resource limit to place an upper limit on the size of the core dump file that will be produced if it receives a "core dump" signal.
Use setrlimit() to configure RLIMIT_CORE to a larger value to obtain complete coredumps.
The most common format of executables/shared-objects is ELF. On Linux, the dynamic loading and linking of these shared-objects is performed by ld.so
ld.so is loaded in the address space of a newly created process (by exec in this case) and executed first. This is possible as its path is embedded into the .interp section of the executable binary during compilation.
The dynamic linker then
loads the initial executable image
loads the dependent shared-objects(*.so files)
jumps to the entry-point of the executable
Reference : Dynamic linker on ELF-based Unix-like systems
Related
I am going to learn a little bit about Dalvik VM, dex and Smali.
I have read about smali, but still cannot clearly understand where its place in chain of compilers. And what its purpose.
Here some questions:
As I know, dalvik as other Virtual Machines run bytecode, in case of Android it is dex byte code.
What is smali? Does Android OS or Dalvik Vm work with it directly, or it is just the same dex bytecode but more readable for the human?
Is it something like dissasembler for Windows (like OllyDbg) program executable consist of different machines code (D3 , 5F for example) and there is appropriate assembly command to each machine code, but Dalvik Vm also is software, so smali is readable representation of bytecodes
There is new ART enviroment. Is it still use bytecodes or it executes directly native code?
Thank you in advance.
When you create an application code, the apk file contains a .dex file, which contains binary Dalvik bytecode. This is the format that the platform actually understands. However, it's not easy to read or modify binary code, so there are tools out there to convert to and from a human readable representation. The most common human readable format is known as Smali. This is essentially the same as the dissembler you mentioned.
For example, say you have Java code that does something like
int x = 42
Assuming this is the first variable, then the dex code for the method will most likely contain the hexadecimal sequence
13 00 2A 00
If you run baksmali on it, you'd get a text file containing the line
const/16 v0, 42
Which is obviously a lot more readable then the binary code. But the platform doesn't know anything about smali, it's just a tool to make it easier to work with the bytecode.
Dalvik and ART both take .dex files containing dalvik bytecode. It's completely transparent to the application developer, the only difference is what happens behind the scenes when the application is installed and run.
High level language programming include extra tools to make programming easier & save time for the programmer. After compiling the program, if it was to be decompiled, going back to the original source code would need a lot of code analysis, to determine structure & flow of program code, most likely a few more than 1 pass/parse. Then the decompiler would have to structure the source based on the features of the compiler that compiled the code, the version or the compiler, and the operating system it was compiled on eg. if an OS specific features or frameworks or parsers or external libraries were involved, such as .net or dome.dll, and their versions, etc
The next best result would be to output the whole program flow, as if the source code was written in one large file ie. no separate objects, libraries, dependencies, inheritances, classes or api. This is where the decompiler would spit out code which when compiled, would result in errors since there's no access to the source codes & structure of the other files/dependencies. See example here.
The 3rd & best option would be to follow what the operating system is doing based on the programmed instructions, which would be machine code, or dex (in case of Android). Unless you're sitting in the Nebuchadnezzar captained by Morpheus and don't have time to decode every opcode in the instruction set of the architecture your processor is running, you'd want something more readable than unicode characters scrolling on the screen as you monitor the program flow/execution.
This is where assembly code makes the difference; it's almost the direct translation of machine code, in a human readable format. I say "almost" direct because microprocessors have helpers like microcodes, multithreaders for pipelining & hardware accelerators to give a better user experience.
If you have the source code, you'd be editing in the language the code is written in. Similarly, if you don't have the source code, and you're editing the compiled app, you'd still be editing in the language the code is written in; in this case, it's machine code, or the next best thing: smali.
Here's a diagram to illustrate "Dalvik VM, dex and Smali" and "its place in chain of compilers".
during the execution of th MLO I create a variable, whose value I want
to make accessible to user space applications in Android. How can this
be achieved?
One way would be to write the contents of the variable to external
memory and let it read by the user space process. However, I would need
to make sure that during boot no other process is overwriting the address.
Do you know of any other ways, ATAGs? If ATAGs can be used, how would one do this? Is it necessary to develop a kernel module?
Cheers
From linux userspace, you can get info from U-Boot environment variables using "fw_printenv" application. During U-Boot execution you would "setenv variablename value", then saveenv.
Your U-Boot MLO would need CONFIG options set to enable the env commands. MLO usually wants (and needs) small code footprint, env commands will make bigger code footprint, that could be an obstacle.
At the linux side, you would need "fw_printenv" configured for your particular target's memory. That can be done at runtime, see fw_env.config. You can get the target executable built in u-boot/tools/env/. This assumes that android carries over the linux mechanisms in this area; I am not familiar with android platform details.
How can I get the trace of dalvik bytecode that is being executed while the device is running an app (like logcat does)?
There is no existing tool to do what you want, but it might be possible to implement such a tool without too much work.
The general idea is that you want to connect to dalvik's vm as a debugger, using the jdwp protocol, and single step the virtual machine, inspecting the state between each step and printing out the instruction. You should be able to get the... instruction index. Or maybe the instruction offset, via JDWP.
Additionally, you will need to read the dex file, and look up the exact instruction being executed, based on the instruction index/offset you get from JDWP.
You should be able to use AndBug to implement the first part of the above solution, and then use dexlib to access the dex file, in order to get the instruction.
You might want to check out traceview
It is realy too slow to trace the method not to speak the byte code,when you want to know what the app is doing the best way is to trace method first,then trace the byte code in it.
Android stores it's programs in APK format, which is a modified version of ZIP/JAR.
When these APK files are installed, they are stored in /system/app/$APKNAME.apk.
Some of the apps in this dir also have a $APKNAME.obex file.
These APK files contain some of the fallowing
META-INF
MANIFEST.MF
CERT.RSA
CERT.SF
SHA1-Digest
res
AndroidManifest.xml
classes.dex
resources.arsc
So what I want to know is what are the .obex files and are androids program decompressed from the APK/ZIP/JAR at runtime and how?
The way that this works is pretty interesting, and gives some key insights into Android's runtime model. The first thing I'd recommend watching is Dalvik VM internals, if you plan on doing any serious amount of systems stuff with Android. (Although, it's obviously old.) Now, when the Android package manager gets an intent which requires starting a new app it forks off a new virtual machine from an already running zygote process. This is basically a technique which allows the system to get a lot of nice memory properties (sharing the pages mapped, etc..). Then, the system loads up a (potentially pre optimized and verified) file to load so the vm can start executing it. You should read this document, which will tell you quite a bit about how this works. (Perhaps this thread will also help.) Keep in mind that as all systems are different -- for example, if you're on a new architecture, you won't get JIT support unless you explicitly write it! -- you can't know for sure how Dalvik will load code to run your app.
I started my emulator with ./emulator -trace profile -avd emulator_15. I then tracked down the trace files to ~/.android/avd/rodgers_emulator_15.avd/traces/profile, where there are six files: qtrace.bb, qtrace.exc, qtrace.insn, qtrace.method, qtrace.pid, qtrace.static. I can't figure out what to do with these files. I've tried both dmtracedump and traceview on all of the files, but none seem to generate any output I can do anything with.
How can I view the proportion of time taken by native method calls on Android?
You need to use tracedmdump to convert the output. This is a shell function defined in build/envsetup.sh in the full Android sources. If you're using the SDK, rather than building from a full tree, I'm not sure this will work.
(If you don't have the sources and want to take a peek at the tracedmdump function, you can see it here.)
If you used emulator -trace profile, you'd run tracedmdump profile. This will dig through various binaries to retrieve symbolic information and associate it with the trace data, generating an HTML summary and a traceview-compatible trace file.
It's worth noting that the VM will execute more slowly with profiling enabled (the interpreter has overhead on every method call and return, and it's running in the slower "debug" interpreter), while native code continues to run at full speed, so you have to be careful when drawing conclusions.
General comment: don't forget to use F9 or one of the method calls to start/stop the tracing -- the -trace flag just enables the feature.
In order to use those six files, there are other scripts in the same directory as that of dmtracedump such as read_pid, read_trace, profile_trace etc. U should first run post_trace on the trace directory containing the six files, then you can use any one of them to get profile info such as how often each basic block executes, the pids they belong to etc.