Well,
since I'm interested in reengineering I spend a lot of time on Android reengineering so far.
Nevertheless I got to a point, where I had the problem of compiled, binary C-Code (NDK) and I got to know that it's very difficult to decompile it back to C/C++ than decompiling a DEX-file back to more or less well Java sources.
What's the reason for this? I mean the bytecode is executed by the Dalvik VM and in case of a usual binary file it's executed by the real processor directly instead. Both are pretty similar except for some additional emulation layers, isn't it? I don't see that much differences at the moment or the reason for this problem.
Do you have any information for me why it's more difficult to decompile a usual binary file (e.g. ELF or MS EXE) back to the C source?
Thanks.
The short answer is that the C/C++ code does not contain any reflective information in it and C/C++ has inline functions, macros, and unrolled loops that the Java compiler just doesn't do (as much as C/C++ compilers do). It is also possible to optimize C/C++ so extensively that all you can do is decompile to assembly because there are no references to the applications own functions. (References to the system's functions will be found though.)
BTW, Hex-Rays ARM Decompiler makes reverse-engineering job much easier, check this out: http://www.hex-rays.com/hexarm_compare0.shtml
The other question is that it costs much...
Related
What are the implications for converting a Java algorithm to NDK in order to protect it? This Java algorithm takes data as input and outputs processed data. That's it. It does not interact with the UI in any way.
Will the converted algorithm cause compatibility issues on some of the Android devices?
Since the algorithm is converted to c++, does that mean it cannot be decompiled but can only be disassembled?
Is there any tool that allows me to convert this Java algorithm to NDK without me writing the algorithm myself?
Thank you.
The difference is that Java code is compiled into byte code, which is then run on the JVM, while C/C++ code is compiled into machine code. I don't think it's reasonable to convert Java code to C/C++ because of security reasons, it's mostly done because the code written in C/C++ is much faster.
As far as I know preventing decompilation is one of the main reasons to use NDK along with improving performance as mentioned by Andrej. When it comes to your questions:
1) You will have to make sure you compile your native code for all five supported ABIs, i.e. x86, x86_64, armeabi, armeabi-v7a and arm64-v8a.
2) Nothing is 100% impossible to decompile, but chances that your native code will be successfully decompiled are a lot lower than decompiling java bytecode.
3) No you have to write C or C++ code yourself. However, headers containing the API you want to expose to Java can be generated by javah.
Android Studio makes it quite easy to write small native libraries and integrate them into existing projects, you need only very basic knowledge of CMake.
According to wiki (http://en.wikipedia.org/wiki/Android_Runtime) Dalvik gets entirely replaced by ART in Lollipop i.e. from that release onwards any app will be compiled to native code upon installation. This begs the question, is there a point in writing computing intense routines in NDK if the app will be compiled to native code anyway?
The Dalvik VM also compiled code to native code. The difference is that Dalvik did it "just in time", and only for the parts of code that were executed frequently.
The compiler in Art has a number of performance improvements over the one in Dalvik, but if you felt the need to go native for performance before, you will most likely continue to feel that need.
ART does not make pure "native code" in the sense of C language etc. It is still bytecode generated from Java sources.
So yes, there is still a lot of advantages to write some routines with the NDK, of course :)
we all know that google is shifting from Dalvik to ART ( 4.4.2 has even experimental support too) for obvious reason, A very basic question is arising in my mind after i recently started working on mdifying the generated smali files for some purpose. Here is my question:
After expanding any apk file we see dalvik executable and various smali file containing dalvik opcodes and other related things. So i was surprised if this structure inside the apk file will be containing the similar things going forward once Google forces to use ART i.e. I am not clear if modifying the smali file will be valid once ART takes over.
This may be very basic question for many but is my first question in this forum so please excuse me for my limited knowledge on this topic.
Forenote: This is an extension of the thread started on /r/haskell
Lets start with the facts:
Android is one awesome Operating System
Haskell is the best programming language on the planet
Therefore, clearly, combining them would make Android development that much better. So essentially I would just like to know how I can write Haskell programs for the Android OS. My question is:
How can I get a Haskell program to execute/run on the Android OS?
How you do it is by first getting a Haskell compiler which can target C with the android NDK which comes with a GCC port for ARM architectures. JHC can trivially do this with a very small inf style file which describes the platform (word size, c-compiler, etc) I've done this with the Wii homebrew dev kit and it was quite easy. However jhc still has some stability issues with complex code such as using a monad transformer stack with IO but jhc has been improving a lot over the last 6 months. There is only one person working on JHC I just wished more people could help him.
The other option is to build an "unregistered" port of GHC targeting the ndk gcc, this is a lot more involved process because GHC is not a true cross-compiler at the moment and you need to understand the build system what parts you need to change. Another option is NHC which can cross-compile to C, like GHC you need to build nhc targeting a C compiler, NHC does not have many Haskell extensions like GHC.
Once you have Haskell compiler targeting NDK GCC, you will need to write bindings to either the android NDK JNI glue code framework (added since android 2.3) or you must write JNI glue code between Java-C-Haskell, the former option is the easier solution and if I remember correctly might actually be backwards compatible with previous versions of Android below 2.3.
Once you have this you must build Haskell code as shared library or static library which gets linked into the NDK java glue code (which itself is a shared library). As far as I'm aware you can not officially run native executables on android. You could probably do it with a rooted phone, thus I assume this means you can not distribute native executables on the app store even when the NDK gcc port can generate native executables just fine. This also probably kills the option for using LLVM unless you can get the NDK JNI working with LLVM.
The biggest hurdle isn't so much of getting a Haskell compiler for android (which is still a big hurdle) the biggest problem is that some one needs to write binding APIs for NDK libraries which is a huge task and the situation is worse if you need to write android UI code because there are no NDK APIs for this part of the android SDK. If you want to do android UI code in Haskell somebody will have to write Haskell bindings to Java through JNI/C. Unless there is a more automated process to writing binding libraries (I know there are some, they are just not automated enough for me) then chances of some one doing it are quite low.
L01man: Is there a tutorial about how to do this? For the
first part, I understand I have to download JHC. What do I have to
write in the inf file and how to use it?
Please note before I answer this question I haven't used jhc for quite sometime since I originally wrote this and newer versions have been released since so I do not know how stable jhc currently is when it comes to code generation of more complex Haskell programs. This is a warning to anyone before you consider making a large Haskell program with JHC, you should do some small tests before you go full on.
jhc does have a manual http://repetae.net/computer/jhc/manual.html and a section on setting-up cross-compilation and .ini file with options: http://repetae.net/computer/jhc/manual.html#crosscompilation.
L01man: The second part is an alternative to the first. I don't know how to do what you said in the
third.
Before you begin you should have some knowledge of C and be comfortable with using the Haskell foreign function interface (FFI) and tools such as hs2c. You should also be familiar with using the Android NDK and building .apk with shared libraries. You will need to know these to interface between C-Haskell, Java/C-Haskell and develop Haskell programs for Android that you can officially distribute/sell on the market store.
L01man: I understand that its goal is to create a binding for the
Android API. But... does the 4th part says we can't make .apk with
Haskell?
.apk is just an app package file format and is built with the tools that come with the Android SDK (not NDK), this has very little to do building the binaries itself. Android packages can contain native shared libraries, this what your Haskell program will be and the native shared/static libraries are generated via the Android NDK.
A language that has recently come to my attention is Eta.
Eta's compiler is a fork of GHC 7.10 which has a JVM backend. It is possible to use the generated JAR files to write Android apps and even use its Foreign Function Interface to call native Android Java libraries.
Brian McKenna has written a blog post about how to configure an Android Studio project to use an Eta library.
There is https://github.com/neurocyte/android-haskell-activity demonstrating Haskell code running.
I once came across the same Reddit thread, but it was old, and comments were closed. I sent a message to the OP, but am not sure whether it reached the recipient. My suggestion here (may work for older Androids where native activities were not possible).
I (developed in Haskell some time ago, but currently switched to Smalltalk) am currently developing a port of Squeak VM to Android. The way I am doing this is similar to what might be dealt with in a haskell-on-android project: a lump of C code which needs to be called from Java part of the application (basically all that can be done in Android is to handle various events; an application cannot poll for events itself and does not have any event loop). In my case the code is generated by the Squeak VM building tools, in the case of haskell on android this will be output from GHC of JHC or whatever front end used. This repo may be worth looking at:
http://gitorious.org/~golubovsky/cogvm/dmg-blessed/trees/master/platforms/android/project
Under "src" there is the Java code which provides for user events interception and sending them to the native code (see the CogView class). The C code of the VM itself is not completely there (see squeakvm.org, the Cog branch for that), but one may get the idea. One also might look under http://gitorious.org/~golubovsky/cogvm/dmg-blessed/trees/master/platforms/android/vm which is the C frontend to the interpreter (including user events handling, some timekeeping, etc.)
Hope this helps.
Dmitry
There is https://github.com/conscell/hugs-android a port of HUGS Haskell interpreter to Android.
I think the general answer should come from source->source transformations, since loading specially compiled shared objects seems to be a bit of a kludge (involving ghc->c and a c->java step in the answers above). This question thus falls under the heading of Haskell on the JVM, which has been tried (with one step as a Java intermediate representation) and discussed at length. You could use frege if the libraries you need compile there. The only remaining steps would be the beginnings of the Android framework API translated into IO() actions and maybe a wrapper for building the manifest xml and apk.
I just read that Android has a 450% performance improvement because it added a JIT compiler, I know what JIT is, but I don't really understand why is it faster than normal compiled code? or what's the difference with the older approach from the Android platform (the Java like run compiled bytecode).
Thanks!
EDIT: This is hugely interesting, thanks!, I wish I could pick every answer as correct :)
First a disclaimer, I'm not at all familiar with Android. Anyway...
There are two applications of JIT compilation that I am familiar with. One is to convert from byte-codes into the actual machine instructions. The second is Superoptimisation.
JIT bytecode compilation speeds things up because the bytecodes are only interpeted once, instead of each time it is executed. This is probably the sort of optimisation you are seeing.
JIT superoptimsation, which searches for the truly optimal set of instructions to implement a programs logic, is a little more esoteric. It's probably not what your talking about, though I have read reports of 100% - 200% speed increases as a result.
The VM needs to turn compiled byte code into machine instructions to run. Previously this was done using an interpreter which is fine for code that is only invoked once but is suboptimal for functions that are called repeatedly.
The Java VM saw similar speedups when asa JIT-versions of the VM replaced the initial interpreter versions.
The JIT compiler knows about it's system, it can use that knownledge to produce highly efficient code compared to bytecode, and rumors go it can surpass pre-compiled programs.
That's why it can go faster than the traditional system of java, where the code was run as bytecode only, which Android used, too.
Besides compiling java code to native code, which could be done with a compiler too, a JIT does optimizations, that you can only do at runtime.
A JIT can monitor the applications behavior over time and optimize those usage patterns that really make a difference, even at the expense of other branches in the execution path of the code, if those are less frequently used.