I am writing an Android application that would both store data and communicate with a server using protocol buffers. However, the stock implementation of protocol buffers compiled with the LITE flag (in both the JAR library and the generated .java files) has an overhead of ~30 KB, where the program itself is only ~30 KB. In other words, protocol buffers doubled the program size.
Searching online, I found a reference to an Android specific implementation. Unfortunately, there seems to be no documentation for it, and the code generated from the standard .proto file is incompatible with it. Has anyone used it? How do I generate code from a .proto file for this implementation? Are there any other lightweight alternatives?
I know it's not a direct answer to your question, but an extra 30kb doesn't sound that bad to me. Even on EDGE that'll only take an extra 1 to 2 seconds to download. And memory is tight on android, but not THAT tight -- 30 kb is only about 1/10th of one percent of the available application memory space.
Are there any other lightweight alternatives?
I'm taking this to mean "to using protocol buffers", rather than "for using protocol buffers with an Android application". I apologise if you are already commited to protocol buffers.
This site is about "comparing serialization performance and other aspects of serialization libraries on the JVM". You'll find many alternatives listed there.
While there is no mention of the memory footprint of the different implementations at the moment I am sure it is a metric which the people on the wiki would be interested in.
Just to revive this archaic thread for anyone seeing it, the answer is to use Square's Wire library (https://github.com/square/wire)
As they mention themselves:
Wire messages declare public final fields instead of the usual getter methods. This cuts down on both code generated and code executed. Less code is particularly beneficial for Android programs.
They also internally build using the Lite runtime I believe.
And of course Proguard, the new Android 2.0 minify tools, [other generic answers], etc etc.
Use ProGuard[1] on your project. It will reduce the size of jars included in APK file.
[1] http://developer.android.com/guide/developing/tools/proguard.html
Related
For the past six months as my final university project, I've been writing a PlayStation 1 emulator in Java to prove it can be performant - part of my strategy involves writing a custom class loader that imports bytecode I have just generated from an array into a new class - in effect a Java bytecode dynarec core which speeds up the emulated CPU orders of magnitude (in theory). All quite possible on an Oracle JVM, and done before by others.
My question is, aside from the fact I would need to generate dalvik bytecode rather than Java bytecode, there doesn't seem to be anyway to dynamically load classes into a running Android app that doesn't involve loading them from a dex file on flash somewhere. I know similar things have been asked before, but as I would eventually like to port this emulator (and have it be quicker than its currently unplayable speed), is there anyway around this? I don't want to be continually writing to flash when a new section of MIPS code is converted to bytecode, as it could wear the flash out and probably isn't very fast either.
A thought I had was maybe mounting a tmpfs using a small JNI lib and storing class files there to be loaded, so in effect storing them in RAM as before - is this even possible for an unprivileged app to do though? I'd appreciate peoples input/thoughts.
No, that might be possible on a jailbroken device but it's not possible in a sandboxed app.
I tried several ways to load dynamic code on Android but the only feasible way is via the DexClassLoader where the dex file must be stored in a privileged region.
You can have a look at my project Byte Buddy where I implemented such class loading: https://github.com/raphw/byte-buddy/blob/master/byte-buddy-android/src/main/java/net/bytebuddy/android/AndroidClassLoadingStrategy.java
I'm developing an Android application which contains native code.
The native code is compiled in a .so file that has important algorithms inside.
I'm really worrying about the possibility that my .so file can be edited or modified and then re-build (re-pack). Like apks they can be modified and repacked to create a new one
I have several questions here:
1) Is there any way to edit/modify .so files and re-build?
2) If there are, how do people do that?
3) How to prevent .so files from being edited then re-built?
The short answer is that anything that a computer can read and understand, it can also modify. There is no bullet-proof signature mechanism in Android for Java or native code. Still, the so files are generally considered much less vulnerable than the Java code, even with obfuscation turned on.
Reverse engineering a shared library is hard but possible. Disassembly, change, and assembly back is not hard if one knows what to change.
There are many ways to strengthen protection of your C++ code against reverse engineering, but none will hold against a determined and well-funded attack. So, if the stakes are very high, consider running the important part of your algorithm on your server, and prey for its security.
I want to transform/instrument Dex files. The goals of transformation include measuring code coverage. Note that the source files are not available. So instrumenting Dex is the only option.
I am wondering if there are any existing code base that I could look at as examples to write a tool to achieve my goal.
I know about the Smali project and a host of other projects that build on Smali. However, none of these projects are good examples for my purpose.
I am looking for code that automatically transforms smali code or the dexlib representation, from which smali is generated. The later option is preferred for my purpose because the overhead of generating smali can be avoided.
It's a lot of code, but dx's DexMerger is an example program that transforms dex files. It's made quite complicated by the fact that it needs to guess the size of the output in order make forward-references work.
You'd also need to create infrastructure to rewrite dalvik instructions. DexMerger's InstructionTransformer does a shallow rewrite: it adjusts offsets from one mapping to another. To measure code coverage your instruction rewriting would probably need to be much more sophisticated.
Another option that have become available recently is Dexpler. It is an extension of Soot, which is a framework for analysis and instrumentation of Java programs. Dexpler reads in .apk files and converts to Jimple intermediate format. Jimple code can then be arbitrarily instrumented, and eventually dumped into a new apk.
(For the record, I am answering my own question here)
Eventually I did not find any tool that fit my requirements. So I ended up building my own tool, called Ella, based on DexLib. Out of the box, it does a few things such as measuring code coverage, recording method traces, etc. But it can be easily extended to do other types of transformations.
In some cases smali itself does a small amount of instruction rewriting while re-assembling a dex file. Things like replacing a const-string with a const-string/jumbo, or a goto instruction with a "larger" one, if the target is out of range. This involves replacing instructions in the instruction list with potentially larger ones, and the corresponding fixing up of offsets.
CodeItem.fixInstructions is the method responsible for this.
Additionally, there is the asmdex library. I'm not all that familiar with it, but it sounds like it might be relevant to what you're wanting to do.
I know it's a bit late but just in case you're still interested or perhaps for some other readers. ASMDEX has been mentioned already. And I think that's your best bet for the moment for what you're trying to achieve.
As for adding new registers take a look at org.ow2.asmdex.util.RegisterShiftMethodAdapter class. It's not perfect! As a matter of fact as it is it's horrible changing existing 4bit instructions when adding a register would mean some register would end up being 0xF and won't fit in 4 bits.
But it should be a good start.
I have a lot of data (text format) to send from a device. It obviously means that I should compress it. But my question is whether there are any ways of doing it other than by zip algorithm (like this). The reason I am asking this question is over here - for a text file i.e. 7-zip is twice (!) better than zip. Which is a significant gain. And maybe there are even better algorithms.
So are there any effective ways of data compression (better than zip) available for Android?
You would need to compile another library into your code, since I doubt that compression algorithms other than zlib are available as part of the standard libraries on the Android.
The 7-zip algorithm you refer to is actually called LZMA, which you can get in library form in the LZMA SDK. The source code is available in Java as well as C. If you can link C code into your application, that would be preferable for speed.
Since there's no such thing as a free lunch, the speed is important. LZMA will require much more memory and much more execution time to achieve the improved compression. You should experiment with LZMA and zlib on your data to see where you would like the tradeoff to fall between execution time and compression, both to choose a package and to pick compression levels within a package.
If you find that you'd like to go the other way, to less compression and even higher speed than zlib, you can look at lz4.
Your question is too general.
You can use any library, as long as it is in Java or C/C++ (via the NDK). If you don't want to use external libraries, you have to stick to what's in the SDK. Depending on how you are sending the data, there might be standard ways to do this. For example, HTTP uses gzip and has the necessary headers already defined.
In short, test different things with your expected data format and size, find the best one and integrate it in your app.
sorry if it's a silly question. Do comments in the java or xml file effect the memory usage of the android application? has anyone tried to monitor the memory usage of his/her application with and without the comments?
No, comments do not use any memory.
It's important to understand that, in programming in C, Java, etc. what you're writing is source code which, before being run on the computer (or, specifically, your Android device) is compiled into a machine code format. The processor does not run your source code as you see it. The source code you write typically contains lots of stuff like comments (which do NOT have any effect on the actual code) or perhaps things like compiler directives (which may control how the compiler compiles sections of your code).
(I realise it's more correct to use the term byte code in the case of Java, but trying to keep the answer simple here.)
An exception to this however would be if you're talking about the case where you insert a file (e.g. XML file) as a raw resource within your Android application. But, I think this topic is an advanced one for you to learn about later.
Comments in your code are compiled out and have no effect whatsoever on memory usage in an application.