I'm parsing a big XML file using SAXParser in Android and was wondering if there's a faster way of doing string comparisons? I've heard rumours that you can use the Dalvik VM to do something which will save memory allocated and hence speed it all up, but I can't find anything online.
Can anyone point me in the right direction to either use Dalvik to speed up my parsing or a better, faster way of doing XML parsing?
String's compareTo() method is implemented internally in dalvik as a special hand-crafted assembly routine. It's unlikely you'll be able to beat its performance for string comparisons
If this is an xml file that you are downloading, I don't know of any way to speed up your parsing time. However, if you can include it in the apk as a resource (in the res/xml folder), then it will be compiled into Android's binary xml format when you build the apk, and you can access it via XmlResourceParser, which should be significantly faster
I would use the goold old String.compareTo(String).
A faster way may exist right now, but compareTo() is the standard way, and is therefore the only that will benefit of future optimizations made by Android. So at the end an optimized comparison may become slower than the standard way to do.
This is quite similar to the member fields access time. At the beginning of Android it was way faster to do
final AnyClass local = mMyMember;
local.something();
than
mMyMember.something();
But this is no longer the case.
Related
For the past six months as my final university project, I've been writing a PlayStation 1 emulator in Java to prove it can be performant - part of my strategy involves writing a custom class loader that imports bytecode I have just generated from an array into a new class - in effect a Java bytecode dynarec core which speeds up the emulated CPU orders of magnitude (in theory). All quite possible on an Oracle JVM, and done before by others.
My question is, aside from the fact I would need to generate dalvik bytecode rather than Java bytecode, there doesn't seem to be anyway to dynamically load classes into a running Android app that doesn't involve loading them from a dex file on flash somewhere. I know similar things have been asked before, but as I would eventually like to port this emulator (and have it be quicker than its currently unplayable speed), is there anyway around this? I don't want to be continually writing to flash when a new section of MIPS code is converted to bytecode, as it could wear the flash out and probably isn't very fast either.
A thought I had was maybe mounting a tmpfs using a small JNI lib and storing class files there to be loaded, so in effect storing them in RAM as before - is this even possible for an unprivileged app to do though? I'd appreciate peoples input/thoughts.
No, that might be possible on a jailbroken device but it's not possible in a sandboxed app.
I tried several ways to load dynamic code on Android but the only feasible way is via the DexClassLoader where the dex file must be stored in a privileged region.
You can have a look at my project Byte Buddy where I implemented such class loading: https://github.com/raphw/byte-buddy/blob/master/byte-buddy-android/src/main/java/net/bytebuddy/android/AndroidClassLoadingStrategy.java
I have 9000 files in the memory card, I have created an application that read each file and parse it in to a String and put that String in a HashMap in order to avoid repetition,and save the names of only unrepeated files, but that operation is taking about 7 to 10 seconds.. is there any way to make that faster.. I tried to find better method but I didn't find anything...
Object : get the names and Strings of all the unrepeated files..
Is there any way to do that faster using java, new IO, or NDK, is it worth to install NDK and try that using C language and pointers..
I appreciate any help..
Its obviously that even just loading of 9k files takes some time. Imagine there a lot of android devices and there are many devices which are not as fast as device which you are using to test. So the operation gonna take even more time on such devices. The problem is a numerous of small files. Maybe you should pack it in one, lets say a JSON or XML? Reading of 1 file is more faster and reliable.
I want to transform/instrument Dex files. The goals of transformation include measuring code coverage. Note that the source files are not available. So instrumenting Dex is the only option.
I am wondering if there are any existing code base that I could look at as examples to write a tool to achieve my goal.
I know about the Smali project and a host of other projects that build on Smali. However, none of these projects are good examples for my purpose.
I am looking for code that automatically transforms smali code or the dexlib representation, from which smali is generated. The later option is preferred for my purpose because the overhead of generating smali can be avoided.
It's a lot of code, but dx's DexMerger is an example program that transforms dex files. It's made quite complicated by the fact that it needs to guess the size of the output in order make forward-references work.
You'd also need to create infrastructure to rewrite dalvik instructions. DexMerger's InstructionTransformer does a shallow rewrite: it adjusts offsets from one mapping to another. To measure code coverage your instruction rewriting would probably need to be much more sophisticated.
Another option that have become available recently is Dexpler. It is an extension of Soot, which is a framework for analysis and instrumentation of Java programs. Dexpler reads in .apk files and converts to Jimple intermediate format. Jimple code can then be arbitrarily instrumented, and eventually dumped into a new apk.
(For the record, I am answering my own question here)
Eventually I did not find any tool that fit my requirements. So I ended up building my own tool, called Ella, based on DexLib. Out of the box, it does a few things such as measuring code coverage, recording method traces, etc. But it can be easily extended to do other types of transformations.
In some cases smali itself does a small amount of instruction rewriting while re-assembling a dex file. Things like replacing a const-string with a const-string/jumbo, or a goto instruction with a "larger" one, if the target is out of range. This involves replacing instructions in the instruction list with potentially larger ones, and the corresponding fixing up of offsets.
CodeItem.fixInstructions is the method responsible for this.
Additionally, there is the asmdex library. I'm not all that familiar with it, but it sounds like it might be relevant to what you're wanting to do.
I know it's a bit late but just in case you're still interested or perhaps for some other readers. ASMDEX has been mentioned already. And I think that's your best bet for the moment for what you're trying to achieve.
As for adding new registers take a look at org.ow2.asmdex.util.RegisterShiftMethodAdapter class. It's not perfect! As a matter of fact as it is it's horrible changing existing 4bit instructions when adding a register would mean some register would end up being 0xF and won't fit in 4 bits.
But it should be a good start.
I have a lot of data (text format) to send from a device. It obviously means that I should compress it. But my question is whether there are any ways of doing it other than by zip algorithm (like this). The reason I am asking this question is over here - for a text file i.e. 7-zip is twice (!) better than zip. Which is a significant gain. And maybe there are even better algorithms.
So are there any effective ways of data compression (better than zip) available for Android?
You would need to compile another library into your code, since I doubt that compression algorithms other than zlib are available as part of the standard libraries on the Android.
The 7-zip algorithm you refer to is actually called LZMA, which you can get in library form in the LZMA SDK. The source code is available in Java as well as C. If you can link C code into your application, that would be preferable for speed.
Since there's no such thing as a free lunch, the speed is important. LZMA will require much more memory and much more execution time to achieve the improved compression. You should experiment with LZMA and zlib on your data to see where you would like the tradeoff to fall between execution time and compression, both to choose a package and to pick compression levels within a package.
If you find that you'd like to go the other way, to less compression and even higher speed than zlib, you can look at lz4.
Your question is too general.
You can use any library, as long as it is in Java or C/C++ (via the NDK). If you don't want to use external libraries, you have to stick to what's in the SDK. Depending on how you are sending the data, there might be standard ways to do this. For example, HTTP uses gzip and has the necessary headers already defined.
In short, test different things with your expected data format and size, find the best one and integrate it in your app.
I am using Html.fromHtml(STRING).toString() to convert a string that may or may not have html and/or html entities in it, to a plain text string.
This is pretty slow, I think my last calculation was that it took about 22ms on avg. With a large batch of these it can add over a minute. So I am looking for a faster, performance built option.
Is there anyway to speed this up or are there other decoding options available?
Edit: Since there doesn't appear to be a built in method that is faster or built for performance specifically, I will reward the bounty to anyone that can point me in the direction of a library that:
Works well with Android
Licensed for free use
Faster than Html.fromHtml(String).toString();
As a note, I already tried Jsoup with this method: Jsoup.parse(String).text() and it was slower.
What about org.apache.commons.lang.StringEscapeUtils's unescapeHtml(). The library is available on Apache site.
(EDIT: June 2019 - See the comments below for updates about the library)
fromHtml() does not have a high-performance HTML parser, and I have no idea how quick the toString() implementation on SpannedString is. I doubt either were designed for your scenario.
Ideally, the strings are clean before they get to a low-power phone. Either clean them up in the build process (for resources/assets), or clean them up on a server (before you download them).
If, for whatever reason, you absolutely need to clean them up on the device, you can perhaps use the NDK to create a C/C++ library that does the cleaning for you faster.
This is an incredibly fast and simple option:
Unbescape
It greatly improved our parsing performance which requires every string to be run through a decoder.
Have you looked at Strip HTML from Text JavaScript
With a large batch of these it can add
over a minute
Any parsing will take some time. 22ms seems to me like fast.
Anyway, can you do it in background? Can help you some kind of caching?
Although I have not tried them yet, I found some possible solutions:
HTML Java Parsers
HTML Parsing
More HTML Parsing
I hope it helps.