I've been doing Android development for a little bit and I'm getting to a point in one of my projects where I would like to use Proguard to shrink the size of my APk and help with the dex limit. Unfortunately, I am getting a few errors and stack overflow has answers but they seem to be targeted for those with more experience.
My question is what is the relationship with your proguard-android.txt and proguard-rules.pro? Why are there two separate files and why are they in separate formats? When are the statements in these files called and in what order? I am just looking for an explanation of the overall context of using Progurad in a development environment.
Thank you in advance.
ProGuard manipulates Java bytecode the way you tell it with your configuration files and the rules they contain. ProGuard can do many things. And it can completely break your app so you have to make sure to add the correct rules.
I assume you use Gradle based builds for your apps. Then you've probably encountered this snippet that enables ProGuard for release builds of your app (or Android library):
android {
buildTypes {
release {
minifyEnabled true
proguardFiles getDefaultProguardFile(‘proguard-android.txt'),
'proguard-rules.pro'
}
}
...
}
In the config the list proguardFiles tells the build what files that contain ProGuard rules it has to use. This list can contain any number of files.
Why are the files (proguard-android.txt and proguard-rules.pro) defined differently?
The magical getDefaultProguardFile(‘proguard-android.txt') loads file named proguard-android.txt from the standard location in the Android SDK (the location is ${ANDROID_SDK}/tools/proguard/).
Other config files are resolved locally, so file proguard-rules.pro is expected to be at the root of the current Gradle module.
Why are there two separate files? And what is the relationship between proguard-android.txt and proguard-rules.pro?
ProGuard configuration is additive. You can define some rules in one file and other in other files. The rules are internally concatenated into single list of rules.
File getDefaultProguardFile(‘proguard-android.txt') contains several general rules for all Android apps (check them yourself, in the file in your SDK). The local proguard-rules.pro is expected to contain rules specific for your own app. For example you want to make sure that a class is not stripped away when you use it only through reflection (I'll get to that later).
Note that having multiple local files is very useful. For example you can use two local config files for debug builds - one with the release rules for your app and the second containing rules disabling obfuscation.
Also note that the additive behaviour of the configurations can be a bit troubling. If you add a rule in one config file, you cannot remove it in another. So be careful with very general rules (e.g. imagine adding -keep class ** { *; }).
When are the statements in these files called and in what order?
You can define them in any order, there's no difference. And you can define the same rule in multiple files, it doesn't matter. The order of the specified files doesn't matter either.
ProGuard itself is run as a single job within the Android build (single Gradle task to be precise). The task is provided all the inputs:
classes to manipulate
library classes to use but not manipulate
output path for generated processed jar
ProGuard rules specifying the manipulation
output paths for various output information (what was removed, mapping, …)
And then it processes the files and generates an output which is further processed by the Gradle build.
How does ProGuard actually work? And why do I need the rules?
ProGuard traverses the whole call graph of classes/methods/fields/…. It starts with the classes/methods/… defined by the provided rules. Then traverses the call graph and marks classes/methods/fields/… as necessary and keeps them for the output. So if you call it with no matching keep rules it will generate an empty output (or maybe it will throw an error and tell you to define some, I don't remember now). ProGuard doesn't recognize calls done via reflection, so you have to add some rules to handle that. There are many other cases that require you to add some rules, check the documentation for that.
Final notes
If you check ProGuard documentation you can find various rules
you can use. But not all of the rules are good for Android (ProGuard is a general Java tool).
Some rules are generated by Android build itself, you don't have to define them yourself. There are 2 types of such rules:
General config rules like -injars, -libraryjars, …
Rules generated from AndroidManifest.xml and resources (layouts). Android build (aapt tool) generates rules to keep classes mentioned in the manifest (activities, services, receivers, …) and custom views used in layouts. You can check these generated rules in build/intermediates/proguard-rules/${PRODUCT_FLAVOR}/${BUILD_TYPE}/aapt_rules.txt
Some rules can come from aar libraries. The libraries can contain ProGuard config necessary for the library to work (there can be proguard.txt file inside).
When writing Android libraries yourself be extremely careful with the rules you want to add to the aar. Because of the additive nature of the rules, it can cause problems for the app that bundles the library.
Related
I'm creating an app for android and ios, and i already know that it's theoretically possible to decompile an android app. The app contains sensitive information that i don't want users to have access to as the app interfaces with a webserver. If a user gained access to some information available in the source code, they could potentially spam my web server with requests.
Is there any way to authenticate a connection between the app and the server, assuming that the source code is accessible, or is there any way to obfuscate my code to prevent a malicious user from spamming my webserver.
Thankss
[UPDATE]
**
When you build your application using Android gradle plugin version > 3.4.0, the plugin chooses R8 to optimize and obfuscate the code. The rules can now be configured on proguard-rules.pro or proguard-app.conf files. the rules to indicate what to exclude from the obfuscation are similar to the ones in proguard.cfg used earlier.
You can import your proguard files in your build.gradle like
buildTypes{
...
release{
proguardFiles getDefaultProguardFile(
'proguard-android-optimize.txt'),
'proguard-rules.pro'
}
}
R8 picks up all the existing proguard rules files as long as they're included in the build.gradle. You can also configure what pieces to obfuscate for different product flavors that you may have.
**
[OLD BUT RELEVANT INFO]
Proguard is a tool that will help you obfusate your code. This comes as part of your android tools and you just need to activate it. This link and this will help further.
Proguard's default configuration (in proguard.cfg) will be enough to sufficiently obfuscate your code. However you might want to tweak your proguard configuration when you have methods/classes that are being dynamically accessed.
For instance, accessing classes/methods with Reflection will need you to have the code to be intact. You might sometimes experience ClassNotFoundException if proguard obfuscates it.
If you have classes that are being accessed in the AndroidManifest/ Layout Files, you should prevent proguard from obfuscating them.
This can be done by adding
-keep public class <MyPackage.MyClass>
to your proguard.cfg.
**
While Proguard makes static analysis harder, DexGuard protects from both static and dynamic analysis. DexGuard is specifially for android applications and is only commercially available while Proguard is open source and is for any java bytecode obfuscation / optimization.
You cannot prevent decompiling android apk, you can just increase the difficulty of decompilation, proguard is the best option.
DexGuard provides even better security then ProGuard but it is NOT free: https://www.saikoa.com/dexguard
DexGuard can even obfuscate String constants.
As instant app features are compiled into seperate APKs the proguard step becomes a little bit more tricky.
For example you can no longer just have one proguard file defined in the runnable module and let it obsfuscate all features. Every feature must have its own proguard.
I would like to know how you can easily define one proguard file and use some gradle magic to apply them to all feature modules?
Having multiple proguard files in each module is just unmaintainable.
I have enabled Proguard by uncommenting the following line in project.properties file:
proguard.config=${sdk.dir}/tools/proguard/proguard-android.txt:proguard-project.txt
When I tried to extract the classes and resources using dex2jar , I was able to extract them as it is. Do I have to change some other files / properties / configurations in my project to prevent direct extraction of classes? Should I add some configuration parameters in the project.properties(project root location) or proguard.android file(SDK location)?
It seems you're using the "standard" Proguard obfuscator properties file. Try using the "advanced" Proguard obfuscator properties file. It's here:
proguard.config=${sdk.dir}/tools/proguard/proguard-android-optimize.txt
Your codes will be obfuscated a bit further, but not all. For example, activities, services, Java classes you've declared in the manifest file, or declared with -keep directives in Proguard, will not be completely obfuscated.
On a side note: Even if you use the commercial DexGuard, you're still not 100% protected from decompilation. Experienced hackers can use smali/baksmali techniques to reverse engineer your codes, read it, modify it at will. Remember, if codes can be read as 0s and 1s, it can be hacked.
Did this occur after you built your application in release mode and exported it using valid certificate and the Export Wizard in Eclipse ? If no - do the steps: right click on your project, choose "Export", type "export android application", Next, then choose "Use existing keystore" or "Create new keystore", finish the wizard. ProGuard runs only when you build your application in release mode.
There is no way to prevent the decompilation of classes and resources. Obfuscation, such as proguard, are tools to make reverse engineering of java harder through name munging, string munging, and flow control changing. Though as stated before, everything can always be reverse engineered. Obfuscation is used to make the barrier to entry higher, and to deter people from wanting to reverse engineer your code due to level of effort.
History/Context
I have a project[1] where size really matters - recently I moved stuff to a shared lib[2] and thought proguard will take care and remove the unused classes because I had a config that was drastically reducing the size but by using the lib i came over the magic 100kb mark so I investigated: classes which I do not use for sure are in the resulting dex file - and even with full name ( not shortened to single-char ) - e.g. I see the SquareView in the dex which I in no way use in the App.
Question
Surprisingly I found in the proguard documentation the following:
The library jars themselves always remain unchanged.
Can I somehow tell/trick proguard (in)to process them? I find this really strange especially because I expect more stuff to be removeable in the lib than in the App itself..
[1] https://github.com/ligi/FAST
[2] https://github.com/ligi/AndroidHelper
The Eclipse/Ant/Gradle build processes in the Android SDK automatically specify your code (from bin/classes) and its libraries (from libs) with the option -injars. This means that the complete application is compacted, optimized, and obfuscated (in release builds, assuming ProGuard is enabled).
The build processes only specify the Android runtime android.jar with the option -libraryjars. It is necessary to process the code, but it should not end up in the processed apk, since it is already present on the device.
So it should all work out automatically. You may still see entire libraries with their original names in processed apks, if your configuration proguard-project.txt contains lines like -keep class org.mylibrary.** { *; }. Such configuration is typically a conservative solution to account for reflection. With some research and experimentation, you can often refine the configuration and get better results. You can figure out why classes are being kept with the option -whyareyoukeeping.
I believe you have to use -injars:
-injars class_path
Specifies the input jars (or wars, ears, zips, or directories) of the application to be processed. The class files in these jars will be
processed and written to the output jars. By default, any non-class
files will be copied without changes. Please be aware of any temporary
files (e.g. created by IDEs), especially if you are reading your input
files straight from directories. The entries in the class path can be
filtered, as explained in the filters section. For better readability,
class path entries can be specified using multiple -injars options.
Source: http://proguard.sourceforge.net/index.html#manual/usage.html
In the case, if I will
build a project
clean up all binaries
build it again (no source/resources and etc has changed).
Does Proguard guarantee to provide the same mapping.txt file?
ProGuard is deterministic: for the same input, it will generate the same output.
There is one subtlety though: if the operating system lists input files in a directory (notably class files that are not inside an archive) in a different order, then they may be processed in a different order, and the output can be different.
It might actually happen, but i don't think proguard guarantees that.
I found this in the Proguard documentation that will allow you to reuse your mapping.txt to avoid changes on the mappings
-applymapping filename
Specifies to reuse the given name mapping that was printed out in a previous obfuscation run of ProGuard. Classes and class members that are listed in the mapping file receive the names specified along with them. Classes and class members that are not mentioned receive new names. The mapping may refer to input classes as well as library classes. This option can be useful for incremental obfuscation, i.e. processing add-ons or small patches to an existing piece of code. If the structure of the code changes fundamentally, ProGuard may print out warnings that applying a mapping is causing conflicts. You may be able to reduce this risk by specifying the option -useuniqueclassmembernames in both obfuscation runs. Only a single mapping file is allowed. Only applicable when obfuscating.
If you want a guarantee then you have to use the mappings file as input to the obfuscation process. But then you carefully have to check all warnings about conflicts relating to that mapping file. If you ignore that, you may get subtle errors, when working with reflection.