Since I am using many dependencies in my app, I am reaching the 65k Method Limit (I am reaching 76k methods). I've read on android.developer that proguard is used to shrink the code.
So - does proguard only shrink my application code or does it shrink the code of my dependencies too? Do I need to be wary of something when shrinking code with proguard? How do I do that?
My Gradle Build:
apply plugin: 'com.android.application'
android {
compileSdkVersion 21
buildToolsVersion "21.1.2"
defaultConfig {
applicationId "some.Path"
minSdkVersion 15
targetSdkVersion 21
versionCode 1
versionName "1.0"
}
packagingOptions {
exclude 'META-INF/DEPENDENCIES'
exclude 'META-INF/NOTICE'
exclude 'META-INF/NOTICE.txt'
exclude 'META-INF/LICENSE'
exclude 'META-INF/LICENSE.txt'
}
buildTypes {
release {
minifyEnabled true
shrinkResources true
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
debug {
debuggable true
minifyEnabled true
shrinkResources true
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
}
configurations {
compile.exclude group: 'org.apache.xmlbeans'
}
repositories {
maven { url "https://jitpack.io" }
}
dependencies {
compile fileTree(include: ['*.jar'], dir: 'libs')
compile 'com.android.support:appcompat-v7:21.0.3'
compile 'com.github.PhilJay:MPAndroidChart:v2.1.0'
compile 'com.opencsv:opencsv:3.4'
compile 'org.apache.poi:poi:3.12'
compile 'org.apache.poi:poi-ooxml:3.12'
}
TL; DR: invert your -keep option unless you love troubles
Firstly: I believe, that you are making right choice by using Proguard to overcome the dex limitation. I would not recommend using multidex support library under any circumstances: it introduces problem of multiple classloaders in your application, and that may backfire in many non-obvious ways.
Here is my personal approach to shrinking the app efficiently:
Pick a couple of hugest third-party dependencies you have;
Check if those really support Proguard;
If they do, shrink them with Proguard;
If you still don't fit in maximum method count, do steps above for some of remaining dependencies;
If you still don't fit, possibly reevaluate some, that do not support Proguard, possibly read their source code to get better idea why they don't, and apply Proguard to them yourself;
In the worst case, apply Proguard to your own code;
If absolutely nothing of above helps, use multidex.
Picking dependencies for shrinking
In your case, there aren't many (direct) dependencies in the first place. You may want to look at output of gradlew dependencies to get better idea of your indirect dependencies, some of which may be biggest contributors to total app size. Then you may proceed to use some of tools listed in "Dex" section of Android Arsenal to learn which libraries contribute most to dex method count. You seem to already have a general idea of it, so I won't dwell much on this part.
Remember: shrinking executable code is somewhat non-trivial intervention in library internals, so you'd rather shrink less to avoid mysterious problems in future. If in doubt, start from libraries, that openly declare, that they do support Proguard officially (in your case that would be Android Support libraries).
Note, that "supporting Proguard" may mean different things for different developers. You can expect Android Support Library developers to be at least basically competent, but many others will ship with consumer Proguard rules like this:
-keep class com.example.library.** { *; }
In case you wonder, the above config is based upon many real-life configs, such as Square's Leak Canary Proguard configuration. It does not say anything about overall competency of developers in question, just reminder that using Proguard can be hard. And yes, this kind of configuration will completely prevent shrinking and obfuscation of the library, unless you build it's local copy from source code and remove such helpful consumer-proguard-rules.pro from there.
Evaluating dependencies for Proguard
As shown above, even experienced developers sometimes choose to ignore Proguard. If Google searches regarding the library and it's compatibility with Proguard return nothing (and even if they do return some results!) you may have to make your own judgement regarding usage of Proguard. Here is how I personally do:
If there are words "framework", "enterprise", "reflection" anywhere on the library site, it is likely to be poorly compatible with Proguard;
If the library has anything to do with compile-time code generation (a-la Butterknife, Dagger etc.), think twice before using Proguard;
If the library messes with JNI, think a couple more times before using Proguard on it, and Google for it's effects on Proguard even if you don't shrink the library itself;
If in doubt, Google for it and/or read library source code: usage of Class.forName as well as Proxy.getInvocationHandler and similar reflection code are usual bad signs.
Libraries, that offer Android UI components (such as MPAndroidChart) are usually ok to shrink, at least if you keep getDefaultProguardFile('proguard-android.txt') in your Gradle config.
The most important part
A lot of developers (including Proguard developers themselves!) will offer you a misguided recommendation to start from empty Proguard config + default Android Proguard configuration, and eventually add -keep rules when necessary.
DO NOT DO THAT!!
Those advices come from people, who are either too badass to understand problem of average developer (read: "the Proguard developer himself") or don't have a clue about using Proguard properly. In fact, these kind of misguided practices are the very reason, why many answers to this question warn you against using Proguard: it's default behavior is like suggesting someone to start mountaineering from scaling the Everest.
Default Proguard configuration will obfuscate, shrink and optimize everything—your entire application with all dependencies except some classes you explicitly exclude. You don't want that, unless you have absolute understanding of every library and line of code in your projects: how they work and interact with each other, which techniques they internally use etc.
Instead you want to do the minimal necessary intervention (shrinking the code to reduce dex method count) in the minimal possible range (few hugest libraries) with minimal consequences (only where Proguard is known to work for sure). Here is my Proguard config for such cases:
-dontoptimize
-dontobfuscate
# Prints some helpful hints, always add this option
-verbose
-keepattributes SourceFile,LineNumberTable,Exceptions,InnerClasses,Signature,Deprecated,*Annotation*,EnclosingMethod
# add all known-to-be-safely-shrinkable classes to the beginning of line below
-keep class !com.android.support.**,!com.google.android.**,** { *; }
Add the above rules to your app's proguard-rules.pro, they will shrink only classes, that you explicitly allow to shrink. Append wildcards for other safely shrinkable packages (exactly as above—with ! and .** parts) to beginning of the -keep line.
As an alternative to ProGuard you could use the built-in Gradle shrinker by turning off ProGuard but still reference a ProGuard config. This will remove unused code but not obfuscate or do any other "magic". Although recommended only for Debug builds I don't see why you can't use it for Release builds as well if you don't think that you need obfuscation.
The main benefit, compared to ProGuard (in my opinion) is that you avoid tight coupling between a ProGuard configuration and the structure of your codebase and third party dependencies.
build.gradle:
minifyEnabled true
useProguard false
proguardFiles ('proguard-basic.pro', getDefaultProguardFile('proguard-android.txt'))
proguard-basic.pro:
-dontwarn javax.**
-keep class com.mycompany.** { *; }
If you enable minification via ProGuard, it will also minify your dependencies.
Libraries are typically not already obfuscated/minified with ProGuard. Some libraries will not work properly by default if they are obfuscated, so you should check any libraries you use to see if they have any documentation surrounding ProGuard. Butterknife, for example, has a few special ProGuard rules that you need to include to ensure that it continues working properly.
For me you should rather look for multidex, to go beyond 65k limit, not proguard as in longer run the later is not a solution to your problems.
See docs: https://developer.android.com/tools/building/multidex.html
If you enable minification in your build.grade file, then yes it will also shrink your dependencies.
Keep in mind that Proguard may introduce unwanted side effects. Not all libraries/dependencies can be shrunk as Proguard also obfuscates the code. (i.e. turns String name into String n) and removes unused code.
Take a look at this Github project: https://github.com/krschultz/android-proguard-snippets
As an alternative, you can look into using MultiDex. You can read about it here: https://developer.android.com/tools/building/multidex.html
As per the new Android Studio update in version 3.2 new code shrinker that also obfuscates by adding the line below to your project’s gradle.properties file
Add This line:
android.enableR8 = true
Related
I started working in an Android application and realized that it implemented so many dependencies in the gradle file. I've been able to clean it a little bit and the weight and number of methods has decreased a lot. Good news!
Currently the only thing that worries me is this code line...
implementation 'com.android.support:design:28.0.0'
...because we only use one of its feature in one screen, TabLayout (it seems really necessary for the statistics screen).
Is there any way for implementing just this feature with gradle or proguard?
Thank you very much.
If you use ProGuard, all unused code / resources will be stripped out automatically.
Google's libraries (e.g. com.android.support:design) have ProGuard configs that will do this when it is enabled.
Enabling it just requires the following in your app-level build.gradle, and running a release build:
buildTypes {
release {
minifyEnabled true
proguardFiles getDefaultProguardFile('proguard-android.txt'),
'proguard-rules.pro'
}
}
If you need further help setting up ProGuard, I suggest the official docs.
I want to use the facebook sdk just for analytics, is there an optimized proguard configuration that I could use to strip out the rest?
When it comes down to huge size, it is mostly about the resources rather than worrying about minimising java code itself. So you could actually try out a few things mentioned below.
Proguard works on the Java code. Unfortunately, it doesn’t work on the resources folder. As a consequence, if an image my_image in res/drawable is not used, Proguard only strips it’s reference in the R class but keeps the associated image in place.
Lint is a static code analyzer that helps you to detect all unused resources with a simple call to ./gradlew lint. It generates an HTML-report and gives you the exhaustive list of resources that look unused under the “UnusedResources: Unused resources” section. It is safe to remove these resources as long as you don’t access them through reflection in your code.
However Lint can tell you where are unused resources, but with fb sdk it will be hard to delete the resources as it comes from maven repository.
Minimising Resource Configuration (build.gradle)
For eg Fb sdk provides support all languages which you might not need, or all folders images like mdpi which may not be useful for you.
defaultConfig {
resConfigs "en", "de", "fr", "it"
resConfigs "nodpi", "hdpi", "xhdpi", "xxhdpi", "xxxhdpi"
}
If all of this doesn't work that means either native code is bloating your apk, where application binary interface split might help reducing your apk size.
ABI Split:-
splits {
density {
enable true
reset()
include "ldpi", "mdpi"
}
abi {
// Enables building multiple APKs per ABI.
enable true
// By default all ABIs are included, so use reset() and include to specify that we only
// want APKs for x86, armeabi-v7a, and mips.
// Resets the list of ABIs that Gradle should create APKs for to none.
reset()
// Specifies a list of ABIs that Gradle should create APKs for.
include "x86", "armeabi-v7a", "mips"
// Specifies that we do not want to also generate a universal APK that includes all ABIs.
universalApk false
}
}
I think something could be done here as I opened facebook sdk gradle file ... it has few transitive dependencies, which is redundant and might conflict with your support version so you could either import the same in your files
dependencies {
// Facebook Dependencies
compile 'com.android.support:support-v4:25.3.1'
compile 'com.android.support:appcompat-v7:25.3.1'
compile 'com.android.support:cardview-v7:25.3.1'
compile 'com.android.support:customtabs:25.3.1'}
It could be removed from final fat jar as you might be already using the support dependencies in your project that too different or conflicting ones .. so you could ideally exclude transitive dependencies based on your requirements something like below
compile ('com.facebook.android:facebook-android-sdk:4.+') {
exclude group: 'com.android.support' //by group
}
My question was wrong, there is no way to strip the code in proguard, probably it can be done playing with exclusions in gradle, but I do not think is doable using just proguard
In any case facebook finally has modularised their sdk, so for people like me that are insterested just on analytics, facebook-core is the only depedency we need to import
The ProGuard config included in the Facebook SDK has been updated and so it will correctly remove all classes that are not being used by your app (e.g. everything not in Analytics).
... except for anything Serializable which Facebook feels it is unsafe to remove ...
It now looks a lot like the older answers around here:
-keepclassmembers class * implements java.io.Serializable {
private static final java.io.ObjectStreamField[] serialPersistentFields;
private void writeObject(java.io.ObjectOutputStream);
private void readObject(java.io.ObjectInputStream);
java.lang.Object writeReplace();
java.lang.Object readResolve();
}
(as of 20 May 2016)
Which means that we don't need our own ProGuard settings for Facebook anymore. Gradle will automatically use the rules supplied by the SDK.
Of course, this still requires that your build.gradle is configured to run ProGuard:
minifyEnabled true
shrinkResources true
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules-release.pro'
-keep class com.facebook.** {
*;
}
-keepattributes Signature
I use these 2 lines and working everything fine
This might be too broad, but I would like an explanation on how Proguard and minification configurations are passed between projects and their dependencies to understand how deeply are these operations made in my project's dependency tree.
I have on build.gradle of `themodule':
buildTypes {
release {
minifyEnabled true
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
dependencies {
compile project(':someothermodule')
compile 'some.maven.central.library'
}
By the configuration it seems clear that the classes inside themodule will be minifyed and obfuscated, but what happens with the classes of someothermodule? Will they also be minifyed and obfuscated? Even if someothermodule has minifyEnabled true?
What happens if 'someothermodule' is just a .jar dependency?
What happens with the configurations of some.maven.central.library?
Are the Proguard configurations of the module being built cascading down to its dependencies or each of them follows its own rules?
If a module get obfuscated (minifyEnabled true) by itself, the used configuration is not automatically inherited by the consuming module (in your case the application).
There is a mechanism in the Android gradle plugin to enabled this:
consumerProguardFiles 'proguard-rules.pro'
The rules that are contained in proguard-rules.pro will be automatically included in the application project and merged with other configuration files.
This will only work for Android library projects (.aar). If you have a dependency to a .jar file on maven central for example, no such consumer rules will be available, and you have to add the needed configuration to your application project yourself.
Keep in mind that the configuration to obfuscate a module and the one used by the consuming application / module does not need to be identical. The consumer rules will in most cases only be a set of -keep rules.
Technically, it is the following :
Library projects by themselves don't run ProGuard, so they don't use
any configuration.
Application projects obfuscate the entire code base, including any
referenced libraries, so they need proper configuration for the
application code and for the library code.
I had a small case where I had a Facebook library as a gradle dependency and since we were obfuscating the code with minifyEnabled:true we had to keep all its code from being obfuscated, using the regular keep commmands such as :
-keep class com.facebook.** { *; }
Additionally, and regarding the .jar obfuscation, you can check this other post
Regards,
I am getting the following error when I compile my app:
[2014-05-07 21:48:42 - Dex Loader] Unable to execute dex: Cannot merge new index 65536 into a non-jumbo instruction!
I am at the point that if I declare a new method anywhere in my package, I get this error. If I don't, the app compiles.
I would like to know what exactly (and accurately) does this error mean. My app is big, but I don't think its that big! So:
Does the error mean I have too many methods? public? static? package? members?
Is it related to the methods/members of my root package, or also to the included JAR libraries?
Is there a way to get more debug information about this?
I already know about that "jumbo" enabling flag addressed in the similar questions here in SO, however, I think jumbo mode is not available on the API level I'm targeting (ICS).
Your error is for the amount of strings (methods, members, etc) in a single dex file.
You need to compile you app using jumbo in dex with:
dex.force.jumbo=true
in project.properties
This increment the limit for strings in a dex files. And your project will probably compile.
Also with jumbo set, the is another limit of 64K only for methods in an single dex. If you get this limit in the future , you will need to remove some dependencies.
UPDATE: for build with Gradle:
In Gradle you can enable jumboMode also in the build.gradle file with:
dexOptions {
jumboMode = true
}
Check:
Android Build: Dex Jumbo Mode in Gradle
Also with Gradle you can avoid the 64K limit for methods using multidex build, tutorial here:
https://developer.android.com/tools/building/multidex.html
For gradle build, just add the dexOptions into build.gradle to enable jumbo mode:
android {
dexOptions {
jumboMode = true
}
}
Remember to run "gradle clean" before your new building.
It's related to the number of methods of libraries included in the project. For example if you have tracking in your app, just Google Analytics is ~7000 methods.
In one of my projects using Lombok (2MB of JAR) gave me these problem. Solved getting rid of this library.
It looks like the problem occurs because all the class files from your project and JAR files are packed together before DEXing. This may not be completely true but any way of controlling this in our project has proven to be quite difficult. Even removing stuff that initially caused this problem, cleaning and rebuilding didn't fix the issue for us in a consistent way.
So we took this opportunity to switch our project to Android Studio and managed to solve the problem by turning on ProGuard for debug builds as well. More precisely we only use the shrink phase of the ProGuard's processing chain.
Gradle makes it very easy to turn on ProGuard for debug builds:
buildTypes {
debug {
runProguard true
proguardFile 'proguard-project-debug.txt'
}
}
And here is the debug ProGuard config we use:
-keep class com.your.code.**
# Use -keep to explicitly keep any other classes shrinking would remove
-dontoptimize
-dontobfuscate
-ignorewarnings
This does increase the build time of the project but the good side is that the debugger still works.
The only faster alternative I can think of is that any JAR files are manually stripped of the unused class files. But this is not only difficult to do it is also inconvenient when you want to use a slightly larger part of a library at a later time.
I hope this helps other developers struggling with this issue. And perhaps in the future Google can improve the compiler that does this pruning by default. Our APK DEX file went from 8MB to 2.9MB.
Newer gradle (1.0.0+) versions
In newer Versions of Android studio (1.0+) the bundled Gradle got updated. There were some changes on how the build mechanism works so your project Gradle file can now take advantage of the minifyEnabled and shrinkResources parameters. Current version is 1.1.0.
Keeping up with changes on a fast moving platform like Android takes effort but it is often rewarded with new features, tools and faster build times. So updating Android Studio and (carefully) updating your projects is worth the time you invest.
buildTypes {
debug {
proguardFile 'proguard-project-debug.txt'
minifyEnabled true
shrinkResources true
}
}
Some interesting observations. Same error may appear if you have multi-flavor project. It's confusing. Turned out that I attempted run app with generic command: gradlew installDebug. When I've changed command line to look like this problem is gone. Don't forget to replace Flavor part with your actual one.
gradlew installFlavorDebug
I'm developing a big application how has many libraries, I reached the 65k methods limit and I want to clean some libraries to shrink the methods size. Now I'm using proguard, but this take 5 minutes to compile! so, I need a method to clean before proguard (like exclude with build.gradle or something like that)
I'm trying to use something like this :
compile ('com.google.android.gms:play-services:4.3.+'){
exclude group: 'com.google.android.gms.drive'
exclude group: 'com.google.android.gms.games'
}
but it doesn't works , any idea?
If you disable ProGuard's optimization step (-dontoptimize) and obfuscation step (-dontobfuscate), ProGuard may be fast enough for debug builds. The shrinking step can go a long way in reducing the number of methods.
Could you be running into GRADLE-2964 which occurs because Gradle is not using Zip64? This is fixed, according to jira, so maybe it works with a snapshot of Gradle.
Example using snapshot of Gradle 1.12:
gradle-wrapper.properties:
distributionUrl=http\://services.gradle.org/distributions-snapshots/gradle-1.12-20140327133732+0000-bin.zip
Then in whatever task that is zipping (extends from ZipTask) slap on this:
{
....
zip64 = true
....
}