How to shrink code - 65k method limit in dex - android

I have a rather large Android app that relies on many library projects. The Android compiler has a limitation of 65536 methods per .dex file and I am surpassing that number.
There are basically two paths you can choose (at least that I know of) when you hit the method limit.
1) Shrink your code
2) Build multiple dex files (see this blog post)
I looked into both and tried to find out what was causing my method count to go so high. The Google Drive API takes the biggest chunk with the Guava dependency at over 12,000. Total libs for Drive API v2 reach over 23,000!
My question I guess is, what do you think I should do? Should I remove Google Drive integration as a feature of my app? Is there a way to shrink the API down (yes, I use proguard)? Should I go the multiple dex route (which looks rather painful, especially dealing with third party APIs)?

It looks like Google has finally implementing a workaround/fix for surpassing the 65K method limit of dex files.
About the 65K Reference Limit
Android application (APK) files contain
executable bytecode files in the form of Dalvik Executable (DEX)
files, which contain the compiled code used to run your app. The
Dalvik Executable specification limits the total number of methods
that can be referenced within a single DEX file to 65,536, including
Android framework methods, library methods, and methods in your own
code. Getting past this limit requires that you configure your app
build process to generate more than one DEX file, known as a multidex
configuration.
Multidex support prior to Android 5.0
Versions of the platform prior to Android 5.0 use the Dalvik runtime
for executing app code. By default, Dalvik limits apps to a single
classes.dex bytecode file per APK. In order to get around this
limitation, you can use the multidex support library, which becomes
part of the primary DEX file of your app and then manages access to
the additional DEX files and the code they contain.
Multidex support for Android 5.0 and higher
Android 5.0 and higher uses a runtime called ART which natively
supports loading multiple dex files from application APK files. ART
performs pre-compilation at application install time which scans for
classes(..N).dex files and compiles them into a single .oat file for
execution by the Android device. For more information on the Android
5.0 runtime, see Introducing ART.
See: Building Apps with Over 65K Methods
Multidex Support Library
This library provides support for building
apps with multiple Dalvik Executable (DEX) files. Apps that reference
more than 65536 methods are required to use multidex configurations.
For more information about using multidex, see Building Apps with Over
65K Methods.
This library is located in the /extras/android/support/multidex/
directory after you download the Android Support Libraries. The
library does not contain user interface resources. To include it in
your application project, follow the instructions for Adding libraries
without resources.
The Gradle build script dependency identifier for this library is as
follows:
com.android.support:multidex:1.0.+ This dependency notation specifies
the release version 1.0.0 or higher.
You should still avoid hitting the 65K method limit by actively using proguard and reviewing your dependencies.

you can use the multidex support library for that, To enable multidex
1) include it in dependencies:
dependencies {
...
compile 'com.android.support:multidex:1.0.0'
}
2) Enable it in your app:
defaultConfig {
...
minSdkVersion 14
targetSdkVersion 21
....
multiDexEnabled true
}
3) if you have a application class for your app then Override the attachBaseContext method like this:
package ....;
...
import android.support.multidex.MultiDex;
public class MyApplication extends Application {
....
#Override
protected void attachBaseContext(Context context) {
super.attachBaseContext(context);
MultiDex.install(this);
}
}
4) if you don't have a application class for your application then register android.support.multidex.MultiDexApplication as your application in your manifest file. like this:
<application
...
android:name="android.support.multidex.MultiDexApplication">
...
</application>
and it should work fine!

Play Services 6.5+ helps:
http://android-developers.blogspot.com/2014/12/google-play-services-and-dex-method.html
"Starting with version 6.5, of Google Play services, you’ll be able to
pick from a number of individual APIs, and you can see"
...
"this will transitively include the ‘base’ libraries, which are used
across all APIs."
This is good news, for a simple game for example you probably only need the base, games and maybe drive.
"The complete list of API names is below. More details can be found on
the Android Developer site.:
com.google.android.gms:play-services-base:6.5.87
com.google.android.gms:play-services-ads:6.5.87
com.google.android.gms:play-services-appindexing:6.5.87
com.google.android.gms:play-services-maps:6.5.87
com.google.android.gms:play-services-location:6.5.87
com.google.android.gms:play-services-fitness:6.5.87
com.google.android.gms:play-services-panorama:6.5.87
com.google.android.gms:play-services-drive:6.5.87
com.google.android.gms:play-services-games:6.5.87
com.google.android.gms:play-services-wallet:6.5.87
com.google.android.gms:play-services-identity:6.5.87
com.google.android.gms:play-services-cast:6.5.87
com.google.android.gms:play-services-plus:6.5.87
com.google.android.gms:play-services-appstate:6.5.87
com.google.android.gms:play-services-wearable:6.5.87
com.google.android.gms:play-services-all-wear:6.5.87

In versions of Google Play services prior to 6.5, you had to compile the entire package of APIs into your app. In some cases, doing so made it more difficult to keep the number of methods in your app (including framework APIs, library methods, and your own code) under the 65,536 limit.
From version 6.5, you can instead selectively compile Google Play service APIs into your app. For example, to include only the Google Fit and Android Wear APIs, replace the following line in your build.gradle file:
compile 'com.google.android.gms:play-services:6.5.87'
with these lines:
compile 'com.google.android.gms:play-services-fitness:6.5.87'
compile 'com.google.android.gms:play-services-wearable:6.5.87'
for more reference, you can click here

Use proguard to lighten your apk as methods that are unused will not be in your final build. Double check you have following in your proguard config file to use proguard with guava (my apologies if you already have this, it wasn't known at time of writing) :
# Guava exclusions (http://code.google.com/p/guava-libraries/wiki/UsingProGuardWithGuava)
-dontwarn sun.misc.Unsafe
-dontwarn com.google.common.collect.MinMaxPriorityQueue
-keepclasseswithmembers public class * {
public static void main(java.lang.String[]);
}
# Guava depends on the annotation and inject packages for its annotations, keep them both
-keep public class javax.annotation.**
-keep public class javax.inject.**
In addition, if you are using ActionbarSherlock, switching to the v7 appcompat support library will also reduce your method count by a lot (based on personal experience). Instructions are located :
http://developer.android.com/tools/support-library/features.html#v7-appcompat Actionbar
http://developer.android.com/tools/support-library/setup.html#libs-with-res

You could use Jar Jar Links to shrink huge external libraries like Google Play Services (16K methods!)
In your case you will just rip everything from Google Play Services jar except common internal and drive sub-packages.

For Eclipse users not using Gradle, there are tools that will break down the Google Play Services jar and rebuild it with only the parts you want.
I use strip_play_services.sh by dextorer.
It can be difficult to know exactly which services to include because there are some internal dependencies but you can start small and add to the configuration if it turns out that needed things are missing.

I think that in the long run breaking your app in multiple dex would be the best way.

Multi-dex support is going to be the official solution for this issue. See my answer here for the details.

If not to use multidex which making build process very slow.
You can do the following.
As yahska mentioned use specific google play service library.
For most cases only this is needed.
compile 'com.google.android.gms:play-services-base:6.5.+'
Here is all available packages Selectively compiling APIs into your executable
If this will be not enough you can use gradle script. Put this code in file 'strip_play_services.gradle'
def toCamelCase(String string) {
String result = ""
string.findAll("[^\\W]+") { String word ->
result += word.capitalize()
}
return result
}
afterEvaluate { project ->
Configuration runtimeConfiguration = project.configurations.getByName('compile')
println runtimeConfiguration
ResolutionResult resolution = runtimeConfiguration.incoming.resolutionResult
// Forces resolve of configuration
ModuleVersionIdentifier module = resolution.getAllComponents().find {
it.moduleVersion.name.equals("play-services")
}.moduleVersion
def playServicesLibName = toCamelCase("${module.group} ${module.name} ${module.version}")
String prepareTaskName = "prepare${playServicesLibName}Library"
File playServiceRootFolder = project.tasks.find { it.name.equals(prepareTaskName) }.explodedDir
def tmpDir = new File(project.buildDir, 'intermediates/tmp')
tmpDir.mkdirs()
def libFile = new File(tmpDir, "${playServicesLibName}.marker")
def strippedClassFileName = "${playServicesLibName}.jar"
def classesStrippedJar = new File(tmpDir, strippedClassFileName)
def packageToExclude = ["com/google/ads/**",
"com/google/android/gms/actions/**",
"com/google/android/gms/ads/**",
// "com/google/android/gms/analytics/**",
"com/google/android/gms/appindexing/**",
"com/google/android/gms/appstate/**",
"com/google/android/gms/auth/**",
"com/google/android/gms/cast/**",
"com/google/android/gms/drive/**",
"com/google/android/gms/fitness/**",
"com/google/android/gms/games/**",
"com/google/android/gms/gcm/**",
"com/google/android/gms/identity/**",
"com/google/android/gms/location/**",
"com/google/android/gms/maps/**",
"com/google/android/gms/panorama/**",
"com/google/android/gms/plus/**",
"com/google/android/gms/security/**",
"com/google/android/gms/tagmanager/**",
"com/google/android/gms/wallet/**",
"com/google/android/gms/wearable/**"]
Task stripPlayServices = project.tasks.create(name: 'stripPlayServices', group: "Strip") {
inputs.files new File(playServiceRootFolder, "classes.jar")
outputs.dir playServiceRootFolder
description 'Strip useless packages from Google Play Services library to avoid reaching dex limit'
doLast {
def packageExcludesAsString = packageToExclude.join(",")
if (libFile.exists()
&& libFile.text == packageExcludesAsString
&& classesStrippedJar.exists()) {
println "Play services already stripped"
copy {
from(file(classesStrippedJar))
into(file(playServiceRootFolder))
rename { fileName ->
fileName = "classes.jar"
}
}
} else {
copy {
from(file(new File(playServiceRootFolder, "classes.jar")))
into(file(playServiceRootFolder))
rename { fileName ->
fileName = "classes_orig.jar"
}
}
tasks.create(name: "stripPlayServices" + module.version, type: Jar) {
destinationDir = playServiceRootFolder
archiveName = "classes.jar"
from(zipTree(new File(playServiceRootFolder, "classes_orig.jar"))) {
exclude packageToExclude
}
}.execute()
delete file(new File(playServiceRootFolder, "classes_orig.jar"))
copy {
from(file(new File(playServiceRootFolder, "classes.jar")))
into(file(tmpDir))
rename { fileName ->
fileName = strippedClassFileName
}
}
libFile.text = packageExcludesAsString
}
}
}
project.tasks.findAll {
it.name.startsWith('prepare') && it.name.endsWith('Dependencies')
}.each { Task task ->
task.dependsOn stripPlayServices
}
project.tasks.findAll { it.name.contains(prepareTaskName) }.each { Task task ->
stripPlayServices.mustRunAfter task
}
}
Then apply this script in your build.gradle, like this
apply plugin: 'com.android.application'
apply from: 'strip_play_services.gradle'

If using Google Play Services, you may know that it adds 20k+ methods. As already mentioned, Android Studio has the option for modular inclusion of specific services, but users stuck with Eclipse have to take modularisation into their own hands :(
Fortunately there's a shell script that makes the job fairly easy. Just extract to the google play services jar directory, edit the supplied .conf file as needed and execute the shell script.
An example of its use is here.

If using Google Play Services, you may know that it adds 20k+ methods. As already mentioned, Android Studio has the option for modular inclusion of specific services, but users stuck with Eclipse have to take modularisation into their own hands :(
Fortunately there's a shell script that makes the job fairly easy. Just extract to the google play services jar directory, edit the supplied .conf file as needed and execute the shell script.
An example of its use is here.
Just like he said, I replaces compile 'com.google.android.gms:play-services:9.0.0' just with the libraries that I needed and it worked.

Related

Is it possible to get dependency version at runtime, including from library itself?

Background
Suppose I make an Android library called "MySdk", and I publish it on Jitpack/Maven.
The user of the SDK would use it by adding just the dependency of :
implementation 'com.github.my-sdk:MySdk:1.0.1'
What I'd like to get is the "1.0.1" part from it, whether I do it from within the Android library itself (can be useful to send to the SDK-server which version is used), or from the app that uses it (can be useful to report about specific issues, including via Crashlytics).
The problem
I can't find any reflection or gradle task to reach it.
What I've tried
Searching about it, if I indeed work on the Android library (that is used as a dependency), all I've found is that I can manage the version myself, via code.
Some said I could use BuildConfig of the package name of the library, but then it means that if I forget to update the code a moment before I publish the dependency, it will use the wrong value. Example of using this method:
plugins {
...
}
final def sdkVersion = "1.0.22"
android {
...
buildTypes {
release {
...
buildConfigField "String", "SDK_VERSION", "\"" + sdkVersion + "\""
}
debug {
buildConfigField "String", "SDK_VERSION", "\"" + sdkVersion + "-unreleased\""
}
}
Usage is just checking the value of BuildConfig.SDK_VERSION (after building).
Another possible solution is perhaps from gradle task inside the Android-library, that would be forced to be launched whenever you build the app that uses this library. However, I've failed to find how do it (found something here)
The question
Is it possible to query the dependency version from within the Android library of the dependency (and from the app that uses it, of course), so that I could use it during runtime?
Something automatic, that won't require me to update it before publishing ?
Maybe using Gradle task that is defined in the library, and forced to be used when building the app that uses the library?
You can use a Gradle task to capture the version of the library as presented in the build.gradle dependencies and store the version information in BuildConfig.java for each build type.
The task below captures the version of the "appcompat" dependency as an example.
dependencies {
implementation 'androidx.appcompat:appcompat:1.4.0'
}
task CaptureLibraryVersion {
def libDef = project.configurations.getByName('implementation').allDependencies.matching {
it.group.equals("androidx.appcompat") && it.name.equals("appcompat")
}
if (libDef.size() > 0) {
android.buildTypes.each {
it.buildConfigField 'String', 'LIB_VERSION', "\"${libDef[0].version}\""
}
}
}
For my example, the "appcompat" version was 1.4.0. After the task is run, BuildConfig.java contains
// Field from build type: debug
public static final String LIB_VERSION = "1.4.0";
You can reference this field in code with BuildConfig.LIB_VERSION. The task can be automatically run during each build cycle.
The simple answer to your question is 'yes' - you can do it. But if you want a simple solution to do it so the answer transforms to 'no' - there is no simple solution.
The libraries are in the classpath of your package, thus the only way to access their info at the runtime would be to record needed information during the compilation time and expose it to your application at the runtime.
There are two major 'correct' ways and you kinda have described them in your question but I will elaborate a bit.
The most correct way and relatively easy way is to expose all those variables as BuildConfig or String res values via gradle pretty much as described here. You can try to generify the approach for this using local-prefs(or helper gradle file) to store versions and use them everywhere it is needed. More info here, here, and here
The second correct, but much more complicated way is to write a gradle plugin or at least some set of tasks for collecting needed values during compile-time and providing an interface(usually via your app assets or res) for your app to access them during runtime. A pretty similar thing is already implemented for google libraries in Google Play services Plugins so it would be a good place to start.
All the other possible implementations are variations of the described two or their combination.
You can create buildSrc folder and manage dependencies in there.
after that, you can import & use Versions class in anywhere of your app.

Stripping data files from ICU4J when used in a Library

I have an internal library that is included in different android apps.
This library depends on ICU4J. This means we add ~10MB to the final APK.
The lib uses a subset of ICU4J so I would like to remove all the non-necessary data files.
From the ICU documentation:
Currently ICU4J provides no tool for revealing these dependencies between data files, so trimming the data directly in the ICU4J project is a hit-or-miss affair. The key point when you remove data is to make sure to remove all dependencies on that data as well.
I'd like to remove the data files when the app is built.
One question on StackOverflow is related: Exclude specific resources from an aar dep. Unfortunately, the exploded-aar directory does not exist anymore.
Do you know at which step I can remove the files from the ICU4J dependency ? Here's what I've tried to remove the cjdict.dict file:
tasks.create("excludeTask") << {
["java/com/ibm/icu/impl/data/icudt60b/brkitr/cjdict.dict"]
.collect { "${getGroup()}/${getName()}/${getVersion()}/${it}"}
.each {
// Question 2. From which dir should I remove the files?
File file = file("${buildDir}/intermediates/exploded-aar/${it}")
println("Excluding file " + file)
if (file.exists()) {
file.delete();
}
}
}
tasks.whenTaskAdded({
// Question 1. Before which task should I inject my excludeTask?
if (it.name.matches(/^transformClassesAndResources.*$/)) {
it.dependsOn excludeTask
}
})
Question 1: Before which task should I inject my excludeTask?
Question 2: From which dir should I remove the files?
I know that ICU4J can be used in Android 7.0 without all this overhead but I'd like to make the APK as light as I can for older devices.
Proguard/shrinking resources won't work for this (unless I miss something?)
A better mechanism for slicing ICU locale data is a common feature request, so we are working on rolling this out in an upcoming ICU release (either 63 or 64 depending on how things go). In the mean time, freel free to contribute to our design doc:
https://docs.google.com/document/d/1Lt9sHy7VbMLA2KbbEpg-TgNAI8bqtetwM767enWjcUg/edit#

Ignore proguard configuration of an external library

So, I want to add an external library to my project. The library itself is quite small, around 300 methods. But it is configured to be very liberal with it's proguard configuration. I ran a simple test with/without the library and with/without proguard on a barebones project and this is what I came up with
Proguard Lib Method Count
N N 15631
Y N 6370
N Y 15945
Y Y 15573
As you can see, with proguard enabled, the count is ~6000. But the moment I add the lib, count shoots up to ~15000 despite the library itself being only ~300 methods.
So my question is, how do I ignore the proguard configuration of this particular library?
UPDATE:
It is not possible with android gradle plugin now. I found android bug which doesn't have priority at all. Please avoid answers with mentioning "it is not possible" and keep question opened until a workaround or an official decision is possible. Otherwise, you will collect half of bounty without adding value. Thanks!
In this specific case you have a few options:
extract the classes.jar file from the aar and include it as normal jar dependency in your project (will not work when the aar includes resources)
change the aar and remove the consumer proguard rules from it
use DexGuard which allows you to filter out unwanted consumer rules
do a bit of gradle hacking, see below
Add the following to your build.gradle:
afterEvaluate {
// All proguard tasks shall depend on our filter task
def proguardTasks = tasks.findAll { task ->
task.name.startsWith('transformClassesAndResourcesWithProguardFor') }
proguardTasks.each { task -> task.dependsOn filterConsumerRules }
}
// Let's define our custom task that filters some unwanted
// consumer proguard rules
task(filterConsumerRules) << {
// Collect all consumer rules first
FileTree allConsumerRules = fileTree(dir: 'build/intermediates/exploded-aar',
include: '**/proguard.txt')
// Now filter the ones we want to exclude:
// Change it to fit your needs, replace library with
// the name of the aar you want to filter.
FileTree excludeRules = allConsumerRules.matching {
include '**/library/**'
}
// Print some info and delete the file, so ProGuard
// does not pick it up. We could also just rename it.
excludeRules.each { File file ->
println 'Deleting ProGuard consumer rule ' + file
file.delete()
}
}
When using DexGuard (7.2.02+), you can add the following snippet to your build.gradle:
dexguard {
// Replace library with the name of the aar you want to filter
// The ** at the end will include every other rule.
consumerRuleFilter '!**/library/**,**'
}
Mind that the logic is inverted to the ProGuard example above, the consumerRuleFilter will only include consumer rules that match the pattern.
In case you're using R8 (which replaced ProGuard since Android Gradle plugin 3.4.0) - you can filter-out specific consumer rule files by adding the following work-around to your module's build.gradle:
tasks.whenTaskAdded { Task task ->
// Once 'minifyEnabled' is set to 'true' for a certain build type/variant -
// a 'minify<variantName>WithR8' task will be created for each such variant
//
// - This task is implemented by com.android.build.gradle.internal.tasks.R8Task
// - R8Task extends from ProguardConfigurableTask
// - ProguardConfigurableTask exposes property 'configurationFiles'
// - configurationFiles contains all files that will be contributing R8 rules
// - configurationFiles is mutable (its type is ConfigurableFileCollection)
//
// Thus - we can overwrite the list of files and filter them out as we please
//
// More details: https://cs.android.com/android-studio/platform/tools/base/+/mirror-goog-studio-main:build-system/gradle-core/src/main/java/com/android/build/gradle/internal/tasks/R8Task.kt
if (task.name.startsWith("minify") && task.name.endsWith("WithR8")) {
afterEvaluate {
def filteredList = task.configurationFiles.filter {
// Example paths in this collection:
// /Users/me/MyProject/myModule/proguard-rules.pro
// (for library dependencies) /Users/me/.gradle/caches/<...>/okhttp3.pro
// The below filter condition will, for example, exclude consumer ProGuard rules file from the AndroidX RecyclerView library
!it.path.contains("recyclerview-1.1.0")
}
task.configurationFiles.setFrom(filteredList.files)
}
}
}
The above work-around was confirmed to be working on Android Gradle plugin 4.2.2
If one decides to rely on such a work-around - it might be a good idea to also add some sort of automated checks and/or tests to make sure this filtering is working. Given that the solution is quite fragile and can break with future updates of Android Gradle plugin.
Inspired by Jonas' answer, modified for Kotlin DSL and confirmed working on Android Gradle plugin 7.2.1:
import com.android.build.gradle.internal.tasks.ProguardConfigurableTask
afterEvaluate {
// Get each ProguardConfigurableTask
tasks.withType(ProguardConfigurableTask::class.java).forEach { task ->
// Remove proguard rules from lifecycle-runtime library
val filteredConfigurationFiles = task.configurationFiles.filter { file ->
!file.path.contains("lifecycle-runtime")
}
task.configurationFiles.setFrom(filteredConfigurationFiles)
}
}

DexIndexOverflowException with build tool 21.0.0 [duplicate]

I have seen various versions of the dex erros before, but this one is new. clean/restart etc won't help. Library projects seems intact and dependency seems to be linked correctly.
Unable to execute dex: method ID not in [0, 0xffff]: 65536
Conversion to Dalvik format failed: Unable to execute dex: method ID not in [0, 0xffff]: 65536
or
Cannot merge new index 65950 into a non-jumbo instruction
or
java.util.concurrent.ExecutionException: com.android.dex.DexIndexOverflowException: method ID not in [0, 0xffff]: 65536
tl;dr: Official solution from Google is finally here!
http://developer.android.com/tools/building/multidex.html
Only one small tip, you will likely need to do this to prevent out of memory when doing dex-ing.
dexOptions {
javaMaxHeapSize "4g"
}
There's also a jumbo mode that can fix this in a less reliable way:
dexOptions {
jumboMode true
}
Update: If your app is fat and you have too many methods inside your main app, you may need to re-org your app as per
http://blog.osom.info/2014/12/too-many-methods-in-main-dex.html
Update 3 (11/3/2014)
Google finally released official description.
Update 2 (10/31/2014)
Gradle plugin v0.14.0 for Android adds support for multi-dex. To enable, you just have to declare it in build.gradle:
android {
defaultConfig {
...
multiDexEnabled true
}
}
If your application supports Android prior to 5.0 (that is, if your minSdkVersion is 20 or below) you also have to dynamically patch the application ClassLoader, so it will be able to load classes from secondary dexes. Fortunately, there's a library that does that for you. Add it to your app's dependencies:
dependencies {
...
compile 'com.android.support:multidex:1.0.0'
}
You need to call the ClassLoader patch code as soon as possible. MultiDexApplication class's documentation suggests three ways to do that (pick one of them, one that's most convenient for you):
1 - Declare MultiDexApplication class as the application in your AndroidManifest.xml:
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.android.multidex.myapplication">
<application
...
android:name="android.support.multidex.MultiDexApplication">
...
</application>
</manifest>
2 - Have your Application class extend MultiDexApplication class:
public class MyApplication extends MultiDexApplication { .. }
3 - Call MultiDex#install from your Application#attachBaseContext method:
public class MyApplication {
protected void attachBaseContext(Context base) {
super.attachBaseContext(base);
MultiDex.install(this);
....
}
....
}
Update 1 (10/17/2014):
As anticipated, multidex support is shipped in revision 21 of Android Support Library. You can find the android-support-multidex.jar in /sdk/extras/android/support/multidex/library/libs folder.
Multi-dex support solves this problem. dx 1.8 already allows generating several dex files.
Android L will support multi-dex natively, and next revision of support library is going to cover older releases back to API 4.
It was stated in this Android Developers Backstage podcast episode by Anwar Ghuloum. I've posted a transcript (and general multi-dex explanation) of the relevant part.
As already stated, you have too many methods (more than 65k) in your project and libs.
Prevent the Problem: Reduce the number of methods with Play Services 6.5+ and support-v4 24.2+
Since often the Google Play services is one of the main suspects in "wasting" methods with its 20k+ methods. Google Play services version 6.5 or later, it is possible for you to include Google Play services in your application using a number of smaller client libraries. For example, if you only need GCM and maps you can choose to use these dependencies only:
dependencies {
compile 'com.google.android.gms:play-services-base:6.5.+'
compile 'com.google.android.gms:play-services-maps:6.5.+'
}
The full list of sub libraries and it's responsibilities can be found in the official google doc.
Update: Since Support Library v4 v24.2.0 it was split up into the following modules:
support-compat, support-core-utils, support-core-ui, support-media-compat and support-fragment
dependencies {
compile 'com.android.support:support-fragment:24.2.+'
}
Do note however, if you use support-fragment, it will have dependencies to all the other modules (ie. if you use android.support.v4.app.Fragment there is no benefit)
See here the official release notes for support-v4 lib
Enable MultiDexing
Since Lollipop (aka build tools 21+) it is very easy to handle. The approach is to work around the 65k methods per dex file problem to create multiple dex files for your app. Add the following to your gradle build file (this is taken from the official google doc on applications with more than 65k methods):
android {
compileSdkVersion 21
buildToolsVersion "21.1.0"
defaultConfig {
...
// Enabling multidex support.
multiDexEnabled true
}
...
}
dependencies {
compile 'com.android.support:multidex:1.0.1'
}
The second step is to either prepare your Application class or if you don't extend Application use the MultiDexApplication in your Android Manifest:
Either add this to your Application.java
#Override
protected void attachBaseContext(Context base) {
super.attachBaseContext(base);
MultiDex.install(this);
}
or use the provided application from the mutlidex lib
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.android.myapplication">
<application
...
android:name="android.support.multidex.MultiDexApplication">
...
</application>
</manifest>
Prevent OutOfMemory with MultiDex
As further tip, if you run into OutOfMemory exceptions during the build phase you could enlarge the heap with
android {
...
dexOptions {
javaMaxHeapSize "4g"
}
}
which would set the heap to 4 gigabytes.
See this question for more detail on the dex heap memory issue.
Analyze the source of the Problem
To analyze the source of the methods the gradle plugin https://github.com/KeepSafe/dexcount-gradle-plugin can help in combination with the dependency tree provided by gradle with e.g.
.\gradlew app:dependencies
See this answer and question for more information on method count in android
Your project is too large. You have too many methods. There can only be 65536 methods per application. see here https://code.google.com/p/android/issues/detail?id=7147#c6
The below code helps, if you use Gradle. Allows you to easily remove unneeded Google services (presuming you're using them) to get back below the 65k threshold. All credit to this post: https://gist.github.com/dmarcato/d7c91b94214acd936e42
Edit 2014-10-22: There's been a lot of interesting discussion on the gist referenced above. TLDR? look at this one: https://gist.github.com/Takhion/10a37046b9e6d259bb31
Paste this code at the bottom of your build.gradle file and adjust the list of google services you do not need:
def toCamelCase(String string) {
String result = ""
string.findAll("[^\\W]+") { String word ->
result += word.capitalize()
}
return result
}
afterEvaluate { project ->
Configuration runtimeConfiguration = project.configurations.getByName('compile')
ResolutionResult resolution = runtimeConfiguration.incoming.resolutionResult
// Forces resolve of configuration
ModuleVersionIdentifier module = resolution.getAllComponents().find { it.moduleVersion.name.equals("play-services") }.moduleVersion
String prepareTaskName = "prepare${toCamelCase("${module.group} ${module.name} ${module.version}")}Library"
File playServiceRootFolder = project.tasks.find { it.name.equals(prepareTaskName) }.explodedDir
Task stripPlayServices = project.tasks.create(name: 'stripPlayServices', group: "Strip") {
inputs.files new File(playServiceRootFolder, "classes.jar")
outputs.dir playServiceRootFolder
description 'Strip useless packages from Google Play Services library to avoid reaching dex limit'
doLast {
copy {
from(file(new File(playServiceRootFolder, "classes.jar")))
into(file(playServiceRootFolder))
rename { fileName ->
fileName = "classes_orig.jar"
}
}
tasks.create(name: "stripPlayServices" + module.version, type: Jar) {
destinationDir = playServiceRootFolder
archiveName = "classes.jar"
from(zipTree(new File(playServiceRootFolder, "classes_orig.jar"))) {
exclude "com/google/ads/**"
exclude "com/google/android/gms/analytics/**"
exclude "com/google/android/gms/games/**"
exclude "com/google/android/gms/plus/**"
exclude "com/google/android/gms/drive/**"
exclude "com/google/android/gms/ads/**"
}
}.execute()
delete file(new File(playServiceRootFolder, "classes_orig.jar"))
}
}
project.tasks.findAll { it.name.startsWith('prepare') && it.name.endsWith('Dependencies') }.each { Task task ->
task.dependsOn stripPlayServices
}
}
I've shared a sample project which solve this problem using custom_rules.xml build script and a few lines of code.
I used it on my own project and it is runs flawless on 1M+ devices (from android-8 to the latest android-19). Hope it helps.
https://github.com/mmin18/Dex65536
Faced the same problem and solved it by editing my build.gradle file on the dependencies section, removing:
compile 'com.google.android.gms:play-services:7.8.0'
And replacing it with:
compile 'com.google.android.gms:play-services-location:7.8.0'
compile 'com.google.android.gms:play-services-analytics:7.8.0'
Try adding below code in build.gradle, it worked for me -
compileSdkVersion 23
buildToolsVersion '23.0.1'
defaultConfig {
multiDexEnabled true
}
The perfect solution for this would be to work with Proguard. as aleb mentioned in the comment.
It will decrease the size of the dex file by half.
You can analyse problem (dex file references) using Android Studio:
Build -> Analyse APK ..
On the result panel click on classes.dex file
And you'll see:
gradle + proguard solution:
afterEvaluate {
tasks.each {
if (it.name.startsWith('proguard')) {
it.getInJarFilters().each { filter ->
if (filter && filter['filter']) {
filter['filter'] = filter['filter'] +
',!.readme' +
',!META-INF/LICENSE' +
',!META-INF/LICENSE.txt' +
',!META-INF/NOTICE' +
',!META-INF/NOTICE.txt' +
',!com/google/android/gms/ads/**' +
',!com/google/android/gms/cast/**' +
',!com/google/android/gms/games/**' +
',!com/google/android/gms/drive/**' +
',!com/google/android/gms/wallet/**' +
',!com/google/android/gms/wearable/**' +
',!com/google/android/gms/plus/**' +
',!com/google/android/gms/topmanager/**'
}
}
}
}
}
Remove some jar file from Libs folder and copy to some other folder, And Go to _Project Properties > Select Java Build Path, Select Libraries, Select Add External Jar, Select the Removed jar to your project, Click save, this will be added under Referenced Library instead of Libs folder. Now clean and Run your project. You dont need to add Any code for MultDex. Its simply worked for me.
I was facing the same issue today what worked for is below down
For ANDROID STUDIO... Enable Instant Run
In File->Preferences->Build, Execution, Deployment->Instant Run-> Check Enable Instant run for hot swap...
Hope it helps

Custom Class Loading in Dalvik with Gradle (Android New Build System)

As per the introduction of Custom Class Loading in Dalvik by Fred Chung on the Android Developers Blog:
The Dalvik VM provides facilities for developers to perform custom
class loading. Instead of loading Dalvik executable (“dex”) files from
the default location, an application can load them from alternative
locations such as internal storage or over the network.
However, not many developers have the need to do custom class loading. But those who do and follow the instructions on that blog post, might have some problems mimicking the same behavior with Gradle, the new build system for Android introduced in Google I/O 2013.
How exactly one can adapt the new build system to perform the same intermediary steps as in the old (Ant based) build system?
My team and I recently reached the 64K method references in our app, which is the maximum number of supported in a dex file. To get around this limitation, we need to partition part of the program into multiple secondary dex files, and load them at runtime.
We followed the blog post mentioned in the question for the old, Ant based, build system and everything was working just fine. But we recently felt the need to move to the new build system, based on Gradle.
This answer does not intend to replace the full blog post with a complete example. Instead, it will simply explain how to use Gradle to tweak the build process and achieve the same thing. Please note that this is probably just one way of doing it and how we are currently doing it in our team. It doesn't necessarily mean it's the only way.
Our project is structured a little different and this example works as an individual Java project that will compile all the source code into .class files, assemble them into a single .dex file and to finish, package that single .dex file into a .jar file.
Let's start...
In the root build.gradle we have the following piece of code to define some defaults:
ext.androidSdkDir = System.env.ANDROID_HOME
if(androidSdkDir == null) {
Properties localProps = new Properties()
localProps.load(new FileInputStream(file('local.properties')))
ext.androidSdkDir = localProps['sdk.dir']
}
ext.buildToolsVersion = '18.0.1'
ext.compileSdkVersion = 18
We need the code above because although the example is an individual Java project, we still need to use components from the Android SDK. And we will also be needing some of the other properties later on... So, on the build.gradle of the main project, we have this dependency:
dependencies {
compile files("${androidSdkDir}/platforms/android-${compileSdkVersion}/android.jar")
}
We are also simplifying the source sets of this project, which might not be necessary for your project:
sourceSets {
main {
java.srcDirs = ['src']
}
}
Next, we change the default configuration of the build-in jar task to simply include the classes.dex file instead of all .class files:
configure(jar) {
include 'classes.dex'
}
Now we need to have new task that will actually assemble all .class files into a single .dex file. In our case, we also need to include the Protobuf library JAR into the .dex file. So I'm including that in the example here:
task dexClasses << {
String protobufJarPath = ''
String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''
configurations.compile.files.find {
if(it.name.startsWith('protobuf-java')) {
protobufJarPath = it.path
}
}
exec {
commandLine "${androidSdkDir}/build-tools/${buildToolsVersion}/dx${cmdExt}", '--dex',
"--output=${buildDir}/classes/main/classes.dex",
"${buildDir}/classes/main", "${protobufJarPath}"
}
}
Also, make sure you have the following import somewhere (usually at the top, of course) on your build.gradle file:
import org.apache.tools.ant.taskdefs.condition.Os
Now we must make the jar task depend on our dexClasses task, to make sure that our task is executed before the final .jar file is assembled. We do that with a simple line of code:
jar.dependsOn(dexClasses)
And we're done... Simply invoke Gradle with the usual assemble task and your final .jar file, ${buildDir}/libs/${archivesBaseName}.jar will contain a single classes.dex file (besides the MANIFEST.MF file). Just copy that into your app assets folder (you can always automate that with Gradle as we've done but that is out of scope of this question) and follow the rest of the blog post.
If you have any questions, just shout in the comments. I'll try to help to the best of my abilities.
The Android Studio Gradle plugin now provides native multidex support, which effectively solves the Android 65k method limit without having to manually load classes from a jar file, and thus makes Fred Chung's blog obsolete for that purpose. However, loading custom classes from a jar file at runtime in Android is still useful for the purpose of extensibility (e.g. making a plugin framework for your app), so I'll address that usage scenario below:
I have created a port of the original example app on Fred Chung's blog to Android Studio on my github page over here using the Android library plugin rather than the Java plugin. Instead of trying to modify the existing dex process to split up into two modules like in the blog, I've put the code which we want to go into the jar file into its own module, and added a custom task assembleExternalJar which dexes the necessary class files after the main assemble task has finished.
Here is relevant part of the build.gradle file for the library. If your library module has any dependencies which are not in the main project then you will probably need to modify this script to add them.
apply plugin: 'com.android.library'
// ... see github project for the full build.gradle file
// Define some tasks which are used in the build process
task copyClasses(type: Copy) { // Copy the assembled *.class files for only the current namespace into a new directory
// get directory for current namespace (PLUGIN_NAMESPACE = 'com.example.toastlib')
def namespacePath = PLUGIN_NAMESPACE.replaceAll("\\.","/")
// set source and destination directories
from "build/intermediates/classes/release/${namespacePath}/"
into "build/intermediates/dex/${namespacePath}/"
// exclude classes which don't have a corresponding .java entry in the source directory
def remExt = { name -> name.lastIndexOf('.').with {it != -1 ? name[0..<it] : name} }
eachFile {details ->
def thisFile = new File("${projectDir}/src/main/java/${namespacePath}/", remExt(details.name)+".java")
if (!(thisFile.exists())) {
details.exclude()
}
}
}
task assembleExternalJar << {
// Get the location of the Android SDK
ext.androidSdkDir = System.env.ANDROID_HOME
if(androidSdkDir == null) {
Properties localProps = new Properties()
localProps.load(new FileInputStream(file('local.properties')))
ext.androidSdkDir = localProps['sdk.dir']
}
// Make sure no existing jar file exists as this will cause dx to fail
new File("${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar").delete();
// Use command line dx utility to convert *.class files into classes.dex inside jar archive
String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''
exec {
commandLine "${androidSdkDir}/build-tools/${BUILD_TOOLS_VERSION}/dx${cmdExt}", '--dex',
"--output=${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar",
"${buildDir}/intermediates/dex/"
}
copyJarToOutputs.execute()
}
task copyJarToOutputs(type: Copy) {
// Copy the built jar archive to the outputs folder
from 'build/intermediates/dex/'
into 'build/outputs/'
include '*.jar'
}
// Set the dependencies of the build tasks so that assembleExternalJar does a complete build
copyClasses.dependsOn(assemble)
assembleExternalJar.dependsOn(copyClasses)
For more detailed information see the full source code for the sample app on my github.
See my answer over here. The key points are:
Use the additionalParameters property on the dynamically created dexCamelCase tasks to pass --multi-dex to dx and create multiple dex files.
Use the multidex class loader to use the multiple dex files.

Categories

Resources