maven-metadata.xml keeps getting requested when building - android

AGP 4.2.2
gradle-6.7.1
arcticfox 2020.3.1
Everytime I build my android project I see this maven-metadata.xml and it just seems to slow the build and it just seems to hang there. Which can take 10 minutes for each build.
Is there a way to avoid this?
Gradle: Download maven-metadata.xml...
When I run one the command line I see this:
This is my gradle.properties
org.gradle.jvmargs=-Xmx8g -XX:MaxPermSize=512m -XX:ReservedCodeCacheSize=256m -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8
org.gradle.daemon=true
org.gradle.parallel=true
org.gradle.caching=true
org.gradle.configureondemand=true
kotlin.coroutines=enable
kotlin.code.style=official
kapt.use.worker.api=true
kapt.incremental.apt=true
android.enableD8.desugaring=true
android.useAndroidX=true
android.enableJetifier=true
I am using some maven repositories as specified here in build.gradle(app):
maven { url "https://www.jitpack.io" }
maven { url 'http://......./repo/maven2' }
maven { url "https://....../repositories/snapshots" }
maven { url 'https://....../maven/release' }
maven { url "http://......./bintray.com/maven" }
maven { url "https://....../objectbox/objectbox" }

Gradle usually needs to download a maven-metadata.xml file, if
you use a dynamic/changing version for one of your dependencies, and
the time to live (TTL) in the dependency cache is over.
For (1) you should look for versions like 1.2.3-SNAPSHOT or 1.2.+ on your declared dependencies (or at ./gradlew dependencies for simplicity). If you have any such versions, do you really need them? Using fixed versions like 1.2.3 should be faster.
For (2), have you maybe changed the default TTL threshold or do you maybe always build with --refresh-dependencies? If not, then the slowness should at least not occur more often than once a day.
If the above doesn’t help, then maybe try running your build with --info or --info --refresh-dependencies and carefully watch the log output. That should show which dependency and/or which repository is the culprit (i.e., the one on which the logging is stuck for the longest time). If you find such a culprit, then you could look into things like
replacing the dependency (or the repository),
improving the speed of the repository server or the network, or
pruning old snapshot versions from the problematic maven-metadata.xml file (if you control the repository).
If all this still doesn’t turn up actionable items, then maybe it could be worth looking for bigger maven-metadata.xml files in your Gradle dependency cache. Maybe the repeated download of such big files is the issue and one of the approaches from the previous list could help. Here’s an idea for how to find the ten biggest maven-metadata.xml files (with a Unix shell command):
find ~/.gradle/caches/modules-2/resources-* \
-name maven-metadata.xml \
-printf '%s\t%p\n' \
| sort -k1 \
| tail

The question is kind of insufficient to tell, because no build.gradle had been provided. But I'd assume, that the reason is any -SNAPSHOT dependency, which is not being cached. Building against a local copy of that dependency should result in not downloading the maven-metadata.xml over and over again; or use mavenLocal(). Building against a stable version should also result in caching.

Other answers give very good context on why this happens. Especially when using -SNAPSHOT dependencies, if you have no other way, I have a trick for you that might help a lot.
It is to use offline mode 🎉
When running via CLI, do
./gradlew someTask --ofline
When running via IDE, enable offline mode.

The cause of slow downloads for maven-metadata.xml files is that one of your proxy repositories has a very slow remote.
This problem will be compounded if either of the "metadata max age" and "not found cache ttl" settings is set too low. Thus it will check the circle too fast and download it everytime from scratch. Turn it up to 12 hours. Default should be 30min (1800sec) what is a too short period of time.
For setting the max age you can simple follow along this site.
Another good article to manage Http Cache Headers.

Chriki answer directed me in the right direction to find a simple solution. After you have downloaded all dependencies (in my case, I am using -SNAPSHOT library, which wants to be updated on every run), you can set Gradle in offline mode. This will prevent Gradle to download new metadata, etc.
In Android Studio you have a dedicated button to toggle online mode (see image
Android studio settings).

Related

Gradle build time too slow

I have an Android project that has around 15 modules each of them is a separate GitHub project, some of the modules have interdependencies, its basically a sample client with a library that has 14 components(modules).
Every time I make a single change and re build/run it takes takes about 1 min 40 seconds on an i7 with 28GB Ram
I have tried many things to improve the build time, including the tips in this article,But I have not seen a significant change.
https://medium.com/#erikhellman/boosting-the-performance-for-gradle-in-your-android-projects-6d5f9e4580b6
it looks like it goes through all the modules and see if they are Up to Date, which takes time, but also compiling and the dexing.
Does any one have an idea on how to improve this?
I am using Gradle plugin 1.2.3, BuildTools 22.0.1 and TaskWrapper 2.4
I was suffering from the same problem before but as google announced in there last I/O that there is a big improvement in performance in Gradle and android studio in common.
but for me this single trick helps me a lot and reduce the time by more than 50%.
make Gradle run offline.
The next official planned release of Gradle plugin is 1.3. If you watch the talk on gradle from a presentation at Google I/O https://youtu.be/f7ihSQ44WO0?t=4m23s then Google are making claims of big performance improvements. If these claims prove to be true then I believe that this will be the best chance of cutting down build times for Android projects.
Currently you can use the latest beta version which is
1.3.0-beta4 by adding it to your build.gradle file. Example below:
// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
repositories {
jcenter()
}
dependencies {
classpath 'com.android.tools.build:gradle:1.3.0-beta4'
}
}
allprojects {
repositories {
jcenter()
}
}
Checking for any new beta versions can be done at jcenter https://jcenter.bintray.com/com/android/tools/build/gradle/
If you don't need that all dependency update every time you run your project, I suggest you to configure gradle to work offline. You can do that in Android Studio in the Gradle tab in the Setings windows. I hope that this can help you.

android studio 1.2 gradle is very slow

it's been a while that I'm using Android Studio, and up until now I was using 1.0.1,
gradle was a bit slow, around 1.5 minute for assembleDebug (my project is really big!)
but today I updated my AS to 1.2 and now same process takes about 7 to 10 minutes, and sometimes even with no result!
is there any setting I have to change to make it faster ?
honestly taking 10 minute for every debug run is a nightmare !
Also most of the time, my cpu usage is arround 10 percent! (it is actually idle!)
cause before when gradle was working it was on 100% almost all the time
had the same problem.
What I did was to change the global gradle settings to offline work which can be done by going to Preferences => Gradle. This did make a difference.
Another method I have seen people use, but which I have not used yet is to create a gradle.properties in the gradle folder like so:
Just create a file named gradle.properties in the following directory:
/home/<username>/.gradle/ (Linux)
/Users/<username>/.gradle/ (Mac)
C:\Users\<username>\.gradle (Windows)
Add this line to the file:
org.gradle.daemon=true
Please check out this link for more options as well as a detailed explanation on speeding up gradle.
Hope this helps!.
I was testing my app with Google+ log in. So I added release signing to debug version. App compiling in ~ 26 seconds.
build.gradle Module: app file
signingConfigs {
debug {
storeFile file(project.property("MyApp.signing"))
storePassword project.property("MyApp.signing.password")
keyAlias project.property("MyApp.signing.alias")
keyPassword project.property("MyApp.signing.password")
}
}
When I remove that ~ 7.5 seconds.
Next I tested offline grade
File - Settings - Build, Execution... - Build Tools - Gradle - Offline work
Now my app compiling in ~ 4.5 seconds.
Of course I also added turn on
- Compile independent modules in parallel (may require larger heap size)
- Make project automatically (only works while not running / debugging)
File - Settings - Build, Execution... - Compiler
Complete answer for this issue is as below:
Upgrade android studio to version 1.3(stable) or above 1.4(beta at the time of writing this).
Upgrade gradle to 1.3.+(+ can be replaced with some positive number) change it in your build.gradle file.
change your gradle-wrapper.properties files and add distributionUrl=https\://services.gradle.org/distributions/gradle-2.4-all.zip in last(you can remove any old entry).
Go to Preference -> Gradle and set it to work offline.
woila!!! I am able to compile and run the code in less then ~5 sec (I really mean it)
The reason could be multiDex,
turn multiDexEnabled to false in your build.gradle file (for debug only, keep it for release).
android {
...
defaultConfig {
...
multiDexEnabled false
...
}
}
In addition you should consider to use the lastest version (2.4 at the moment) by editing the gradle-wrapper.properties file and set gradle-2.4-all.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-2.4-all.zip
What is MultiDex : https://developer.android.com/tools/building/multidex.html
From settings go to HTTP connection and disable any proxy and you will find speed you want

Ignore Gradle Build Failure and continue build script?

Managing Android's dependencies with Gradle is done in a weird way. They have to be downloaded differently into a local repo. This is a pain when setting up CI build as there are multiple nodes this could run on. As such I'm using the sdk-manager-plugin to have the Android dependencies downloaded at build time. There seems to be an old bug that I'm experiencing with the sdk-manager-plugin though in that it will download the dependencies at build time, but they won't be available on that command.
The next time the command is run everything works fine (as everything is already downloaded), but I need to find a way to ignore the build failure of the first gradle command so that everything is downloaded and good to go for the second. I realize this is hacky, but I'm done messing with this.
Ideally something like this would work:
./gradlew clean --ignoreBuildFailures
./gradlew distributeCIBuild
The closest thing I could find in the Gradle documentation is --quite but that doesn't look like it'd work.
Any creative solutions welcome.
The flag to use is --continue.
From the documentation:
Continues task execution after a task failure.
add this in the build.gradle file :
tasks.withType(JavaCompile) {
options.failOnError(false)
}
You can use ignoreExitValue
task ktlint(type: JavaExec, group: "verification") {
description = "Check Kotlin code style."
ignoreExitValue = true
}

Android Studio: Snapshot Dependencies Don't Update properly

I'm working With Android Studio 8.9
I've got a build.gradle with the following dependency defined:
compile ('my.program.commons:my-program-commons:0.0.2-SNAPSHOT#jar')
This dependency is stored in a private Sonatype nexus repository.
When I make changes in the my.program.commons code, I upload to nexus.
The problem is that when I then try to compile against the new SNAPSHOT android studio will fail to pick up changes.
When run from the command line gradle will build succesfully - but Android Studio will not recognize the new files.
If i do a version tick - say from 0.0.2-SNAPSHOT to 0.0.3-SNAPSHOT Android Studio will understand the new version and download and everything works out fine.
I don't want to have to do a minor version tick on every single change.
In my case, use changing = true not work for me. But configure cache changing modules solve my problem. Sample code below, add in build.gradle file:
configurations.all {
// Don't cache changing modules at all.
resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
}
See: https://docs.gradle.org/current/userguide/dependency_management.html
You can also put a flag called "changing" that will trigger Gradle to always pull the latest, for example:
compile ('my.program.commons:my-program-commons:0.0.2-SNAPSHOT#jar') {
changing = true;
}
You need to configure the cache duration, by default gradle won't look for updates for 24 hours:
http://www.gradle.org/docs/current/userguide/dependency_management.html#sec:controlling_caching
In my case, removing the whole <project_root>/.idea/libraries directory, was the only solution that worked. AndroidStudio stores some cached dependencies configurations there. Removing the directory makes it refetch all of them one more time.
You can write some script/task that will automate this removal and run it as part of the Gradle clean task.

Issues with Maven and Android project : transitive dependencies, compilation time and IDEA integration

I have an Android project which (unfortunately) uses Maven as a building manager. The thing is that I have a lot of standard java .jar dependencies.
The first issue I am facing is the following :
1) These jar have other dependencies, and sometimes are redundant. The problem is that the removeDuplicates functionality of the android-maven-plugin does not seem to work as intended and I encounter the classic "multiple dex files define class X". I have resolved this issue by excluding the redundant dependencies one by one, which is very tedious and provides me with a pom which can only be qualified as disgusting. So here is my question : is there any way to easily manage transitive dependencies with android-maven-plugin and avoid the "class defined in multiple dex files" error ? Does Gradle handle this kind of stuff in a better way ? Or am I doomed to use "*" in every dependency exclusions and basically remove the transitive dependency feature from Maven ?
The second issue is about compilation time and the behavior of IDEA when using Maven dependencies in an Android project :
2) First, having to dex that many dependencies is obviously very time consuming. Is there any way to avoid pre-dexing all the jars everytime ? If I understand things correctly, if the jar does not change between 2 builds, then it should not be dexed again. I guess it is because I call mvn clean everytime, however if I do not the "class defined in multiple dex files" error shows up again. The temporary workaround for me was to adequatly declare dependencies in my pom using Maven and use the "Make" function of IDEA, which I guess calls the android tools and build the stuff appropriately. It worked for some time, and it was actually great. The pre-dexing worked as I expected and did not re-dex everything at each compile command. However, I hit the same wall all over again, the "Make" command produces "class defined in multiple dex files", I guess that removing duplicates here does not work either (just in case I was not clear, the same project, with the same pom, builds successfully with Maven but encounters the error when using "Make").
Basically, what I really would like is to declare and manage my dependencies using Maven (actually I do not really have a choice in the matter) and build by calling "Make" in IDEA. The ideal case would be to have the same behavior as "Make" when invoking "mvn clean install". Just in case, I would like to point out the fact that using parallel building is a noticeable improvement but it is quite far from what I would expect.

Categories

Resources