Develop Reference

How to Run a Kotlin Task from Android Project's build.gradle?

My project's build.gradle is in Groovy but I'd like to run as a gradle task the main function of a Kotlin class in a root directory kt file.
I don't know where to begin in terms of syntax.
something like this? I came up with this after searching around a bit.
task doKotlinTask(type: JavaExec) {
classpath "/"
main = "KotlinTaskKt"
}

Create in the root directory buildSrc/src/main/kotlin and place MyKotlinClass.Ktthere. Create buildSrc/build.gradle to add any dependencies for MyKotlinClass.
Then in your project or app's build.gradle, append the following:
import my.package.for.the.kotlin.class
task doKotlinTask(type: MyKotlinClass) {
someParamForTheTask = "hello world"
}

Run all subprojects tests using Gradle

I have a multi-module android project with the structure:
:a
:b
:c
:d
:e
I'm trying to run a jacoco report on module :b so that it runs on :b, :c, :d, and :e without running :a. I want all of the xml reports to be in a common folder with names of their project.xml (e.g. b.xml, c.xml, etc.) I have a pretty standard jacoco setup
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {
reports {
xml.enabled = true
xml.destination = file(allTestCoverageDir + project.name + ".xml")
html.enabled = true
}
def fileFilter = [
'**/R.class',
'**/R$*.class',
'**/BuildConfig.*',
'**/Manifest*.*',
'**/*Test*.*',
'android/**/*.*',
//Dagger 2
'**/*Dagger*Component*.*',
'**/*Module.*',
'**/*Module$*.*',
'**/*MembersInjector*.*',
'**/*_Factory*.*',
'**/*Provide*Factory*.*',
]
def kotlinDebug = [fileTree(dir: "${project.buildDir}/tmp/kotlin-classes/debug", excludes: fileFilter)]
def mainSrc = files([
"$project.projectDir/src/main/java",
"$project.projectDir/src/main/kotlin"
])
sourceDirectories = files([mainSrc])
classDirectories = files(kotlinDebug)
executionData = fileTree(dir: project.buildDir, includes: [
'jacoco/testDebugUnitTest.exec', 'outputs/code-coverage/connected/*coverage.ec'
])
}
But when I try to loop through subprojects in a doLast block, the doLast block never runs and trying to access subprojects before that also shows that :a has no subprojects.
Edit I am able to run these for each sub project with ./gradlew b:jacocoTestReport or ./gradlew c:jacocoTestReport and all the reports and in a folder with the correct names. But as my project grows I don't want to have to run dozens of commands (one for each module) I want a single command ./gradlew b:jacocoTestReport (or something similar) which runs for b and it's subtree

As far as I understand at the moment you're using allprojects {} to configure all of the sub projects. Although this has been the canonical to configure a group of projects in the past, it is now discouraged. Furthermore projects should use publications to interface with each other instead of copying files across project boundaries. So you need to do two things:
Instead of configuring the sub projects from the root root project, you should create a plugin to configure jacoco and create configuration that will hold the reports.
To do this, create a pre-compiled script plugin in your project. The idea is to have a kotlin build script in the buildSrc project and create a Gradle plugin from that file on the fly. So you should move the logic that configures jacoco to the file buildSrc/src/main/kotlin/jacoco-conventions.gradle.kts:
plugins {
jacoco
}
val jacocoTestReport by tasks.getting(JacocoReport::class) {
// jacoco configuration
}
configurations.create("jacocoReports") {
isCanBeResolved = false
isCanBeConsumed = true
attributes {
attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class, "jacocoReports"))
}
outgoing.artifact(jacocoTestReport.reports.xml.destination) {
builtBy(jacocoTestReport)
}
}
The last part creates a new configuration in the project the pre-compiled script plugin is applied to. This configuration uses the xml destination file which is builtBy the jacoco report task as an outgoing artifact. The important part here is the USAGE_ATTRIBUTE because we will need this later on to consume the files.
The precompiled script plugin can now be applied in the projects where you want to gather jacoco metrics by:
// for example in c/build.gradle.kts
plugins {
`jacoco-conventions`
}
Now you have configured the sub projects to put the Jacoco xml reports into configurations with usage attribute jacocoReports.
In the root project create a task that copies the report from the configuration.
To do this we need to setup a configuration that consumes the jacocoReports variants and then depend on the variants of the sub projects:
// main build file
val jacocoReports by configurations.creating {
isCanBeResolved = true
isCanBeConsumed = false
attributes {
attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class, "jacocoReports"))
}
}
dependencies {
jacocoReports(project(":b:c"))
jacocoReports(project(":b:d:e"))
// other jacocoReports you want to consume
}
tasks.register<Copy>("aggregateJacocoReports") {
from(jacocoReports)
into(file("$buildDir/jacoco"))
}
As you can see, the jacocoReports configuration has the same usage attribute, so it can be used to resolve files that are in configurations with the same attribute. Then we need to define which project reports we want to consume. This is cone by defining project dependencies using the jacocoReports configuration. The last step is a simple copy task that copies the files into the build directory of the root project. So now when you call ./gradlew aggregateJacocoReports this task resolves all the files from the jacocoReports configuration, which in turn will create the jacoco report for all projects that are the root project has a dependency on.
Why is this better than cross configuration? If projects are not entangled by cross configuration and by tasks that copy stuff between projects, gradle can more efficiently schedule and parallelize the work that has to be done.
I have a created a minimal example that should help you to setup your project this way: https://github.com/britter/gradle-jacoco-aggregate. I have removed the android specific configuration to keep it simple, but I'm sure you will figure it out.

Gradle/Jenkins : Create a gradle file to direct to sub projects

I've got a directory with three android projects in it.
The MainDir looks like that :
/.gradle
/.git
/project1
/project2
/project3
.gitignore
.Jenkinsfile
.README.md
In jenkins I can't run a shell script during the build that launchs gradle tasks for eauch of those projects because he doesn't know these are projects (he says "no sub-project").
In a project dir it looks like :
/.gradle
/app
/build
/gradle
.gitignore
.build.gradle
.gradle.properties
.gradlew
Is there a way to make jenkins understand these are three projects he can launch gradle taks in ? Like creating a build.gradle file in the main directory doing that ?
Or should I just create 3 Jenkins items?

You could make three builds in jenkins but unless there is a need to build the libs seperately then it might just end up being extra effort. Sounds like what you really want is a multi project build [1]. A simple example could sit at the folder above your lib projects as two files, build.gradle and settings.gradle
The settings.gradle will define what projects are included in your build's scope.
For example given your project1, project2 and project3 example your settings.gradle may look like this.
rootProject.name = 'myRootProjectName'
// note the name is not required to match the actual path
include ":project1"
// but if the name is not the same as the path then we can just
// let gradle know where the project is expected
project(":project1").projectDir = new File(settingsDir, "pathToProject1")
include ":project2"
project(":project2").projectDir = new File(settingsDir, "pathToProject2")
include ":project3"
project(":project3").projectDir = new File(settingsDir, "pathToProject3")
//##### below would be instead of the code above, same thing just manual
// project setup vs letting gradle find the subprojects
// note sometimes you have lots of subprojects in that case it's sometimes
// easier to just use a little logic for finding and setting up the subprojects.
// don't use the code above ##### and below only use one or the other
// or you will have errors. The method below is the most scaleable since
// adding projects requires zero modifications to the root project
rootProject.name = 'myRootProjectName'
// set up a couple file filters to find the dirs we consider subprojects
FileFilter projectFilter = { File pathname ->
FileFilter gradleProjectFilter = { File file -> file.name == 'build.gradle' }
// add this folder if is a directory and that directory contains a build.gradle file
// here note `File#listFiles` is true if it's `size() > 0` due to
// groovy's concept of truth (details: http://groovy-lang.org/semantics.html#Groovy-Truth)
return pathname.isDirectory() && pathname.listFiles(gradleProjectFilter)
}
settingsDir.listFiles(projectFilter).each { dir ->
include ":$dir.name"
project(":$dir.name").projectDir = dir
}
now running gradle projects task should show the three submodules.
As for your build.gradle file you could specify some common properties to all the modules if needed or just leave the file blank, it must exist but can be empty. If you wanted to share some configurations then you might set up the build.gradle with something like this.
project.subprojects { Project subproject ->
// anything that is defined here will be executed before the subproject's build.gradle file
subproject.buildscript {
repositories {
jcenter()
// your private maven repo if needed
maven { url 'http://1.2.3.4:8081/nexus/content/repositories/release' }
}
dependencies {
// some plugin that is now available to be applied in any subproject
classpath 'my.sweet.gradle:plugin:0.1'
}
}
subproject.afterEvaluate {
// this block is executed after the subproject's build.gradle file
if (project.tasks.withType(org.gradle.jvm.tasks.Jar)) {
// for example you might want to set the manifest for each subproject
manifest {
attributes 'Implementation-Title': "Lib $subproject.name",
'Implementation-Version': version
}
}
}
}
[1] https://docs.gradle.org/current/userguide/multi_project_builds.html

Android/Gradle: Add an asset after the compile phase

I have a task that generates a metadata file based off the compiled classes in an Android Gradle build. I can get it to run by executing it after the compile task:
android.applicationVariants.all { variant ->
def variantName = variant.name.capitalize()
def compileSourcesTaskName = "compile${variantName}Sources"
def compileSourcesTask = project.tasks.findByName(compileSourcesTaskName)
compileSourcesTask.finalizedBy "myTaskThatGeneratesAssets"
}
Unfortunately, Android has already processed the assets at this point. The new file won't get included in the assembled APK.
An answer to a similar question suggests calling aapt add to add the file to the APK before alignment/signing. This seems like it could work, but the post doesn't go into implementation details. The code to call aapt in the Android Gradle plugin looks fairly complicated for a build script, and I'm not sure how to get access to the IAndroidTarget it references.
I'd appreciate suggestions on how to implement this, or any other solutions!

Okay, here's what I ended up with. It makes two assumptions that may break on later versions of the android gradle plugin (I'm using 1.3.0):
The path to the platform tools is ${android.getSdkDirectory().getAbsolutePath()}/build-tools/${android.buildToolsVersion}/
The path to the intermediate (resources) APK is ${buildDir}/intermediates/res/resources-${variant.baseName}.ap_
So long as those are true, this should generate a task to add the new asset file using aapt after the resources APK has already been built:
def overlayDir = ... // path to a resources overlay directory that contains "assets/my.json"
def addMyAssetTaskName = "add${variantName}MyAsset"
task "${addMyAssetTaskName}" (type: Exec) {
dependsOn myTaskThatGeneratesAssets
workingDir overlayDir
def aaptCommand = "${android.getSdkDirectory().getAbsolutePath()}/build-tools/${android.buildToolsVersion}/aapt"
def apkPath = "${buildDir}/intermediates/res/resources-${variant.baseName}.ap_"
commandLine aaptCommand, 'add', apkPath, "assets/my.json"
}
Then I use finalizedBy like in the question above to addMyAssetTaskName.

Custom Class Loading in Dalvik with Gradle (Android New Build System)

As per the introduction of Custom Class Loading in Dalvik by Fred Chung on the Android Developers Blog:
The Dalvik VM provides facilities for developers to perform custom
class loading. Instead of loading Dalvik executable (“dex”) files from
the default location, an application can load them from alternative
locations such as internal storage or over the network.
However, not many developers have the need to do custom class loading. But those who do and follow the instructions on that blog post, might have some problems mimicking the same behavior with Gradle, the new build system for Android introduced in Google I/O 2013.
How exactly one can adapt the new build system to perform the same intermediary steps as in the old (Ant based) build system?

My team and I recently reached the 64K method references in our app, which is the maximum number of supported in a dex file. To get around this limitation, we need to partition part of the program into multiple secondary dex files, and load them at runtime.
We followed the blog post mentioned in the question for the old, Ant based, build system and everything was working just fine. But we recently felt the need to move to the new build system, based on Gradle.
This answer does not intend to replace the full blog post with a complete example. Instead, it will simply explain how to use Gradle to tweak the build process and achieve the same thing. Please note that this is probably just one way of doing it and how we are currently doing it in our team. It doesn't necessarily mean it's the only way.
Our project is structured a little different and this example works as an individual Java project that will compile all the source code into .class files, assemble them into a single .dex file and to finish, package that single .dex file into a .jar file.
Let's start...
In the root build.gradle we have the following piece of code to define some defaults:
ext.androidSdkDir = System.env.ANDROID_HOME
if(androidSdkDir == null) {
Properties localProps = new Properties()
localProps.load(new FileInputStream(file('local.properties')))
ext.androidSdkDir = localProps['sdk.dir']
}
ext.buildToolsVersion = '18.0.1'
ext.compileSdkVersion = 18
We need the code above because although the example is an individual Java project, we still need to use components from the Android SDK. And we will also be needing some of the other properties later on... So, on the build.gradle of the main project, we have this dependency:
dependencies {
compile files("${androidSdkDir}/platforms/android-${compileSdkVersion}/android.jar")
}
We are also simplifying the source sets of this project, which might not be necessary for your project:
sourceSets {
main {
java.srcDirs = ['src']
}
}
Next, we change the default configuration of the build-in jar task to simply include the classes.dex file instead of all .class files:
configure(jar) {
include 'classes.dex'
}
Now we need to have new task that will actually assemble all .class files into a single .dex file. In our case, we also need to include the Protobuf library JAR into the .dex file. So I'm including that in the example here:
task dexClasses << {
String protobufJarPath = ''
String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''
configurations.compile.files.find {
if(it.name.startsWith('protobuf-java')) {
protobufJarPath = it.path
}
}
exec {
commandLine "${androidSdkDir}/build-tools/${buildToolsVersion}/dx${cmdExt}", '--dex',
"--output=${buildDir}/classes/main/classes.dex",
"${buildDir}/classes/main", "${protobufJarPath}"
}
}
Also, make sure you have the following import somewhere (usually at the top, of course) on your build.gradle file:
import org.apache.tools.ant.taskdefs.condition.Os
Now we must make the jar task depend on our dexClasses task, to make sure that our task is executed before the final .jar file is assembled. We do that with a simple line of code:
jar.dependsOn(dexClasses)
And we're done... Simply invoke Gradle with the usual assemble task and your final .jar file, ${buildDir}/libs/${archivesBaseName}.jar will contain a single classes.dex file (besides the MANIFEST.MF file). Just copy that into your app assets folder (you can always automate that with Gradle as we've done but that is out of scope of this question) and follow the rest of the blog post.
If you have any questions, just shout in the comments. I'll try to help to the best of my abilities.

The Android Studio Gradle plugin now provides native multidex support, which effectively solves the Android 65k method limit without having to manually load classes from a jar file, and thus makes Fred Chung's blog obsolete for that purpose. However, loading custom classes from a jar file at runtime in Android is still useful for the purpose of extensibility (e.g. making a plugin framework for your app), so I'll address that usage scenario below:
I have created a port of the original example app on Fred Chung's blog to Android Studio on my github page over here using the Android library plugin rather than the Java plugin. Instead of trying to modify the existing dex process to split up into two modules like in the blog, I've put the code which we want to go into the jar file into its own module, and added a custom task assembleExternalJar which dexes the necessary class files after the main assemble task has finished.
Here is relevant part of the build.gradle file for the library. If your library module has any dependencies which are not in the main project then you will probably need to modify this script to add them.
apply plugin: 'com.android.library'
// ... see github project for the full build.gradle file
// Define some tasks which are used in the build process
task copyClasses(type: Copy) { // Copy the assembled *.class files for only the current namespace into a new directory
// get directory for current namespace (PLUGIN_NAMESPACE = 'com.example.toastlib')
def namespacePath = PLUGIN_NAMESPACE.replaceAll("\\.","/")
// set source and destination directories
from "build/intermediates/classes/release/${namespacePath}/"
into "build/intermediates/dex/${namespacePath}/"
// exclude classes which don't have a corresponding .java entry in the source directory
def remExt = { name -> name.lastIndexOf('.').with {it != -1 ? name[0..<it] : name} }
eachFile {details ->
def thisFile = new File("${projectDir}/src/main/java/${namespacePath}/", remExt(details.name)+".java")
if (!(thisFile.exists())) {
details.exclude()
}
}
}
task assembleExternalJar << {
// Get the location of the Android SDK
ext.androidSdkDir = System.env.ANDROID_HOME
if(androidSdkDir == null) {
Properties localProps = new Properties()
localProps.load(new FileInputStream(file('local.properties')))
ext.androidSdkDir = localProps['sdk.dir']
}
// Make sure no existing jar file exists as this will cause dx to fail
new File("${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar").delete();
// Use command line dx utility to convert *.class files into classes.dex inside jar archive
String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''
exec {
commandLine "${androidSdkDir}/build-tools/${BUILD_TOOLS_VERSION}/dx${cmdExt}", '--dex',
"--output=${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar",
"${buildDir}/intermediates/dex/"
}
copyJarToOutputs.execute()
}
task copyJarToOutputs(type: Copy) {
// Copy the built jar archive to the outputs folder
from 'build/intermediates/dex/'
into 'build/outputs/'
include '*.jar'
}
// Set the dependencies of the build tasks so that assembleExternalJar does a complete build
copyClasses.dependsOn(assemble)
assembleExternalJar.dependsOn(copyClasses)
For more detailed information see the full source code for the sample app on my github.

See my answer over here. The key points are:
Use the additionalParameters property on the dynamically created dexCamelCase tasks to pass --multi-dex to dx and create multiple dex files.
Use the multidex class loader to use the multiple dex files.