I have a multi-module android project with the structure:
:a
:b
:c
:d
:e
I'm trying to run a jacoco report on module :b so that it runs on :b, :c, :d, and :e without running :a. I want all of the xml reports to be in a common folder with names of their project.xml (e.g. b.xml, c.xml, etc.) I have a pretty standard jacoco setup
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {
reports {
xml.enabled = true
xml.destination = file(allTestCoverageDir + project.name + ".xml")
html.enabled = true
}
def fileFilter = [
'**/R.class',
'**/R$*.class',
'**/BuildConfig.*',
'**/Manifest*.*',
'**/*Test*.*',
'android/**/*.*',
//Dagger 2
'**/*Dagger*Component*.*',
'**/*Module.*',
'**/*Module$*.*',
'**/*MembersInjector*.*',
'**/*_Factory*.*',
'**/*Provide*Factory*.*',
]
def kotlinDebug = [fileTree(dir: "${project.buildDir}/tmp/kotlin-classes/debug", excludes: fileFilter)]
def mainSrc = files([
"$project.projectDir/src/main/java",
"$project.projectDir/src/main/kotlin"
])
sourceDirectories = files([mainSrc])
classDirectories = files(kotlinDebug)
executionData = fileTree(dir: project.buildDir, includes: [
'jacoco/testDebugUnitTest.exec', 'outputs/code-coverage/connected/*coverage.ec'
])
}
But when I try to loop through subprojects in a doLast block, the doLast block never runs and trying to access subprojects before that also shows that :a has no subprojects.
Edit I am able to run these for each sub project with ./gradlew b:jacocoTestReport or ./gradlew c:jacocoTestReport and all the reports and in a folder with the correct names. But as my project grows I don't want to have to run dozens of commands (one for each module) I want a single command ./gradlew b:jacocoTestReport (or something similar) which runs for b and it's subtree
As far as I understand at the moment you're using allprojects {} to configure all of the sub projects. Although this has been the canonical to configure a group of projects in the past, it is now discouraged. Furthermore projects should use publications to interface with each other instead of copying files across project boundaries. So you need to do two things:
Instead of configuring the sub projects from the root root project, you should create a plugin to configure jacoco and create configuration that will hold the reports.
To do this, create a pre-compiled script plugin in your project. The idea is to have a kotlin build script in the buildSrc project and create a Gradle plugin from that file on the fly. So you should move the logic that configures jacoco to the file buildSrc/src/main/kotlin/jacoco-conventions.gradle.kts:
plugins {
jacoco
}
val jacocoTestReport by tasks.getting(JacocoReport::class) {
// jacoco configuration
}
configurations.create("jacocoReports") {
isCanBeResolved = false
isCanBeConsumed = true
attributes {
attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class, "jacocoReports"))
}
outgoing.artifact(jacocoTestReport.reports.xml.destination) {
builtBy(jacocoTestReport)
}
}
The last part creates a new configuration in the project the pre-compiled script plugin is applied to. This configuration uses the xml destination file which is builtBy the jacoco report task as an outgoing artifact. The important part here is the USAGE_ATTRIBUTE because we will need this later on to consume the files.
The precompiled script plugin can now be applied in the projects where you want to gather jacoco metrics by:
// for example in c/build.gradle.kts
plugins {
`jacoco-conventions`
}
Now you have configured the sub projects to put the Jacoco xml reports into configurations with usage attribute jacocoReports.
In the root project create a task that copies the report from the configuration.
To do this we need to setup a configuration that consumes the jacocoReports variants and then depend on the variants of the sub projects:
// main build file
val jacocoReports by configurations.creating {
isCanBeResolved = true
isCanBeConsumed = false
attributes {
attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class, "jacocoReports"))
}
}
dependencies {
jacocoReports(project(":b:c"))
jacocoReports(project(":b:d:e"))
// other jacocoReports you want to consume
}
tasks.register<Copy>("aggregateJacocoReports") {
from(jacocoReports)
into(file("$buildDir/jacoco"))
}
As you can see, the jacocoReports configuration has the same usage attribute, so it can be used to resolve files that are in configurations with the same attribute. Then we need to define which project reports we want to consume. This is cone by defining project dependencies using the jacocoReports configuration. The last step is a simple copy task that copies the files into the build directory of the root project. So now when you call ./gradlew aggregateJacocoReports this task resolves all the files from the jacocoReports configuration, which in turn will create the jacoco report for all projects that are the root project has a dependency on.
Why is this better than cross configuration? If projects are not entangled by cross configuration and by tasks that copy stuff between projects, gradle can more efficiently schedule and parallelize the work that has to be done.
I have a created a minimal example that should help you to setup your project this way: https://github.com/britter/gradle-jacoco-aggregate. I have removed the android specific configuration to keep it simple, but I'm sure you will figure it out.
Related
I want to upload NDK symbols on every build i do,
Under my Android inside gradle i use to have:
applicationVariants.all { variant ->
def variantName = variant.name.capitalize()
println("symbols will be added on varinat ${variantName}")
def task = project.task("ndkBuild${variantName}")
task.finalizedBy project.("uploadCrashlyticsSymbolFile${variantName}")
}
this does not compile anymore since i moved to FireBase :
Could not get unknown property 'uploadCrashlyticsSymbolFile
I don't see this task running.
I basiclly need this task to run on every build:
./gradlew app:assembleBUILD_VARIANT\
app:uploadCrashlyticsSymbolFileBUILD_VARIANT
Add this at the bottom of app's build.gradle outside android { ... } block.
afterEvaluate {
android.applicationVariants.all { variant ->
def variantName = variant.name.capitalize()
println("symbols will be added on variant ${variantName}")
def task = tasks.findByName("assemble${variantName}")
def uploader = "uploadCrashlyticsSymbolFile${variantName}"
// This triggers after task completion
task?.finalizedBy(uploader)
// This ensures ordering
task?.mustRunAfter(uploader)
}
}
You can try without afterEvaluate block. It should still work.
Likely you'd need to use Firebase App Distribution, which permits automatic upload of release build artifacts - and if you have the artifact with the matching debug symbols, they could actually be used - without the matching assembly, the symbols are somewhat irrelevant.
Number 1 is obviously a wrongful assumption, because the documentation clearly states:
./gradlew app:assembleBUILD_VARIANT app:uploadCrashlyticsSymbolFileBUILD_VARIANT
And this is already answered here.
In order to always upload, one can create a task dependency:
assembleRelease.finalizedBy uploadCrashlyticsSymbolFileRelease
This may require setting unstrippedNativeLibsDir and strippedNativeLibsDir.
I have a situation where in an Android project with instrumentation tests I have all of the production code precompiled and ready to be installed as an .apk (a React Native environment).
Whenever I run instrumentation tests, I initially build the AndroidTest .apk using Gradle by running:
./gradlew assembleDebugAndroidTest -DtestBuildType=debug
(i.e. in a pretty standard way).
Trouble is that despite explicitly specifying only the xxxAndroidTest task, all of the production code assembly Gradle tasks are run as well. This is an extreme time waster to me since - as I explained, the production apk is already there, and thus code compilation (and packaging, signing, etc.) is scarce.
In essence, I have no dependency in production code from the instrumentation code -- even the ActivityTestRule I use is created dynamically and isn't directly bound to my main activity:
Intent launchIntent = context.getPackageManager().getLaunchIntentForPackage(context.getPackageName());
ResolveInfo resolveInfo = context.getPackageManager().resolveActivity(launchIntent, 0);
Class<?> activityClass = Class.forName(resolveInfo.activityInfo.name);
ActivityTestRule<?> activityTestRule = new ActivityTestRule(activityClass, false, false);
Question is: How can I isolate / restrict Gradle's work so it would only include test-related tasks? I even tried inspecting the tasks tree using this Gradle plugin, but couldn't find a clear place to "cut the tree" down.
Well so far I've come up with this (heuristic) solution, that does 2 things:
I noticed that most of the time that goes to waste is due to sub-projects that are not needed for the job. Therefore, the solution provides an easy way to exclude implementations from test building.
Out of the tasks remaining in the list, still - the plugin iteratively force-disables tasks that are not related but run nonetheless.
It boils down to this helper Gradle script:
// turbo-test-apk.gradle
def isEnabled = System.getProperty('TURBO_TEST_APK') != null
project.ext.dependenciesExcludeTest = { depsClosure ->
if (!isEnabled) {
dependencies(depsClosure)
}
}
gradle.taskGraph.whenReady { graph ->
if (isEnabled) {
def disabledTasks = new ArrayList<Task>(graph.allTasks.size())
[/.*JsAndAssets.*/, /package.*Release/, /package.*Debug/, /compile.*/, /.*[Pp]roguard.*/, /.*[Nn]ew[Rr]elic.*/, /.*AndroidTest.*/].forEach { regex ->
graph.allTasks.findAll { it.name ==~ regex }.forEach({ task ->
disabledTasks.add(task)
task.enabled = false
})
}
graph.allTasks.findAll { it.name ==~ /.*AndroidTest.*/ }.forEach({ task ->
task.enabled = true
})
println '--- Turbo test build: task scanning ---'
disabledTasks.forEach { task ->
if (!task.enabled) {
println 'Force-skipping ' + task
}
}
println '---------------------------------------'
}
}
Namely, the dependenciesExcludeTest enabled the exclusion of unwanted subprojects, and the task-graph-ready callback does the disabling. NOTE that the regex list is custom made, and is not generic. It makes sense for my project as react native projects have a heavy-weight JS-bundling tasks called bundleJsAndAssets, and I also have new relic installed. Nevertheless, this can be easily tailored to any project.
Also, the app.gradle looks something like this:
apply plugin: 'com.android.application'
apply from: './turbo-test-apk.gradle'
dependencies {
implementation "org.jetbrains.kotlin:$kotlin_stdlib:$kotlinVersion"
implementation "com.android.support:support-v4:$supportLibraryVersion"
implementation "com.android.support:appcompat-v7:$supportLibraryVersion"
// etc.
}
// These will be excluded when executing test-only mode
dependenciesExcludeTest {
implementation project(':#react-native-community_async-storage')
implementation project(':any-unneeded-sub-project')
}
So when gradle is run like this (i.e. with a custom TURBO_TEST_APK property):
./gradlew assembleDebugAndroidTest -DtestBuildType=debug -DTURBO_TEST_APK
the script will apply its work and reduce the overall build time.
This solution isn't optimal: tricky to maintain, doesn't omit all of the unnecessary work. I'd be very happy to see more effective solutions.
We have an Android project, and we're using Powermock for some of our test cases and Jacoco for coverage report. We noticed that some our classes are returning as 0% coverage although they are indeed covered. We also observed the message below for affected classes.
"Classes ... do no match with execution data."
A few searches online show that Powermock and Jacoco don't play well and that Offline Instrumentation is a possible workaround.
Has anyone used gradle Offline Instrumentation script for android projects before?
In hindsight, I guess this can be solved with enough android experience and online perusing. However, I was (and still am) relatively new to Android, gradle, and groovy when this fell on my lap, so I'm writing this for the next me :-D
WHAT IS HAPPENING IN A NUTSHELL (excerpt from a jacoco forum)
Source file is compiled into non-instrumented class file
Non-instrumented class file is instrumented (either pre-instrumented offline, or automatically at runtime by Java agent)
Execution of instrumented classes collected into exec file
report decorates source files with information obtained from analysis of exec file and original non-instrumented class files
Message "Classes ... do no match with execution data." during generation of report means that class files used for generation of report are not the same as classes prior to instrumentation.
SOLUTION
The Jacoco Offline Instrumentation page provides the main steps that should occur for offline instrumentation in this excerpt:
For such scenarios class files can be pre-instrumented with JaCoCo,
for example with the instrument Ant task. At runtime the
pre-instrumented classes needs be on the classpath instead of the
original classes. In addition jacocoagent.jar must be put on the
classpath.
The script below does exactly that:
apply plugin: 'jacoco'
configurations {
jacocoAnt
jacocoRuntime
}
jacoco {
toolVersion = "0.8.1"
}
def offline_instrumented_outputDir = "$buildDir.path/intermediates/classes-instrumented/debug"
tasks.withType(Test) {
jacoco.includeNoLocationClasses = true
}
def coverageSourceDirs = [
'src/main/java'
]
task jacocoTestReport(type: JacocoReport, dependsOn: "test") {
group = "Reporting"
description = "Generate Jacoco coverage reports"
classDirectories = fileTree(
dir: 'build/intermediates/classes/debug',
excludes: ['**/R.class',
'**/R$*.class',
'**/BuildConfig.*',
'**/MainActivity.*']
)
sourceDirectories = files(coverageSourceDirs)
executionData = files('build/jacoco/testDebugUnitTest.exec')
}
jacocoTestReport {
reports {
xml.enabled true
html.enabled true
html.destination file("build/test-results/jacocoHtml")
}
}
/* This task is used to create offline instrumentation of classes for on-the-fly instrumentation coverage tool like Jacoco. See jacoco classId
* and Offline Instrumentation from the jacoco site for more info.
*
* In this case, some classes mocked using PowerMock were reported as 0% coverage on jacoco & Sonarqube. The issue between PowerMock and jacoco
* is well documented, and a possible solution is offline Instrumentation (not so well documented for gradle).
*
* In a nutshell, this task:
* - Pre-instruments the original *.class files
* - Puts the instrumented classes path at the beginning of the task's classpath (for report purposes)
* - Runs test & generates a new exec file based on the pre-instrumented classes -- as opposed to on-the-fly instrumented class files generated by jacoco.
*
* It is currently not implemented to run prior to any other existing tasks (like test, jacocoTestReport, etc...), therefore, it should be called
* explicitly if Offline Instrumentation report is needed.
*
* Usage: gradle clean & gradle createOfflineInstrTestCoverageReport & gradle jacocoTestReport
* - gradle clean //To prevent influence from any previous task execution
* - gradle createOfflineInstrTestCoverageReport //To generate *.exec file from offline instrumented class
* - gradle jacocoTestReport //To generate html report from newly created *.exec task
*/
task createOfflineTestCoverageReport(dependsOn: ['instrument', 'testDebugUnitTest']) {
doLast {
ant.taskdef(name: 'report',
classname: 'org.jacoco.ant.ReportTask',
classpath: configurations.jacocoAnt.asPath)
ant.report() {
executiondata {
ant.file(file: "$buildDir.path/jacoco/testDebugUnitTest.exec")
}
structure(name: 'Example') {
classfiles {
fileset(dir: "$project.buildDir/intermediates/classes/debug")
}
sourcefiles {
fileset(dir: 'src/main/java')
}
}
//Uncomment if we want the task to generate jacoco html reports. However, the current script does not exclude files.
//An alternative is to used jacocoTestReport after this task finishes
//html(destdir: "$buildDir.path/reports/jacocoHtml")
}
}
}
/*
* Part of the Offline Instrumentation process is to add the jacoco runtime to the class path along with the path of the instrumented files.
*/
gradle.taskGraph.whenReady { graph ->
if (graph.hasTask(instrument)) {
tasks.withType(Test) {
doFirst {
systemProperty 'jacoco-agent.destfile', buildDir.path + '/jacoco/testDebugUnitTest.exec'
classpath = files(offline_instrumented_outputDir) + classpath + configurations.jacocoRuntime
}
}
}
}
/*
* Instruments the classes per se
*/
task instrument(dependsOn:'compileDebugUnitTestSources') {
doLast {
println 'Instrumenting classes'
ant.taskdef(name: 'instrument',
classname: 'org.jacoco.ant.InstrumentTask',
classpath: configurations.jacocoAnt.asPath)
ant.instrument(destdir: offline_instrumented_outputDir) {
fileset(dir: "$buildDir.path/intermediates/classes/debug")
}
}
}
Usage
The script can be copied into a separate file. For instance: jacoco.gradle
Reference the jacoco file in your build.gradle. For instance: apply from: jacoco.gradle
Ensure proper dependencies: jacocoAnt 'org.jacoco:org.jacoco.ant:0.8.1:nodeps'
In command line run: gradle clean & gradle createOfflineTestCoverageReport & gradle jacocoTestReport
gradle clean will wipe out any previous gradle execution artifacts
gradle createOfflineTestCoverageReport will create offline instrumentation, change order of classpath, generate .exec file
gradle jacocoTestReport will run test and generate jacoco report based on previously generated .exec file
Feeling Lost?
I've put together a github Jacoco Powermock Android project with sample scripts to reproduce and fix the issue. It also contains more information about the solution.
REFERENCE
https://github.com/powermock/powermock/wiki/Code-coverage-with-JaCoCo
https://www.jacoco.org/jacoco/trunk/doc/classids.html
https://www.jacoco.org/jacoco/trunk/doc/offline.html
https://github.com/powermock/powermock-examples-maven/tree/master/jacoco-offline
https://automated-testing.info/t/jacoco-offline-instrumentations-for-android-gradle/20121
https://stackoverflow.com/questions/41370815/jacoco-offline-instrumentation-gradle-script/42238982#42238982
https://groups.google.com/forum/#!msg/jacoco/5IqM4AibmT8/-x5w4kU9BAAJ
I'm trying to setup sonarqube reporting in my Android project. I currently have trouble with showing all test classes in the sonar UI, the coverage is shown in percentages and currently only the unit test from app/src/test/ are shown as Unit Tests.
My project has a test folder app/src/test/ which contains unit test and I have a androidTest folder app/src/androidTest/ which contain android unit, integration and UI tests. When I run all the tests via gradle the android-gradle plugin generates build/jacoco/testDebugUnitTest.exec and build/test-results/debug/TEST-*Test.xml which contains the jacoco results and coverage report for the unit test in the test folder. Also the android-gradle plugin generates build/outputs/code-coverage/connected/coverage.ec and build/outputs/androidTest-results/connected/TEST-*Test.xml contain the results and coverage reports from the androidTest folder
In my build.gradle I can specify the properties for the sonar plugin.
sonarqube {
properties {
property "sonar.sources", "src/main/java,src/main/res"
property "sonar.tests", "src/test/java,src/androidTest/java"
property "sonar.java.coveragePlugin", "jacoco"
property "sonar.jacoco.reportPath", "${project.buildDir}/jacoco/testDebugUnitTest.exec"
property 'sonar.jacoco.itReportPath', "${project.buildDir}/outputs/code-coverage/connected/coverage.ec"
property "sonar.junit.reportsPath", "${project.buildDir}/test-results/debug" // path to junit reports
}
}
With sonar.junit.reportsPath I can specify which xml report is sent to the sonarqube server. When I change it to build/outputs/androidTest-results/connected I get the androidTest shown as Unit Test on the dashboard. Is there a way to make the sonar plugin look in both directories or merge the results together?
Until https://jira.sonarsource.com/browse/SONAR-4101 is fixed, the only option you have is to write a task that copies your test result files into a single place and configure that as sonar.junit.reportsPath, like this:
task combineTestResultsForSonarqube {
group = "Reporting"
def modules = ["app", "and", "other", "modules"];
doLast {
modules.each { module ->
File combined = file("${module}/build/combined-test-results")
if (combined.exists()) {
combined.deleteDir()
}
combined.mkdirs();
def testDirs = [file("${module}/build/test-results/debug/"),
file("${module}/build/outputs/androidTest-results/connected/")];
testDirs.each { testDir ->
if (!testDir.exists()) {
logging.captureStandardOutput LogLevel.WARN
println "WARNING: ignoring non-existant ${testDir.path}"
return;
}
files(testDir.listFiles()).each { file ->
new File(combined, file.getName()) << file.text
}
}
}
}
}
Paths of course have to be adapted when you have flavors in your build.
Since the release of 'com.android.tools.build:gradle:1.1.0' I'm moving most of my java test code from androidTest to the test folder because the JVM tests are a lot faster. But I cannot move all tests. I really need the device tests because of some ContentProvider stuff.
I've had 100% code coverage before I started migrating. When I'm currently running the jacoco code coverage I get 40% for the androidTest folder and 71% for the test folder. My code is 100% tested but I have no report proofing this.
Is there a way to combine both reports? I found JacocoMerge but couldn't get it to work.
Here is the output of the androidTest folder: build/outputs/reports/coverage/debug/index.html
And here the output of the test folder
build/reports/jacoco/generateJacocoTestReports/html/index.html
generated with this gradle task:
def coverageSourceDirs = [
'../library/src/main/java'
]
task generateJacocoTestReports(type: JacocoReport, dependsOn: "test") {
group = "Reporting"
description = 'Generate Jacoco Robolectric unit test coverage reports'
classDirectories = fileTree(
dir: '../library/build/intermediates/classes/debug',
excludes: ['**//*R.class',
'**//*R$*.class',
'***/*//*$ViewInjector*.*',
'**//*BuildConfig.*',
'**//*Manifest*.*']
)
sourceDirectories = files(coverageSourceDirs)
additionalSourceDirs = files(coverageSourceDirs)
executionData = files('../library/build/jacoco/testDebug.exec')
}
Not sure if you still need this, but I recently published Gradle plugin which might help you with that: https://github.com/paveldudka/JacocoEverywhere
There is also the gradle plugin https://github.com/palantir/gradle-jacoco-coverage that acording to docs can do the job, too.
I haven-t tried it for one submodul with two different test-parts but it works well for merging the testparts of to two submoduls.
See Gradle jacoco coverage report with more than one submodule(s)? for details
JacocoMerge task can be used to merge 2 or more jacoco execution data.
Below task can be added to the root gradle file and on successful execution of this task, merged execution data can be found under root build directory. (build/jacoco/mergeJacocoReport.exec)
evaluationDependsOnChildren()
//Missing this might be a problem in fetching JacocoReport tasks from sub-modules.
task mergeJacocoReport(type: org.gradle.testing.jacoco.tasks.JacocoMerge) {
group "Jacoco Report"
description "Merge Jacoco Code Coverage Report"
def executionFiles = fileTree("$rootProject.rootDir", {
includes = ['**/*.exec']
})
setExecutionData(executionFiles)
}
subprojects.each { $project ->
def tasks = $project.tasks.withType(JacocoReport)
if (tasks != null) {
mergeJacocoReport.dependsOn << tasks
}
}
In case you use Jenkins with the JaCoCo plugin you can just configure all jacoco.exec and emma.ec files in "Path to exec files" to have a combined coverage reported.
connectedAndroidTest will result in emma.ec files somewhere in "outputs" by default.