I'm trying to setup sonarqube reporting in my Android project. I currently have trouble with showing all test classes in the sonar UI, the coverage is shown in percentages and currently only the unit test from app/src/test/ are shown as Unit Tests.
My project has a test folder app/src/test/ which contains unit test and I have a androidTest folder app/src/androidTest/ which contain android unit, integration and UI tests. When I run all the tests via gradle the android-gradle plugin generates build/jacoco/testDebugUnitTest.exec and build/test-results/debug/TEST-*Test.xml which contains the jacoco results and coverage report for the unit test in the test folder. Also the android-gradle plugin generates build/outputs/code-coverage/connected/coverage.ec and build/outputs/androidTest-results/connected/TEST-*Test.xml contain the results and coverage reports from the androidTest folder
In my build.gradle I can specify the properties for the sonar plugin.
sonarqube {
properties {
property "sonar.sources", "src/main/java,src/main/res"
property "sonar.tests", "src/test/java,src/androidTest/java"
property "sonar.java.coveragePlugin", "jacoco"
property "sonar.jacoco.reportPath", "${project.buildDir}/jacoco/testDebugUnitTest.exec"
property 'sonar.jacoco.itReportPath', "${project.buildDir}/outputs/code-coverage/connected/coverage.ec"
property "sonar.junit.reportsPath", "${project.buildDir}/test-results/debug" // path to junit reports
}
}
With sonar.junit.reportsPath I can specify which xml report is sent to the sonarqube server. When I change it to build/outputs/androidTest-results/connected I get the androidTest shown as Unit Test on the dashboard. Is there a way to make the sonar plugin look in both directories or merge the results together?
Until https://jira.sonarsource.com/browse/SONAR-4101 is fixed, the only option you have is to write a task that copies your test result files into a single place and configure that as sonar.junit.reportsPath, like this:
task combineTestResultsForSonarqube {
group = "Reporting"
def modules = ["app", "and", "other", "modules"];
doLast {
modules.each { module ->
File combined = file("${module}/build/combined-test-results")
if (combined.exists()) {
combined.deleteDir()
}
combined.mkdirs();
def testDirs = [file("${module}/build/test-results/debug/"),
file("${module}/build/outputs/androidTest-results/connected/")];
testDirs.each { testDir ->
if (!testDir.exists()) {
logging.captureStandardOutput LogLevel.WARN
println "WARNING: ignoring non-existant ${testDir.path}"
return;
}
files(testDir.listFiles()).each { file ->
new File(combined, file.getName()) << file.text
}
}
}
}
}
Paths of course have to be adapted when you have flavors in your build.
Related
I have local and integrated test cases written for my android project. Using Kotlin(1.4.21) Robolectric(4.5.1), sonar(2.7.1), Jacoco(maven plugin 0.8.2)
The problem is that the Sonar and Jacoco is not considering androidTest(integration test case) written in Kotlin for code coverage
However sonar is showing correct coverage for other test cases like-
java unit test cases -> working
koltin unit test case -> working
java integrated test cases -> working
kotlin integrated test cases -> NOT WORKING
Although I have checked the paths I have set for sonar and it's all correct.
properties['sonar.java.binaries'] = files("${buildDir}/intermediates/javac/universalDebug/classes")
properties["sonar.java.binaries"] += files("${buildDir}/tmp/kotlin-classes/universalDebug/")
properties['sonar.java.test.binaries'] = files("${buildDir}/intermediates/javac/universalDebugAndroidTest/classes")
properties['sonar.java.test.binaries'] += files("${buildDir}/tmp/kotlin-classes/universalDebugAndroidTest/")
I have gone through other stackoverflow questions but didn't find same problem. So, I'm unable to find out the issue why sonar is not showing coverage for my integrated test cases written in Kotlin.
Thanks in Advance
UPDATE
within adroidTest folder > I have further 2 packages.
MyApplicationTest> src> com > pkgA
> pkgB
It's considering the Tests files present in pkgA but not the other. I have recently created this pkgB
What could be the possible reason for this? Do I have update some path somewhere?
You might need to do the following
tasks.withType(Test) {
jacoco.includeNoLocationClasses = true
}
Note that there are some issues with Java 11 that might fail your tests so you might want to also exclude jdk.internal as follows
tasks.withType(Test) {
jacoco.includeNoLocationClasses = true
excludes = ['jdk.internal.*']
}
Or a little bit verbose option but works:
subprojects {
pluginManager.withPlugin("com.android.library"){
android.testOptions.unitTests.all {
jacoco {
includeNoLocationClasses = true
excludes = ['jdk.internal.*']
}
}
}
pluginManager.withPlugin("com.android.application"){
android.testOptions.unitTests.all {
jacoco {
includeNoLocationClasses = true
excludes = ['jdk.internal.*']
}
}
}
apply plugin: 'jacoco'
}
I suggest you also upgrade your jacoco and sonar plugin versions if possible
I am trying to integrate code coverage in gradle Kotlin as suggested in https://techblog.tbauctions.com/kotlin-azure-and-code-coverage/ but it's not working, below are the project files.
build.gradle.kts
plugins {
jacoco
}
val test by tasks.getting(Test::class) {
useJUnitPlatform { }
}
tasks.withType(JacocoReport::class.java).all {
reports {
xml.isEnabled = true
xml.destination = File("$buildDir/reports/jacoco/report.xml")
}
}
tasks.withType<Test> {
jacoco {
toolVersion = "0.8.3"
reportsDir = file("$buildDir/reports/jacoco")
}
finalizedBy("jacocoTestReport")
}
Error - Task with name 'test' not found in project ':app'
Also tried https://docs.gradle.org/current/userguide/jacoco_plugin.html
How to generate and publish Code Coverage in gradle kotlin using Azure DevOps?
According to the error message:
Error - Task with name 'test' not found in project ':app'
It seems the task called test is not provided in the project. When using a JVM language plugin like Java in Gradle, a task called test is automatically provided. This task runs all tests under src/test by default.
The Java plugin adds a number of tasks to your project, including the test task:
The Java Plugin:
So, make sure you have add plugin Java in Gradle.
You could check this document for some more details.
Hope this helps.
I have a multi-module android project with the structure:
:a
:b
:c
:d
:e
I'm trying to run a jacoco report on module :b so that it runs on :b, :c, :d, and :e without running :a. I want all of the xml reports to be in a common folder with names of their project.xml (e.g. b.xml, c.xml, etc.) I have a pretty standard jacoco setup
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {
reports {
xml.enabled = true
xml.destination = file(allTestCoverageDir + project.name + ".xml")
html.enabled = true
}
def fileFilter = [
'**/R.class',
'**/R$*.class',
'**/BuildConfig.*',
'**/Manifest*.*',
'**/*Test*.*',
'android/**/*.*',
//Dagger 2
'**/*Dagger*Component*.*',
'**/*Module.*',
'**/*Module$*.*',
'**/*MembersInjector*.*',
'**/*_Factory*.*',
'**/*Provide*Factory*.*',
]
def kotlinDebug = [fileTree(dir: "${project.buildDir}/tmp/kotlin-classes/debug", excludes: fileFilter)]
def mainSrc = files([
"$project.projectDir/src/main/java",
"$project.projectDir/src/main/kotlin"
])
sourceDirectories = files([mainSrc])
classDirectories = files(kotlinDebug)
executionData = fileTree(dir: project.buildDir, includes: [
'jacoco/testDebugUnitTest.exec', 'outputs/code-coverage/connected/*coverage.ec'
])
}
But when I try to loop through subprojects in a doLast block, the doLast block never runs and trying to access subprojects before that also shows that :a has no subprojects.
Edit I am able to run these for each sub project with ./gradlew b:jacocoTestReport or ./gradlew c:jacocoTestReport and all the reports and in a folder with the correct names. But as my project grows I don't want to have to run dozens of commands (one for each module) I want a single command ./gradlew b:jacocoTestReport (or something similar) which runs for b and it's subtree
As far as I understand at the moment you're using allprojects {} to configure all of the sub projects. Although this has been the canonical to configure a group of projects in the past, it is now discouraged. Furthermore projects should use publications to interface with each other instead of copying files across project boundaries. So you need to do two things:
Instead of configuring the sub projects from the root root project, you should create a plugin to configure jacoco and create configuration that will hold the reports.
To do this, create a pre-compiled script plugin in your project. The idea is to have a kotlin build script in the buildSrc project and create a Gradle plugin from that file on the fly. So you should move the logic that configures jacoco to the file buildSrc/src/main/kotlin/jacoco-conventions.gradle.kts:
plugins {
jacoco
}
val jacocoTestReport by tasks.getting(JacocoReport::class) {
// jacoco configuration
}
configurations.create("jacocoReports") {
isCanBeResolved = false
isCanBeConsumed = true
attributes {
attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class, "jacocoReports"))
}
outgoing.artifact(jacocoTestReport.reports.xml.destination) {
builtBy(jacocoTestReport)
}
}
The last part creates a new configuration in the project the pre-compiled script plugin is applied to. This configuration uses the xml destination file which is builtBy the jacoco report task as an outgoing artifact. The important part here is the USAGE_ATTRIBUTE because we will need this later on to consume the files.
The precompiled script plugin can now be applied in the projects where you want to gather jacoco metrics by:
// for example in c/build.gradle.kts
plugins {
`jacoco-conventions`
}
Now you have configured the sub projects to put the Jacoco xml reports into configurations with usage attribute jacocoReports.
In the root project create a task that copies the report from the configuration.
To do this we need to setup a configuration that consumes the jacocoReports variants and then depend on the variants of the sub projects:
// main build file
val jacocoReports by configurations.creating {
isCanBeResolved = true
isCanBeConsumed = false
attributes {
attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class, "jacocoReports"))
}
}
dependencies {
jacocoReports(project(":b:c"))
jacocoReports(project(":b:d:e"))
// other jacocoReports you want to consume
}
tasks.register<Copy>("aggregateJacocoReports") {
from(jacocoReports)
into(file("$buildDir/jacoco"))
}
As you can see, the jacocoReports configuration has the same usage attribute, so it can be used to resolve files that are in configurations with the same attribute. Then we need to define which project reports we want to consume. This is cone by defining project dependencies using the jacocoReports configuration. The last step is a simple copy task that copies the files into the build directory of the root project. So now when you call ./gradlew aggregateJacocoReports this task resolves all the files from the jacocoReports configuration, which in turn will create the jacoco report for all projects that are the root project has a dependency on.
Why is this better than cross configuration? If projects are not entangled by cross configuration and by tasks that copy stuff between projects, gradle can more efficiently schedule and parallelize the work that has to be done.
I have a created a minimal example that should help you to setup your project this way: https://github.com/britter/gradle-jacoco-aggregate. I have removed the android specific configuration to keep it simple, but I'm sure you will figure it out.
We have an Android project, and we're using Powermock for some of our test cases and Jacoco for coverage report. We noticed that some our classes are returning as 0% coverage although they are indeed covered. We also observed the message below for affected classes.
"Classes ... do no match with execution data."
A few searches online show that Powermock and Jacoco don't play well and that Offline Instrumentation is a possible workaround.
Has anyone used gradle Offline Instrumentation script for android projects before?
In hindsight, I guess this can be solved with enough android experience and online perusing. However, I was (and still am) relatively new to Android, gradle, and groovy when this fell on my lap, so I'm writing this for the next me :-D
WHAT IS HAPPENING IN A NUTSHELL (excerpt from a jacoco forum)
Source file is compiled into non-instrumented class file
Non-instrumented class file is instrumented (either pre-instrumented offline, or automatically at runtime by Java agent)
Execution of instrumented classes collected into exec file
report decorates source files with information obtained from analysis of exec file and original non-instrumented class files
Message "Classes ... do no match with execution data." during generation of report means that class files used for generation of report are not the same as classes prior to instrumentation.
SOLUTION
The Jacoco Offline Instrumentation page provides the main steps that should occur for offline instrumentation in this excerpt:
For such scenarios class files can be pre-instrumented with JaCoCo,
for example with the instrument Ant task. At runtime the
pre-instrumented classes needs be on the classpath instead of the
original classes. In addition jacocoagent.jar must be put on the
classpath.
The script below does exactly that:
apply plugin: 'jacoco'
configurations {
jacocoAnt
jacocoRuntime
}
jacoco {
toolVersion = "0.8.1"
}
def offline_instrumented_outputDir = "$buildDir.path/intermediates/classes-instrumented/debug"
tasks.withType(Test) {
jacoco.includeNoLocationClasses = true
}
def coverageSourceDirs = [
'src/main/java'
]
task jacocoTestReport(type: JacocoReport, dependsOn: "test") {
group = "Reporting"
description = "Generate Jacoco coverage reports"
classDirectories = fileTree(
dir: 'build/intermediates/classes/debug',
excludes: ['**/R.class',
'**/R$*.class',
'**/BuildConfig.*',
'**/MainActivity.*']
)
sourceDirectories = files(coverageSourceDirs)
executionData = files('build/jacoco/testDebugUnitTest.exec')
}
jacocoTestReport {
reports {
xml.enabled true
html.enabled true
html.destination file("build/test-results/jacocoHtml")
}
}
/* This task is used to create offline instrumentation of classes for on-the-fly instrumentation coverage tool like Jacoco. See jacoco classId
* and Offline Instrumentation from the jacoco site for more info.
*
* In this case, some classes mocked using PowerMock were reported as 0% coverage on jacoco & Sonarqube. The issue between PowerMock and jacoco
* is well documented, and a possible solution is offline Instrumentation (not so well documented for gradle).
*
* In a nutshell, this task:
* - Pre-instruments the original *.class files
* - Puts the instrumented classes path at the beginning of the task's classpath (for report purposes)
* - Runs test & generates a new exec file based on the pre-instrumented classes -- as opposed to on-the-fly instrumented class files generated by jacoco.
*
* It is currently not implemented to run prior to any other existing tasks (like test, jacocoTestReport, etc...), therefore, it should be called
* explicitly if Offline Instrumentation report is needed.
*
* Usage: gradle clean & gradle createOfflineInstrTestCoverageReport & gradle jacocoTestReport
* - gradle clean //To prevent influence from any previous task execution
* - gradle createOfflineInstrTestCoverageReport //To generate *.exec file from offline instrumented class
* - gradle jacocoTestReport //To generate html report from newly created *.exec task
*/
task createOfflineTestCoverageReport(dependsOn: ['instrument', 'testDebugUnitTest']) {
doLast {
ant.taskdef(name: 'report',
classname: 'org.jacoco.ant.ReportTask',
classpath: configurations.jacocoAnt.asPath)
ant.report() {
executiondata {
ant.file(file: "$buildDir.path/jacoco/testDebugUnitTest.exec")
}
structure(name: 'Example') {
classfiles {
fileset(dir: "$project.buildDir/intermediates/classes/debug")
}
sourcefiles {
fileset(dir: 'src/main/java')
}
}
//Uncomment if we want the task to generate jacoco html reports. However, the current script does not exclude files.
//An alternative is to used jacocoTestReport after this task finishes
//html(destdir: "$buildDir.path/reports/jacocoHtml")
}
}
}
/*
* Part of the Offline Instrumentation process is to add the jacoco runtime to the class path along with the path of the instrumented files.
*/
gradle.taskGraph.whenReady { graph ->
if (graph.hasTask(instrument)) {
tasks.withType(Test) {
doFirst {
systemProperty 'jacoco-agent.destfile', buildDir.path + '/jacoco/testDebugUnitTest.exec'
classpath = files(offline_instrumented_outputDir) + classpath + configurations.jacocoRuntime
}
}
}
}
/*
* Instruments the classes per se
*/
task instrument(dependsOn:'compileDebugUnitTestSources') {
doLast {
println 'Instrumenting classes'
ant.taskdef(name: 'instrument',
classname: 'org.jacoco.ant.InstrumentTask',
classpath: configurations.jacocoAnt.asPath)
ant.instrument(destdir: offline_instrumented_outputDir) {
fileset(dir: "$buildDir.path/intermediates/classes/debug")
}
}
}
Usage
The script can be copied into a separate file. For instance: jacoco.gradle
Reference the jacoco file in your build.gradle. For instance: apply from: jacoco.gradle
Ensure proper dependencies: jacocoAnt 'org.jacoco:org.jacoco.ant:0.8.1:nodeps'
In command line run: gradle clean & gradle createOfflineTestCoverageReport & gradle jacocoTestReport
gradle clean will wipe out any previous gradle execution artifacts
gradle createOfflineTestCoverageReport will create offline instrumentation, change order of classpath, generate .exec file
gradle jacocoTestReport will run test and generate jacoco report based on previously generated .exec file
Feeling Lost?
I've put together a github Jacoco Powermock Android project with sample scripts to reproduce and fix the issue. It also contains more information about the solution.
REFERENCE
https://github.com/powermock/powermock/wiki/Code-coverage-with-JaCoCo
https://www.jacoco.org/jacoco/trunk/doc/classids.html
https://www.jacoco.org/jacoco/trunk/doc/offline.html
https://github.com/powermock/powermock-examples-maven/tree/master/jacoco-offline
https://automated-testing.info/t/jacoco-offline-instrumentations-for-android-gradle/20121
https://stackoverflow.com/questions/41370815/jacoco-offline-instrumentation-gradle-script/42238982#42238982
https://groups.google.com/forum/#!msg/jacoco/5IqM4AibmT8/-x5w4kU9BAAJ
My build.gradle is like this:
productFlavors {
mainFlavor {
// ...
}
}
buildTypes {
debug {
// ...
}
buildType1 {// I write mock data classes for Espresso tests here
// ...
}
}
./gradlew tasks includes connectedMainFlavorDebugAndroidTest but not connectedMainFlavorBuildType1AndroidTest.
Why?
I want to specifically run Espresso tests for buildType1 only.
I'm not the owner of the project, I'm not allowed to use either mainFlavorDebug or someNewFlavorDebug to write Espresso tests
The answer from official documentation:
By default, all tests run against the debug build type. You can change
this to another build type by using the testBuildType property in your
module-level build.gradle file. For example, if you want to run your
tests against your "staging" build type, edit the file as shown in the
following snippet.
android {
...
testBuildType "staging" }