Develop Reference

Code coverage on Android/Kotlin with Kover and Sonar differs on percents

I'm using Kover to get coverage on kotlin and want to share it with sonar, configuration is like this:
plugins {
...
id "org.jetbrains.kotlinx.kover" version "0.4.2"
id "org.sonarqube" version "3.3"
}
sonarqube {
properties {
property "sonar.sourceEncoding", "UTF-8"
...
property "sonar.coverage.jacoco.xmlReportPaths", "${project.buildDir}/reports/kover/report.xml"
}
}
project.tasks["sonarqube"].dependsOn "koverReport"
Now, the HTML report of Kover says I have 33% of coverage meanwhile Sonar shows a 10% coverage. What could be the problem?

The simple answer is that they are computed differently: https://community.sonarsource.com/t/sonarqube-and-code-coverage/4725 .

How to exclude explicitApi warnings for test classes in Kotlin?

I have Android library module with enabled explicitApi kotlin feature in gradle
android {
kotlinOptions {
freeCompilerArgs += '-Xexplicit-api=warning'
}
}
Everything is fine, but the problem is that warnings are also reported for test classes in packages src/test and src/androidTest.
How to exclude test classes from explicit-api control?
Thanks

As far as I know, you can't! I was thinking of opening a bug report just this week but never got to it. In the meantime, I suggest you add something like this to your build script, which will at least fix it for the Kotlin compiler (but you'll still see the IDE warnings):
tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile) {
if (!it.name.contains("Test")) {
kotlinOptions.freeCompilerArgs += "-Xexplicit-api=strict"
}
}
If you're using Gradle Kotlin DSL:
tasks.withType<org.jetbrains.kotlin.gradle.tasks.KotlinCompile> {
if ("UnitTest" !in name) {
kotlinOptions.freeCompilerArgs += "-Xexplicit-api=strict"
}
}
And don't set the compiler argument in android.kotlinOptions, only in that block, so that it is applied only to the non-test source sets.
EDIT: I just checked and the warnings bug was fixed in IntelliJ, so it should be fixed in Android Studio in a few months to a year.

Android - Connected Tests Results Directory

I have an android project where I run normal unit tests as well as instrumented tests.
Now the thing is that the results of the
unit tests are stored in build/reports/tests (./debug or ./release)
instrumented tests are stored in build/outputs/connected
Is it somehow possible to change the result directory of the instrumented tests to build/reports/tests/connected?
Thank you already!

Found the answer myself, To change the directory for instrumented tests use:
android {
testOptions {
reportDir = "$project.buildDir/reports"
resultsDir = "$project.buildDir/test-results"
}
# Or for lint if needed
lintOptions {
htmlOutput = file("$project.buildDir/reports/lint/LINT-results.html")
xmlOutput = file("$project.buildDir/test-results/lint/LINT-results.xml")
}
}

I believe this would be more reliable:
android {
[...]
testOptions {
if (project.hasProperty("customResultsDir")) {
resultsDir "${customResultsDir}"
}
if (project.hasProperty("customReportDir")) {
reportDir "${customReportDir}"
}
}
}
There is the bonus that it checks if the property is passed before trying to use it (a good practice). But the real interesting part is that this ways you are interpolating a closure instead of a variable - the differences allow further possibilities (check string interpolation and closure interpolation).
On my scenario, only by using the closure did I manage to have the properties to populate at the exact time.

sonarqube with both androidTest and test covarage

I'm trying to setup sonarqube reporting in my Android project. I currently have trouble with showing all test classes in the sonar UI, the coverage is shown in percentages and currently only the unit test from app/src/test/ are shown as Unit Tests.
My project has a test folder app/src/test/ which contains unit test and I have a androidTest folder app/src/androidTest/ which contain android unit, integration and UI tests. When I run all the tests via gradle the android-gradle plugin generates build/jacoco/testDebugUnitTest.exec and build/test-results/debug/TEST-*Test.xml which contains the jacoco results and coverage report for the unit test in the test folder. Also the android-gradle plugin generates build/outputs/code-coverage/connected/coverage.ec and build/outputs/androidTest-results/connected/TEST-*Test.xml contain the results and coverage reports from the androidTest folder
In my build.gradle I can specify the properties for the sonar plugin.
sonarqube {
properties {
property "sonar.sources", "src/main/java,src/main/res"
property "sonar.tests", "src/test/java,src/androidTest/java"
property "sonar.java.coveragePlugin", "jacoco"
property "sonar.jacoco.reportPath", "${project.buildDir}/jacoco/testDebugUnitTest.exec"
property 'sonar.jacoco.itReportPath', "${project.buildDir}/outputs/code-coverage/connected/coverage.ec"
property "sonar.junit.reportsPath", "${project.buildDir}/test-results/debug" // path to junit reports
}
}
With sonar.junit.reportsPath I can specify which xml report is sent to the sonarqube server. When I change it to build/outputs/androidTest-results/connected I get the androidTest shown as Unit Test on the dashboard. Is there a way to make the sonar plugin look in both directories or merge the results together?

Until https://jira.sonarsource.com/browse/SONAR-4101 is fixed, the only option you have is to write a task that copies your test result files into a single place and configure that as sonar.junit.reportsPath, like this:
task combineTestResultsForSonarqube {
group = "Reporting"
def modules = ["app", "and", "other", "modules"];
doLast {
modules.each { module ->
File combined = file("${module}/build/combined-test-results")
if (combined.exists()) {
combined.deleteDir()
}
combined.mkdirs();
def testDirs = [file("${module}/build/test-results/debug/"),
file("${module}/build/outputs/androidTest-results/connected/")];
testDirs.each { testDir ->
if (!testDir.exists()) {
logging.captureStandardOutput LogLevel.WARN
println "WARNING: ignoring non-existant ${testDir.path}"
return;
}
files(testDir.listFiles()).each { file ->
new File(combined, file.getName()) << file.text
}
}
}
}
}
Paths of course have to be adapted when you have flavors in your build.

How can I ignore test failures with the gradle robolectric plugin?

I'm using the robolectric-gradle-plugin for robolectric unit tests. I don't want to fail a build on failed tests. Is there a way in DSL or a property not to fail a test on the build similar to -DtestFailureIgnore=true on the Surefire Maven plugin?
I've tried:
robolectric {
ignoreFailures = true
}
and
robolectric {
ignoreFailure = true
}
and -DignoreFailure=true on the command line.
I can't seem to find any documentation of how to do this, or any reference to ignoring tests in the source code.

answering very old question, so that it might help others who bump into here
testOptions {
unitTests.all {
setIgnoreFailures(true)
}
}

I would suggest not to continue building an APK if there are any failing tests. But if you want to build an APK without testing the only way right now is to use gradle build -x test[1]. This will run build and not run any tests.
[1]http://www.gradle.org/docs/current/userguide/userguide_single.html#sec:excluding_tasks_from_the_command_line

try without '='
robolectric {
ignoreFailures true
}