I am running android tests using the Gradle Android plugin and want to see individual test results.
From answers to this Question Gradle: How to Display Test Results in the Console in Real Time? it seems I can either use --info (which prints a LOT of other verbose junk I don't care about) or use this closure which only works for the Java plugin (not the Android plugin)
test {
afterTest { desc, result ->
println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
}
}
Is there some other option / closure I can use when I am running the connectedCheck task just to print the individual test results without all the other "verbosity".
Use Gradle info
This will print all information from Gradle:
gradle --info
or Use Android Gradle plugin:
android.testOptions.unitTests.all {
// Configure whether failing tests should fail the build
ignoreFailures false
testLogging {
events "passed", "skipped", "failed", "standardOut", "standardError"
}
}
or Use Gradle directly:
allprojects {
tasks.withType(Test) {
testLogging {
exceptionFormat "full"
showCauses true
showExceptions true
showStackTraces true
showStandardStreams true
events = ["passed", "skipped", "failed", "standardOut", "standardError"]
}
}
}
See: https://github.com/jaredsburrows/android-gradle-java-app-template/blob/master/gradle/compile.gradle#L20
Output:
io.github.hidroh.materialistic.data.SessionManagerTest > testView PASSED
io.github.hidroh.materialistic.data.SessionManagerTest > testIsViewFalse PASSED
io.github.hidroh.materialistic.data.SessionManagerTest > testIsViewNull PASSED
io.github.hidroh.materialistic.data.SessionManagerTest > testIsViewTrue PASSED
io.github.hidroh.materialistic.data.SessionManagerTest > testViewNoId PASSED
Source: https://github.com/hidroh/materialistic/blob/master/robolectric.gradle
Gradle Docs: https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/testing/logging/TestLogEvent.html
For me all the options mentioned in the other answer still printed a lot of verbose information. So despite the requirement that info should not be used, I successfully use the following. For example, if your tests are in the package com.example.android, you can use:
gradle --info connectedDebugAndroidTest | grep "com\.example\.android\..* >"
Will print e. g.:
com.example.android.login.LoginActivityTest > enterCredentialsTest[Testing_emulator(AVD) - 9] SUCCESS
And the word "SUCCESS" will be green, which is awesome.
For Android Studio (tested on com.android.tools.build:gradle:2.1.0 and gradle version gradle-2.10) I added the following section to print exceptions in full format as well as logging every executed test:
apply plugin: 'com.android.application'
android { ... }
dependencies { ...}
tasks.withType(Test) {
testLogging {
exceptionFormat "full"
}
afterTest { desc, result ->
println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
}
}
Related
I am trying to configure a new Android project to use JaCoCo for test coverage and fail when coverage is below a certain threshold. I have used this documentation, as well as looking at many SO posts. I expect the task to fail, since I don't have test coverage, but it succeeds without any warnings.
My :app build.gradle file includes the following:
plugins {
...
id 'jacoco'
}
jacoco {
toolVersion = "0.8.7"
reportsDirectory = layout.buildDirectory.dir('app/build/reports')
}
task jacocoTestCoverageVerification(type: JacocoCoverageVerification, dependsOn: ['jacocoTestReport']) {
onlyIf = { true }
violationRules {
failOnViolation = true
rule {
limit {
minimum = 1.0
}
}
}
}
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {
finalizedBy jacocoTestCoverageVerification
onlyIf = {true}
reports {
xml.enabled true
html.enabled true
html.outputLocation = layout.buildDirectory.dir('app/build/reports')
}
}
tasks.withType(Test) {
finalizedBy jacocoTestReport
jacoco.includeNoLocationClasses = true
jacoco.excludes = ['jdk.internal.*']
}
android {
...
jacoco{
version = "0.8.7"
}
}
When I try to run tests, using any of the following:
./gradlew test
./gradlew jacocoTestReport
./gradlew jacocoTestCoverageVerification
The results are always the same:
...
> Task :app:connectedDebugAndroidTest
Starting 2 tests on Nexus 5X - 8.1.0
> Task :app:createDebugAndroidTestCoverageReport
> Task :app:createDebugCoverageReport
> Task :app:jacocoTestReport
> Task :app:jacocoTestCoverageVerification
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/7.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 31s
93 actionable tasks: 93 executed
Build Analyzer results available
1:10:48 PM: Task execution finished 'test'.
I expect the task to fail, since I have no test coverage. I have added a utility class and method, and partially implemented a test so that the JaCoCo report shows 60% coverage for one class, but this has not changed anything.
My company uses Buildkite for our CI services. When the CI server triggers a build, all logs from the build are sent to Buildkite and made available through the UI. This is useful since it allows us to check how a build failed.
Buildkite has a feature which creates collapsible groups from logs formatted like:
--- Compiling
logs after this point will be in a collapsible group named 'Compiling'
--- Testing
logs after this point will be in a collapsible group named 'Testing'
How can I add custom logging to a gradle build which will output these 'groups'? I'd like to have a group for compiling/assembling, one for running unit tests, etc.
I've considered adding tasks which perform the logging and making them dependencies of the built-in tasks, but I'm not sure how to do that, or if it's a good idea in general.
I'm using Gradle 2.12 and whipped up an example from the Logging doc. The example does not use Android Studio nor BuildKite, but I believe this will help with the fundamental question.
Given a simple build.gradle file for a typical Java project:
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
testCompile 'junit:junit:4.11'
}
compileJava << { println "TRACER example log from compileJava" }
compileTestJava << { println "TRACER example log from compileTestJava" }
test << { println "TRACER example log from test" }
and an init.gradle file:
useLogger(new CustomEventLogger())
class CustomEventLogger extends BuildAdapter implements TaskExecutionListener {
public void beforeExecute(Task task) {
if (task.name ==~ "compileJava") {
println "--- Compiling"
} else if (task.name == "test") {
println "--- Testing"
}
}
public void afterExecute(Task task, TaskState state) {}
public void buildFinished(BuildResult result) {}
}
then this command-line:
$ gradle -I init.gradle test
yields this output:
--- Compiling
TRACER example log from compileJava
TRACER example log from compileTestJava
--- Testing
TRACER example log from test
When I execute Android tests on the command-line using Gradle, I would like to see test log output. I followed the suggestion in this post but events are not being logged.
My app/build.gradle:
apply plugin: 'com.android.application'
...
//Test Logging
tasks.withType(Test) {
testLogging {
events "started", "passed", "skipped", "failed"
}
}
My project has multiple product flavors (free, paid) and build types (debug, release). I expected this would configure these tasks for test logging:
connectedAndroidTestPaidDebug
connectedAndroidTestFreeDebug
connectedAndroidTestPaidRelease
...
In Android Studio, I see warnings:
Cannot resolve symbol 'testLogging'
Cannot resolve symbol 'events'
have you tried using :
testOptions.unitTests.all {
testLogging {
events 'passed', 'skipped', 'failed', 'standardOut', 'standardError'
}
}
I did search a lot but unfortunately couldn't get it to work.
Based on my search I found that I need to add following code into build.gradle file. However, Gradle seems doesn't recognize it and always says Geadle DSL method not found: test()
test {
testLogging.showStandardStreams = true
testLogging.events("passed", "skipped", "failed", "standardOut", "standardError")
afterTest { desc, result ->
println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
}
}
Update
I can confirm above code or better than that following code is working fine if you create a test project and move all your test cases inside that instead of src/test/java and src/androidTest/java in main project. It's because you can apply java plugin in build.gradle file. However, It's not possible to use following code in any build.gradle file that com.android.* has been defined. Since these two libraries are not compatible :(
apply plugin: 'java'
evaluationDependsOn(':YOUR-LIB')
test {
testLogging.showStandardStreams = true
testLogging {
events "passed", "skipped", "failed", "standardOut", "standardError"
exceptionFormat = 'full'
}
afterTest { desc, result ->
println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
}
forkEvery = 5
maxParallelForks = java.lang.Runtime.runtime.availableProcessors() / 2
}
tasks.withType(Test) {
// Need Gradle to ignore classes that are inner class of Test classes but not actually Tests
scanForTestClasses = false
include "**/*Test.class"
}
So, my question is does anyone's INVENTED any way to print out logs under android plugin?
Hi I have set up my Android project on Jenkins to provide CI. Its working well, running tests on a connected Android handset. The tests run on the Android Test Framework which extends jUnit3.
Unfortunately, the build is marked as a failure if there are any test failures. I'd like to be able to improve this in two ways:
Allowing unstable builds
Being able to mark known test failures
For item 1 I tried adding this to the project build.gradle:
connectedCheck {
ignoreFailures = true
}
But it has no effect. Looking at the build log, I realised the actual test task is called connectedInstrumentTest, but this task is not found:
connectedInstrumentTest {
ignoreFailures = true
}
causes:
Could not find method connectedInstrumentTest() for arguments [build_4ldpah0qgf0ukktofecsq41r98$_run_closure3#9cd826] on project ':Playtime'.
That am I missing?
Thanks
EDIT: Heres my project build.gradle, nothing special:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.android.tools.build:gradle:0.5.+'
}
}
apply plugin: 'android'
dependencies {
compile files('libs/android-support-v4.jar')
}
android {
compileSdkVersion 17
buildToolsVersion "18.0.1"
defaultConfig {
minSdkVersion 8
targetSdkVersion 16
testPackageName "com.bb.pt.test"
testInstrumentationRunner "android.test.InstrumentationTestRunner"
}
}
connectedCheck {
ignoreFailures = true
}
My gradle settings in jenkins:
switches: --stacktrace --info
tasks: :pt:assembleDebug :pt:assembleTest :pt:connectedCheck
EDIT:
I built gradlew and tried that. Same output. I don't want the build to FAIL if there are test failures:
:pt:connectedInstrumentTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':pt:connectedInstrumentTest'.
> There were failing tests. See the report at: file:///home/simon/WorkingCopies/bb/code/trunk/pt/pt/build/reports/instrumentTests/connected/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.
BUILD FAILED
I tried to qualify the task name in build.gradle:
task connectedCheck {
ignoreFailures = true
}
But it thinks I am trying to add a new task rather than modify the existing one.
FAILURE: Build failed with an exception.
* Where:
Build file '/home/simon/WorkingCopies/bb/code/trunk/pt/pt/build.gradle' line: 31
* What went wrong:
A problem occurred evaluating project ':pt'.
> Cannot add task ':pt:connectedCheck' as a task with that name already exists.
After our conversation I believe that:
the problem is gradle configuration only and not jenkins related. Get it to work in gradle.
in gradle I believe (though I am not an expert) you should get the connectedInstrumentTest to ignore failures, but your attempt to use the following failed
connectedInstrumentTest {
ignoreFailures = true
}
maybe the solution is to wrap this config node like this:
project.gradle.taskGraph.whenReady {
connectedInstrumentTest {
ignoreFailures = true
}
}
https://github.com/stanfy/hotandroid/blob/master/part0/build.gradle