Is there a way to run only the failed set of tests on Android using Gradle?
At the moment, I run my tests as follows.
./gradle connectedDebugAndroidTest
There are tests that occasionally fail due to environment issues which are difficult to control and what I would like to do is be able to only run those failed tests and merge the result with the previous test results.
So for example, if I have 100 tests and 90 succeed, I would like to re-run the failing 10 tests. If those 10 pass the second time around, I would like to merge those results with the original test run.
It looks like this has been discussed several times for Gradle but there doesn't seem to be a solution yet.
https://github.com/gradle/gradle/issues/4068
https://github.com/gradle/gradle/issues/4450
https://github.com/gradle/gradle/issues/1283
Thanks!
The reason they don't have a way to only rerun failed tests is because it screws up the way Gradle currently works. This happens because on the first run, Gradle knows 90 tests passed. If you update the code, and then rerun only the 10 failed tests (using this new option you want them to add), then Gradle would think that all the tests have passed. However, this isn't the case because the tests which previously passed might've broken from the update which fixed the failing tests.
Despite this, the problem has been solved. Gradle reruns failed tests first, and provides a --fail-fast flag for the test task. This effectively does what you want (i.e., only reruns failed tests).
If you want to automatically rerun failed tests as part of the same build in which they failed, and succeed the build if they succeed on retry, you can use the Test Retry Gradle plugin. This will rerun each failed test a certain number of times, with the option of failing the build if too many failures have occurred overall.
plugins {
id 'org.gradle.test-retry' version '1.2.0'
}
test {
retry {
maxRetries = 3
maxFailures = 20 // Optional attribute
}
}
Related
I try to make a CI test with Gradle Managed Device and Github action. I made several attempts, but I mostly run into
Execution failed for task ':app:nexusOneApi30Setup'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.ManagedDeviceSetupTask$ManagedDeviceSetupRunnable
> java.lang.IllegalStateException: Gradle was not able to complete device setup for: dev30_aosp_atd_x86_Nexus_One
This could be due to having insufficient resources to provision the number of
devices requested. Try running the test again and request fewer devices or
fewer shards.
As source I use https://github.com/android/testing-samples/tree/main/ui/espresso/ScreenshotSample in my fork which works local properly
In gradle.properties I'm setting
org.gradle.unsafe.configuration-cache=true
This works without errors on my local machine. The output is:
0 problems were found storing the configuration cache.
When I set up a job on GitHub Actions, it only succeeds if I deactivate the configuration cache.
When it is activated I get this log:
3 problems were found storing the configuration cache, 1 of which seems unique.
- Task `:app:buildKotlinToolingMetadata` of type `org.jetbrains.kotlin.gradle.tooling.BuildKotlinToolingMetadataTask$FromKotlinExtension`: invocation of 'Task.project' at execution time is unsupported.
See https://docs.gradle.org/7.4.2/userguide/configuration_cache.html#config_cache:requirements:use_project_during_execution
See the complete report at file:///home/runner/work/***/***/build/reports/configuration-cache/3lar58wlvtv9703t0m3olblg9/gwoz69d8l961obatzzelsv4d/configuration-cache-report.html
> Invocation of 'Task.project' by task ':app:buildKotlinToolingMetadata' at execution time is unsupported.
My primary interest is why this behaves differently.
I'm also wondering what the best workaround is.
(Can tasks be skipped from caching? Or how would you deactivate the configuration cache on the CI server?)
I haven't understood the cause yet.
My current workaround is to disable the configuration cache on CI builds adding the option --no-configuration-cache to all gradle commands e.g.
./gradlew test --no-configuration-cache
This overwrites the setting of gradle.properties.
I tried to generate code report using detekt and when execute the below command in terminal
gradle detekt
it showing build failed with below message.
* What went wrong:
Execution failed for task ':app:detekt'.
> Build failed with 395 weighted issues.
As others have said in the comments, this means you have 395 issues in your code (kind of like lint warnings).
Detekt has a maxissues: property that determines whether or not to fail the build if your issues surpass the allowed number of maxissues. What I did was search the whole project for maxissues, which will take you to your detekt-config.yml or default-detekt-config.yml. There, you can change your maxissues to whatever you want.
In our old code base, we had 900 some issues, so I changed mine from maxissues:0 to masissues:1000. As we clean up the code, I hope to bring that number down.
Why Circleci doesn't see my test?
I have test called MyAppTest.java. But when I run build on circleci it only shows the following results:
Your build ran 2 tests in testDebugUnitTest, testReleaseUnitTest with 0 failures
Slowest test: com.myapp.android.ExampleUnitTest addition_isCorrect (took 0.00 seconds).
Why it doesn't show result for MyAppTest?
The reason for this that configuration of my config.yml file was only for Unit tests. In the end I've found proper configuration for Instrumentation tests, but still faced error of not enough memory.
Whick gradle task is the fastest to check whether the code passes compilation without any syntax error on Android project
Based on the output of the command
./gradlew tasks --all
executed in a directory containing a small Android project, I'd suggest that your best bet is either the command
./gradlew compileReleaseSources
or the command
./gradlew compileReleaseJava
Here's the full list of tasks that compileReleaseSources depends on for my project (might vary slightly for your own):
app:compileReleaseSources
app:checkReleaseManifest
app:compileReleaseAidl
app:compileReleaseJava
app:compileReleaseNdk
app:compileReleaseRenderscript
app:generateReleaseAssets
app:generateReleaseBuildConfig
app:generateReleaseResValues
app:generateReleaseResources
app:generateReleaseSources
app:mergeReleaseAssets
app:mergeReleaseResources
app:preBuild
app:preDebugBuild
app:preReleaseBuild
app:prepareComAndroidSupportAppcompatV72210Library - Prepare com.android.support:appcompat-v7:22.1.0
app:prepareComAndroidSupportSupportV42210Library - Prepare com.android.support:support-v4:22.1.0
app:prepareReleaseDependencies
app:processReleaseManifest
app:processReleaseResources
Note that this includes the Java compilation step, as well as the compilation of other code sources and various resource processing. Depending on your exact needs, calling compileReleaseJava instead of compileReleaseSources may be enough and will be faster. (Unfortunately, I don't know of a way to view Gradle task dependencies in a tree-like structure, so it's not clear to me exactly how much faster the compileReleaseJava task would be).
If you also want to check that test code compiles, you'd need to add an analogous command: e.g. compileDebugTestSources.