Develop Reference

lint unit tests not running: java.lang.NoSuchFieldError: USE_PSI_CLASS_FILES_READING

I've wrote a lint rule to be used in my Android Studio project. It is already working successfully. However, the tests are not running. Everytime I try to execute the tests, I receive the following error:
USE_PSI_CLASS_FILES_READING
java.lang.NoSuchFieldError: USE_PSI_CLASS_FILES_READING
at com.android.tools.lint.UastEnvironmentKt.createKotlinCompilerConfig(UastEnvironment.kt:310)
at com.android.tools.lint.UastEnvironmentKt.access$createKotlinCompilerConfig(UastEnvironment.kt:1)
at com.android.tools.lint.UastEnvironment$Configuration$Companion.create(UastEnvironment.kt:118)
at com.android.tools.lint.LintCliClient.initializeProjects(LintCliClient.kt:1349)
at com.android.tools.lint.client.api.LintClient.performInitializeProjects$lint_api(LintClient.kt:1012)
at com.android.tools.lint.client.api.LintDriver.initializeProjectRoots(LintDriver.kt:570)
at com.android.tools.lint.client.api.LintDriver.doAnalyze(LintDriver.kt:485)
at com.android.tools.lint.client.api.LintDriver.doAnalyze$default(LintDriver.kt:474)
at com.android.tools.lint.client.api.LintDriver.analyze(LintDriver.kt:418)
at com.android.tools.lint.checks.infrastructure.TestLintClient.analyze(TestLintClient.java:671)
at com.android.tools.lint.checks.infrastructure.TestLintClient.analyze(TestLintClient.java:602)
at com.android.tools.lint.checks.infrastructure.TestLintClient.checkLint(TestLintClient.java:349)
at com.android.tools.lint.checks.infrastructure.TestLintRunner.checkLint(TestLintRunner.kt:229)
at com.android.tools.lint.checks.infrastructure.TestLintRunner.run(TestLintRunner.kt:172)
at com.android.tools.lint.checks.infrastructure.TestLintTask.run(TestLintTask.java:933)
Basically, the error occurs everytime I call the run method of lint() in my tests:
lint().files(classFile)
.allowMissingSdk()
.run()
.expectWarningCount(1)
I executed the tests from Android Studio and by using ./gradlew lint-rules:test. In both cases the same error arises.
Thanks for your help!

Re-run failed android tests

Is there a way to run only the failed set of tests on Android using Gradle?
At the moment, I run my tests as follows.
./gradle connectedDebugAndroidTest
There are tests that occasionally fail due to environment issues which are difficult to control and what I would like to do is be able to only run those failed tests and merge the result with the previous test results.
So for example, if I have 100 tests and 90 succeed, I would like to re-run the failing 10 tests. If those 10 pass the second time around, I would like to merge those results with the original test run.
It looks like this has been discussed several times for Gradle but there doesn't seem to be a solution yet.
https://github.com/gradle/gradle/issues/4068
https://github.com/gradle/gradle/issues/4450
https://github.com/gradle/gradle/issues/1283
Thanks!

The reason they don't have a way to only rerun failed tests is because it screws up the way Gradle currently works. This happens because on the first run, Gradle knows 90 tests passed. If you update the code, and then rerun only the 10 failed tests (using this new option you want them to add), then Gradle would think that all the tests have passed. However, this isn't the case because the tests which previously passed might've broken from the update which fixed the failing tests.
Despite this, the problem has been solved. Gradle reruns failed tests first, and provides a --fail-fast flag for the test task. This effectively does what you want (i.e., only reruns failed tests).

If you want to automatically rerun failed tests as part of the same build in which they failed, and succeed the build if they succeed on retry, you can use the Test Retry Gradle plugin. This will rerun each failed test a certain number of times, with the option of failing the build if too many failures have occurred overall.
plugins {
id 'org.gradle.test-retry' version '1.2.0'
}
test {
retry {
maxRetries = 3
maxFailures = 20 // Optional attribute
}
}

CircleCi doesn't see my test

Why Circleci doesn't see my test?
I have test called MyAppTest.java. But when I run build on circleci it only shows the following results:
Your build ran 2 tests in testDebugUnitTest, testReleaseUnitTest with 0 failures
Slowest test: com.myapp.android.ExampleUnitTest addition_isCorrect (took 0.00 seconds).
Why it doesn't show result for MyAppTest?

The reason for this that configuration of my config.yml file was only for Unit tests. In the end I've found proper configuration for Instrumentation tests, but still faced error of not enough memory.

How to run only a test suite with the connectedAndroidTest command

I am having problems with timeouts running my UI tests in circle ci since the command connectedAndroidTest is taking more than 10 minutes to run.
So I am trying to split them into test suits and run each suite at a time.
I found how to create suites for my android tests here: https://developer.android.com/reference/junit/framework/TestSuite.html
But I can't find how to run them with the connectedAndroidTest command.

I have not found any way to execute the TestSuite, but I found two other options to solve my timeout problems with the circle ci tests:
Run tests by package:
./gradlew app:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.package=<package>
Run tests by type:
./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.size=<small|medium|large>
You need to add #SmallTest, #MediumTest, #LargeTest to your test classes or methods to split them into the groups and avoid hitting a timeout.
Another option that I found was to change the timeout of the command:
- ./gradlew app:connectedCheck -PdisablePreDex:
timeout: 1800

To add to #jonathanrz answer:
Run tests by class:
./gradlew app:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class= com.example.app.MyTestClass
Run tests by method:
./gradlew app:connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class= com.example.app.MyTestClass#myTestMethod
Btw: You can add further arguments to your tests by adding:
-Pandroid.testInstrumentationRunnerArguments.foo="My\ Extra\ Info
Just note how you have to escape spaces here if they exist

infinitest usage with Robotium for Android

I cant get infinitest(continuous test runner for TDD) to auto test with android junit test cases. Im using Robotium for UI testing.
Is infinitest compatible with Robotium ?
Here is the Problem page error (within eclipse Problem window) i get even though my test cases run fine when executed manually:
Description Resource Path Location Type
RuntimeException (Stub!) in ProjectTests. ProjectTests.java
line 0 Infinitest Test Failure