Is there a way to run a specific Android instrumentation unit test using Gradle? I've tried
gradle -Dtest.single=UnitTestName connectedInstrumentTest
but it seems to run all the tests in the package.
Using test.single appears to be deprecated. The new correct way to do this is
./gradlew :<module>:test --tests <pattern>
where <pattern> could be something like:
com.example.MyTest to run all test methods in com.example.MyTest
*MyTest to match every method in every class whose name ends with MyTest
*.MyTest.myMethod to run a specific test method in class MyTest in any package
If you have a multi-project build, make sure to give the module path before the test task; otherwise you'll get a misleading error message when it searches for your test pattern in every subproject.
None of this is documented on the Gradle site anywhere I could find it.
This works if you're using an instrumentationTestRunner:
./gradlew test -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
Using gradle 2.10 and android gradle plugin 2.0.0-beta2.
Since you know what test(s) you want to run, you probably know which module / flavor to use too. You can help Gradle out by specifying the exact module and Gradle task. So if your test is in the app module and you want to test the debug flavor:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
You can get even more fancy with the tests_regex argument instead:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=PartialClassName*
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=partialMethodName*
The pattern is -D<testTaskName>.single=<TestClass> so in your example it should be:
gradle -DconnectedInstrumentTest.single=UnitTestName connectedInstrumentTest
NOTE: This answer is outdated. You should use the --tests switch in the latest versions of Gradle. (see other answers for an explanation)
Since Android gradle plugin 1.1.0-rc1, one can run single test class using --tests flag by executing:
./gradlew app:testDebug --tests=com.example.MyTest
See http://tools.android.com/tech-docs/unit-testing-support#TOC-Running-from-Gradle
You gotta check this out.
https://github.com/JCAndKSolutions/android-unit-test
I made an issue in this github repository, and this guy solved my problem and upload to maven, so in my build.gradle file I use this plugin.
Instructions are written in his repository. you can easily follow it.
After using this android-unit-test plugin, I can use like
../gradlew -Dtest.single=SomeTest test
or
../gradlew -Dtest.single=SomeTest clean check
Now it's working and I could only run the specific tests I want to
You should not forget to specify a build variant name after test property declaration like
-Dtest<buildVariantName>=<yourTestName>.
Like if you have a debug build type which gives you debug variant after compilation, then if you want to run a test only for this build variant you should declare a command like this:
./gradlew -DtestDebug=UnitTestName testDebug
Erdi's answer didn't work for me but I have a single parent for all my test classes so I was able to do this:
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if(getClass().getSimpleName().equals("MyTestClassName")) {
super.runTest();
}
}
//...
}
This executes only MyTestClassName. We can extend it further to execute only specific test method (or methods):
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if("MyTestClassName".equals(getClass().getSimpleName())
&& "testMethodName".equals(getName())) {
super.runTest();
}
}
//...
}
the Gradle command does not work for me.
I used below mentioened adb command.
for this you need to build your apk first.
adb shell am instrument -w -r -e package -e debug false .debug.test/android.support.test.runner.AndroidJUnitRunner
Related
So with the release of Android Studio Dolphin & Beta of Electric Eel, I wanted to try the instrumentation tests in gradle. I do however want to exclude some of the tests being run, in order to be able to run specific test suites one at a time.
So here is what I configured so far:
android {
testOptions {
managedDevices {
devices {
pixel2api30 (com.android.build.api.dsl.ManagedVirtualDevice) {
device = "Pixel 2"
apiLevel = 30
systemImageSource = "aosp-atd"
}
}
}
}
}
I know I can run my entire suite using
./gradlew device-nameBuildVariantAndroidTest
In my case that would be
./gradlew pixel2api30gaeDebugAndroidTest
gaeDebug being my build variant. This command is being run in my project root.
If I want to run the tests in the tests/large folder for example
How would I go about doing that? Thanks.
For this you could use 2 different approaches:
Create and run test suites.
For example, for each of these folders, create a TestSuite and define Test classes. For example:
#RunWith(Suite.class)
#Suite.SuiteClasses({
ExampleLargeTest.class,
ExampleTwoLargeTest.class,
ExampleThreeLargeTest.class
})
public class LargeTestsSuite {
}
This suite can be run using following command
./gradlew pixel2api30gaeDebugAndroidTest - Pandroid.testInstrumentationRunnerArguments.class=path.to.your.suite.class
Use Test Categories
Annotate your Test classes like this:
#Category("Large")
public class ExampleLargeTest { ... }
And then you could execute the following command for running all tests with same category:
./gradlew pixel2api30gaeDebugAndroidTest -PtestCategory=Large
Hopefully one of these two approaches will suite you.
I am facing a problem with Robolectric library. I am writing unit tests using Robolectric, it's working locally but when I merge my code, it crashes on pipeline(remotely).
I am using 'org.robolectric:robolectric:4.0.2'
It fails by only adding this line to my test class: #RunWith(RobolectricTestRunner.class)
And the exception is:
FAILED
org.apache.tools.ant.BuildException
Caused by: org.apache.maven.artifact.resolver.MultipleArtifactsNotFoundException
I had the same issue: Robolectric was working fine locally, but once pushed to Jenkins the gradle task to execute the tests fails.
You can execute the gradle task to run the tests with -i -d flags to see more debug output.
./gradlew -i -d test
For me this revealed that Jenkins was unable to download Robolectric dependencies:
13:58:43 13:58:42.904 [DEBUG] [TestEventLogger] com.my.package.Test > my_test_case STANDARD_ERROR
13:58:43 13:58:42.904 [DEBUG] [TestEventLogger] Downloading: org/robolectric/android-all/9-robolectric-4913185-2/android-all-9-robolectric-4913185-2.jar from repository sonatype at https://oss.sonatype.org/content/groups/public/
I could solve it by telling gradle to use our corporate proxy when running on Jenkins. One way to achieve this would be to add the following to your gradle.properties:
systemProp.http.proxyHost=http://proxy.host
systemProp.http.proxyPort=3128
systemProp.https.proxyHost=http://proxy.host
systemProp.https.proxyPort=3128
---- EDIT ----
Actually I found a cleaner solution for my use case then configuring a proxy: Robolectric offers a way to configure the repository it uses during runtime (see http://robolectric.org/configuring/). This way I was able to tell it to use our corporate repository.
android {
testOptions {
unitTests.all {
systemProperty 'robolectric.dependency.repo.url', 'https://local-mirror/repo'
systemProperty 'robolectric.dependency.repo.id', 'local'
}
}
}
In my case the issue was (./gradlew -i -d testDebug printed it):
Caused by:
org.apache.maven.artifact.resolver.MultipleArtifactsNotFoundException: Missing:
----------
1) org.robolectric:android-all:jar:10-robolectric-5803371
...
Robolectric tried to fetch a dependency from the remote repository, but it couldn't.
As the output suggests, I have:
Manually downloaded org.robolectric:android-all:jar:10-robolectric-5803371 jar from the Maven repository.
Located it in /home/user/jars
Installed Maven on my machine: sudo apt install maven
Linked the jar with Robolectic: mvn install:install-file -DgroupId=org.robolectric -DartifactId=android-all -Dversion=10-robolectric-5803371 -Dpackaging=jar -Dfile=/home/user/jars/android-all-10-robolectric-5803371.jar
Now ./gradlew testDebug works fine.
Also check the issue on Github, maybe there will be some more useful information for you.
As #Christian.D said, Robolectric tries to download dependencies from external repository, but we need that it uses internal one. Below the fix:
1) Create a custom RobolectricTestRunner:
public class CustomRobolectricRunner extends RobolectricTestRunner {
public CustomRobolectricRunner(Class<?> testClass) throws InitializationError {
super(testClass);
}
static {
RoboSettings.setMavenRepositoryId("my-nexus");
RoboSettings.setMavenRepositoryUrl("your_custom_url");
}
}
2) Annotate your test class:
#Config(manifest = Config.NONE)
#RunWith(CustomRobolectricRunner.class)
Good coding!
I am interested in how to run Espresso tests from command line (gradle task) individually (run group/suite tests then close app and then run another group/suite of tests).
Found that it is feasible to implement JUnit Test Suites but do not really understand how does it looks like under the hood in a context of instrumentation tests. Does it starts separate processes per Test suite? There is sample application on Github but how to execute it from terminal?
Another interesting discovery is Sharding tests. However, it one sentence documentation.
May be somebody can share with any experience of running Espresso tests individually.
Most of this is documented as part of AndroidJUnitRunner: https://developer.android.com/reference/android/support/test/runner/AndroidJUnitRunner.html
The key piece that is missing is how to pass those parameters via Gradle. You can do that by specifying the options at the commandline as such:
./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=*The full name of your test suite goes here*
I would recommend using the Spoon library to run your tests individually or in parallel on multiple devices. You can either use the jar file or use the Spoon gradle plugin mentioned on the same page. Both of them have adequate documentation to help you set it up.
You can also use Spoon to run an individual test and the command would look something like this:
./gradlew yourSpoonTaskName -PspoonClassName=com.yourPackageName.blah.ClassName
-PspoonMethodName=methodName
In order to know what yourSpoonTaskName is run ./gradlew tasks.
Also, in your build.gradle file add the following spoon configuration:
spoon {
// for debug output
debug = true
// To grant permissions to Android M >= devices
grantAllPermissions = true
// for sharding
/*
this will execute tests in parallel on multiple devices.
*/
shard = true
// Add this to run a specific test class & method
if (project.hasProperty('spoonClassName')) {
className = project.spoonClassName
}
if (project.hasProperty('spoonMethodName')) {
methodName = project.spoonMethodName
}
}
If you are not interested in Spoon and just want a simple solution, then use the following command to run an individual test:
am instrument -w -r -e class com.packageName.blah.TestName#methodName com.packageName.blah.YourIntrumentationRunnerName
You can easily determine these values if you right click the test name in AndroidStudio and run it. In the console, you will see the entire command being printed when the test is bring run.
I am having problem to find documentation how could I solve that case.
I am capable of launching small/medium/large tests with:
./gradlew spoonSmall
./gradlew spoonMedium
./gradlew spoonLarge
Or launching specific tests with usage of this setup:
spoon {
(...)
if (project.hasProperty('spoonClassName')) {
className = project.spoonClassName
if (project.hasProperty('spoonMethodName')) {
methodName = project.spoonMethodName
}
}
}
I can launch specific file:
./gradlew spoon -PspoonClassName=com.package.tests.MyTest;
What I am interested in is a possibility to launch all tests located in:
./gradlew spoon -PspoonClassName=com.package.tests
package. Either method is fine. Some parameter to bash console or maybe way to create my own annotation and launch by something like ./gradlew spoonMyTests.
I am grateful for suggestions/help.
From the official docs:
There are numerous ways to run a specific test, or set of tests. You
can use the Spoon --size, --class-name or --method-name options, or
you can use the --e option to pass arguments to the instrumentation
runner, e.g.
--e package=com.mypackage.unit_tests
The following command should work when executing Spoon directly from command line (not from the gradle task)
java -jar spoon-runner-1.1.9-jar-with-dependencies.jar \
--apk ExampleApp-debug.apk \
--test-apk ExampleApp-debug-androidTest-unaligned.apk \
--e package=com.package.tests
You need to find a way to pass this extra param to the Spoon gradle plugin.
UPDATE
From the official doc of Spoon Gradle plugin:
Custom instrumentation arguments
Use the instrumentationArgs property on spoon extension to pass custom
parameters to your tests:
spoon { instrumentationArgs = ["foo=bar", "name=value"] }
In your case, this should look like the following:
spoon {
instrumentationArgs = ["package=com.package.tests"]
}
I've written a unit test that simply extends TestCase and I have the following:
public class MetricParserTests extends TestCase {
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public void testFailure() {
fail("This needs to fail");
}
}
When I run my tests using ant test or adb shell am instrument I get the following results:
... [exec] OK (1 tests) ...
I'd expect to see a failure on the command line.
I believe I know what the issue is. I was able to reproduce the issue and solve it. The command you use does not rebuild and re-install your test project onto a device. When you call ant test it will just execute the tests which are already installed on that device.
What you need to call is the three commands in your test project's directory:
ant debug
ant installd
ant test
Then all tests will be rebuild and re-installed and latest tests will be executed. If you don't call debug and installd, the changes you did to the tests do not get applied.
I haven't had recent experience in Android testing, but here is what I have found...
You can use normal JUnit tests if your code is totally decoupled from Android (see here
for an example). This would run on your JVM using the JUnit runner.
However, if you are trying to run these tests on an Android device (either via ant, or the command line tools) then you need to create a full android test project (See here).
To test "on device" your test cases need to extend one of the Android test classes like ActivityInstrumentationTestCase2<T>
and are run using the InstrumentationTestRunner in the Dalvik VM on the Android device.
Using an IDE or the command-line tools to create a test project should create a sample test for you to work from.
This blog post linked from the comments of the post above is a good source of information, as is the Android Testing Fundamentals doc.
The method testFailure() does not have a #Test annotation. Is that correct?