So with the release of Android Studio Dolphin & Beta of Electric Eel, I wanted to try the instrumentation tests in gradle. I do however want to exclude some of the tests being run, in order to be able to run specific test suites one at a time.
So here is what I configured so far:
android {
testOptions {
managedDevices {
devices {
pixel2api30 (com.android.build.api.dsl.ManagedVirtualDevice) {
device = "Pixel 2"
apiLevel = 30
systemImageSource = "aosp-atd"
}
}
}
}
}
I know I can run my entire suite using
./gradlew device-nameBuildVariantAndroidTest
In my case that would be
./gradlew pixel2api30gaeDebugAndroidTest
gaeDebug being my build variant. This command is being run in my project root.
If I want to run the tests in the tests/large folder for example
How would I go about doing that? Thanks.
For this you could use 2 different approaches:
Create and run test suites.
For example, for each of these folders, create a TestSuite and define Test classes. For example:
#RunWith(Suite.class)
#Suite.SuiteClasses({
ExampleLargeTest.class,
ExampleTwoLargeTest.class,
ExampleThreeLargeTest.class
})
public class LargeTestsSuite {
}
This suite can be run using following command
./gradlew pixel2api30gaeDebugAndroidTest - Pandroid.testInstrumentationRunnerArguments.class=path.to.your.suite.class
Use Test Categories
Annotate your Test classes like this:
#Category("Large")
public class ExampleLargeTest { ... }
And then you could execute the following command for running all tests with same category:
./gradlew pixel2api30gaeDebugAndroidTest -PtestCategory=Large
Hopefully one of these two approaches will suite you.
Related
I have a need to separate running "unit tests" from more specifically named unit tests called "requirements tests". Directory structure like:
src
|__main\
|__test\
|__testRequirements\
I added a separate srcSet so I can run these all together no problem.
sourceSets {
test {
java.srcDirs = ['src/test/java', 'src/testRequirements/java']
}
}
But, I need to be able to run ONLY the testRequirements by command so I can have ONLY them run in their own CI flow. I see that I can target them specifically by using the filtering by package/class/method a la:
./gradlew test --tests *RequirementsTest
But that relies on humans appending the same string to every class name.
I played around a little with creating a new flavor just for requirements so I could access with a command ./gradlew testRequirementsDebugUnitTest but that would run both the requirements AND all my other unit tests
I am interested in how to run Espresso tests from command line (gradle task) individually (run group/suite tests then close app and then run another group/suite of tests).
Found that it is feasible to implement JUnit Test Suites but do not really understand how does it looks like under the hood in a context of instrumentation tests. Does it starts separate processes per Test suite? There is sample application on Github but how to execute it from terminal?
Another interesting discovery is Sharding tests. However, it one sentence documentation.
May be somebody can share with any experience of running Espresso tests individually.
Most of this is documented as part of AndroidJUnitRunner: https://developer.android.com/reference/android/support/test/runner/AndroidJUnitRunner.html
The key piece that is missing is how to pass those parameters via Gradle. You can do that by specifying the options at the commandline as such:
./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=*The full name of your test suite goes here*
I would recommend using the Spoon library to run your tests individually or in parallel on multiple devices. You can either use the jar file or use the Spoon gradle plugin mentioned on the same page. Both of them have adequate documentation to help you set it up.
You can also use Spoon to run an individual test and the command would look something like this:
./gradlew yourSpoonTaskName -PspoonClassName=com.yourPackageName.blah.ClassName
-PspoonMethodName=methodName
In order to know what yourSpoonTaskName is run ./gradlew tasks.
Also, in your build.gradle file add the following spoon configuration:
spoon {
// for debug output
debug = true
// To grant permissions to Android M >= devices
grantAllPermissions = true
// for sharding
/*
this will execute tests in parallel on multiple devices.
*/
shard = true
// Add this to run a specific test class & method
if (project.hasProperty('spoonClassName')) {
className = project.spoonClassName
}
if (project.hasProperty('spoonMethodName')) {
methodName = project.spoonMethodName
}
}
If you are not interested in Spoon and just want a simple solution, then use the following command to run an individual test:
am instrument -w -r -e class com.packageName.blah.TestName#methodName com.packageName.blah.YourIntrumentationRunnerName
You can easily determine these values if you right click the test name in AndroidStudio and run it. In the console, you will see the entire command being printed when the test is bring run.
I'm using Android Studio and the gradle plugin to build apps from the command line.
Now, suppose I want to run tests on my connected devices, I run:
./gradlew app:connectedCheck
and it will output a junit report XML file on:
./app/build/outputs/androidTest-results/connected/flavors/latest/TEST-Galaxy Nexus - 4.3-app-Latest.xml
This is almost what I want. Just that I would like to specify for example this test output file. For example, I would like to output it to $HOME/junit.xml. And I would like to specify this $HOME/junit.xml from the command line.
How do I do this?
You can specify the output directory for test reports using
android {
testOptions {
reportDir = "$project.buildDir/foo/report"
}
}
http://tools.android.com/tech-docs/new-build-system/user-guide#TOC-Test-reports
I've written a unit test that simply extends TestCase and I have the following:
public class MetricParserTests extends TestCase {
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public void testFailure() {
fail("This needs to fail");
}
}
When I run my tests using ant test or adb shell am instrument I get the following results:
... [exec] OK (1 tests) ...
I'd expect to see a failure on the command line.
I believe I know what the issue is. I was able to reproduce the issue and solve it. The command you use does not rebuild and re-install your test project onto a device. When you call ant test it will just execute the tests which are already installed on that device.
What you need to call is the three commands in your test project's directory:
ant debug
ant installd
ant test
Then all tests will be rebuild and re-installed and latest tests will be executed. If you don't call debug and installd, the changes you did to the tests do not get applied.
I haven't had recent experience in Android testing, but here is what I have found...
You can use normal JUnit tests if your code is totally decoupled from Android (see here
for an example). This would run on your JVM using the JUnit runner.
However, if you are trying to run these tests on an Android device (either via ant, or the command line tools) then you need to create a full android test project (See here).
To test "on device" your test cases need to extend one of the Android test classes like ActivityInstrumentationTestCase2<T>
and are run using the InstrumentationTestRunner in the Dalvik VM on the Android device.
Using an IDE or the command-line tools to create a test project should create a sample test for you to work from.
This blog post linked from the comments of the post above is a good source of information, as is the Android Testing Fundamentals doc.
The method testFailure() does not have a #Test annotation. Is that correct?
Is there a way to run a specific Android instrumentation unit test using Gradle? I've tried
gradle -Dtest.single=UnitTestName connectedInstrumentTest
but it seems to run all the tests in the package.
Using test.single appears to be deprecated. The new correct way to do this is
./gradlew :<module>:test --tests <pattern>
where <pattern> could be something like:
com.example.MyTest to run all test methods in com.example.MyTest
*MyTest to match every method in every class whose name ends with MyTest
*.MyTest.myMethod to run a specific test method in class MyTest in any package
If you have a multi-project build, make sure to give the module path before the test task; otherwise you'll get a misleading error message when it searches for your test pattern in every subproject.
None of this is documented on the Gradle site anywhere I could find it.
This works if you're using an instrumentationTestRunner:
./gradlew test -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
Using gradle 2.10 and android gradle plugin 2.0.0-beta2.
Since you know what test(s) you want to run, you probably know which module / flavor to use too. You can help Gradle out by specifying the exact module and Gradle task. So if your test is in the app module and you want to test the debug flavor:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
You can get even more fancy with the tests_regex argument instead:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=PartialClassName*
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=partialMethodName*
The pattern is -D<testTaskName>.single=<TestClass> so in your example it should be:
gradle -DconnectedInstrumentTest.single=UnitTestName connectedInstrumentTest
NOTE: This answer is outdated. You should use the --tests switch in the latest versions of Gradle. (see other answers for an explanation)
Since Android gradle plugin 1.1.0-rc1, one can run single test class using --tests flag by executing:
./gradlew app:testDebug --tests=com.example.MyTest
See http://tools.android.com/tech-docs/unit-testing-support#TOC-Running-from-Gradle
You gotta check this out.
https://github.com/JCAndKSolutions/android-unit-test
I made an issue in this github repository, and this guy solved my problem and upload to maven, so in my build.gradle file I use this plugin.
Instructions are written in his repository. you can easily follow it.
After using this android-unit-test plugin, I can use like
../gradlew -Dtest.single=SomeTest test
or
../gradlew -Dtest.single=SomeTest clean check
Now it's working and I could only run the specific tests I want to
You should not forget to specify a build variant name after test property declaration like
-Dtest<buildVariantName>=<yourTestName>.
Like if you have a debug build type which gives you debug variant after compilation, then if you want to run a test only for this build variant you should declare a command like this:
./gradlew -DtestDebug=UnitTestName testDebug
Erdi's answer didn't work for me but I have a single parent for all my test classes so I was able to do this:
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if(getClass().getSimpleName().equals("MyTestClassName")) {
super.runTest();
}
}
//...
}
This executes only MyTestClassName. We can extend it further to execute only specific test method (or methods):
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if("MyTestClassName".equals(getClass().getSimpleName())
&& "testMethodName".equals(getName())) {
super.runTest();
}
}
//...
}
the Gradle command does not work for me.
I used below mentioened adb command.
for this you need to build your apk first.
adb shell am instrument -w -r -e package -e debug false .debug.test/android.support.test.runner.AndroidJUnitRunner