Develop Reference

Test Orchestrator Sample

Is anyone aware of a sample project that shows how to get test orchestrator working? I checked the google samples and there doesn't seem to be a good sample project that shows test orchestrator.
https://github.com/googlesamples/android-testing
I have been attempting to get android tests running in the test orchestrator but have been struggling to get it to work correctly. I tried both running the tests through Android Studio (latest 3.2.1) as well as the command line (https://developer.android.com/training/testing/junit-runner#ato-command-line). I used the Android developer document for reference.
https://developer.android.com/training/testing/junit-runner
Here are the steps I followed.
1) Create an empty activity application using the wizard in Android
Studio
2) Enable the test orchestrator using the steps provided here
(https://developer.android.com/training/testing/junit-runner).
3) Run the unit tests from within the IDE and from the command line.
When I do this, I get an error indicating that my "test suite is empty". I get the same error running from command-line.
Note that if I run the test without test orchestrator, then the test runs successfully.
Also note that I am using the latest test orchestrator versions
test-orchestrator (https://maven.google.com/androidx/test/orchestrator/1.1.0/orchestrator-1.1.0.apk)
test-services (https://maven.google.com/androidx/test/services/test-services/1.1.0/test-services-1.1.0.apk)

The complete configuration to run test orchestrator:
Add dependencies:
androidTestImplementation "androidx.test:runner:$testRunner"
androidTestUtil "androidx.test:orchestrator:$testOrchestrator"
Add clear package instruction (within defaultConfig in app's build.gradle):
//allows run all tests on an isolated way. If we need to debug a test, should disable this and the orchestrator
testInstrumentationRunnerArguments clearPackageData: 'true'
Add to testOptions Android/AndroidX orchestrator:
testOptions {
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}

Android Studio: How to run tests in Firebase with Orchestrator enabled?

Is there a way to run instrumentation tests in Firebase Test Lab from Android Studio with Android Test Orchestrator enabled?
In my build.gradle I have all of the required statements:
android {
defaultConfig {
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArguments clearPackageData: 'true'
}
testOptions {
execution 'ANDROID_TEST_ORCHESTRATOR'
}
}
dependencies {
androidTestImplementation 'com.android.support.test:runner:1.0.2'
androidTestUtil 'com.android.support.test:orchestrator:1.0.2'
}
When I run tests on my own physical device, Orchestrator is working.
I was able to create a Run/Debug Configuration in Android Studio to run tests in Firebase Test Lab and they run successfully, but with no Orchestrator. Is there an option to enable Orchestrator in Android Studio so that it would also work in Firebase runs?

Android Studio 2.3 doesn't have Test Instrumentation specifier in UI

After updating to Android Studio 2.3 when I try to run some espresso tests I get the following error:
Test running failed: Unable to find instrumentation info for: ComponentInfo{com.example.android/android.test.InstrumentationTestRunner}
Empty test suite.
This was easily fixable in the past where in the Run Configuration I could specify my own InstrumentationRunner. Now I can't seem to find this option so I can't really specify my runner class now.
Note that my build gradle does contain
defaultConfig {
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
resConfigs "en", "es"
}

I ran into this problem this morning. I deleted the old run configuration (the one that was created before you specified the new runner via build.gradle). I re-ran the tests and the new runner was picked up by Android Studio.

On Android Studio 2.3 this feature is not available as this was an optional to specify the Instrumentation Runner package/class in previous versions of Studio. But Android 2.3 is smart enough to pick this from your build.gradle file provided you have defined Instrumentation package runner there.

The following setup fixed the issue
productFlavors {
doTheTests {
minSdkVersion 18
testInstrumentationRunner "com.company.app.test.TestRunner"
if (System.getenv('CONTINUOUS_INTEGRATION').equals("true")) {
testInstrumentationRunnerArguments(package: "com.company.app.test")
}
}
}
Because Android Studio doesn't allow the testInstrumentationRunnerArguments but executing the tests through terminal indeed requires it!
Obviously on the terminal / CI system, set an environment variable like:
export CONTINUOUS_INTEGRATION=true

What helped to me - starting the test from command line then syncing the project.

Run android unit tests & instrumentation tests on Jenkins (Gradle)

I have some unit and instrumentation tests in my androidTest folder.
I'm trying to run these tests on my local Jenkins.
I've successfully configured my project with Jenkins and I've also created an emulator for the instrumentation tests.
So far all the resources I've seen only focus on ant and earlier versions of Gradle. The syntax for writing tasks has slightly changed in the current version and I'm confused about this.
Here is my build.gradle file:
apply plugin: 'com.android.application'
android {
compileSdkVersion 21
buildToolsVersion "21.1.2"
defaultConfig {
applicationId "myPackageName"
minSdkVersion 16
targetSdkVersion 21
versionCode 1
versionName "1.0"
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
}
dependencies {
compile 'com.android.support:appcompat-v7:21.0.3'
compile 'com.google.android.gms:play-services:6.5.87'
}

You can write a jenkis script like this:
stage('Build & Install') {
//Build the apk and the test apk which will run the tests on the apk
sh 'chmod +x gradlew && ./gradlew --no-daemon --stacktrace clean :app:assembleDevDebug :app:assembleDevDebugAndroidTest'
}
stage('Tests') {
//Start all the existing tests in the test package
sh './gradlew --no-daemon --debug :app:connectedDevDebugAndroidTest'
}
This should install the apk and the test apk into the device and start the test cases in the test apk on installation.
Here I have given DevDebug as I have a flavor constant called Dev and build type is Debug. If you don't have any of those, you don't use them.

Create a job in junkins and (configure adb path) add this command to build steps as execute shell command or as a windows bat cmd
$ adb shell am instrument -w com.xyz.abc.test/android.test.InstrumentationTestRunner
P.S :- For better automation use robotium and spoon with junkins you can automate everything, you push a commit on git and you will get test results in your mail inbox it's that cool.
Edit
Running tests using spoon
Add these commands to build steps
./gradlew assembleDebugAndroidTest
./gradlew assembleDebug
specify debug-build.apk path and test-unaligned.apk path in spoon command properly
java -jar C:\Users\Shivam\Downloads\spoon-runner-1.1.1-jar-with-dependencies.jar --apk C:\Users\Shivam\Downloads\SpoonAndRobotiumTest\app\build\outputs\
apk\app-debug.apk --testapk C:\Users\Shivam\Downloads\SpoonAndRobotiumTest\
app\build\outputs\apk\app-debug androidTest-unaligned.apk --sdk E:\sdk

At first, you must install the Gradle Plugin on your Jenkins-CI installation. You can search it at http://yourhost/jenkins/pluginManager/
To continue integrating it, you can take a look at this presentation, specially on the lastest slides

Android gradle test framework: single class

Is it possible to run a single test class using the new Android gradle build framework?
I have a test package that has multiple test classes (All of them are InstrumentationTestCase classes). I've been able to setup my build.gradle file to run the test package
defaultConfig{
testPackageName "com.company.product.tests"
testInstrumentationRunner "android.test.InstrumentationTestRunner"
}
But is there a way to test only one test case in that package? Otherwise I'll be using the age old adb shell am instrument -w .......
P.S. I don't have time right now to switch to Roboelectric, but I do see that its pretty much the defacto framework nowadays.

Using android.test.InstrumentationTestRunner no, this is not possible. You do, however, have options:
Custom Test Runner
Extend android.test.InstrumentationTestRunner
Add a buildConfigField 'String', 'TEST_NAME', '${testName}', where testName is '"${project.properties.get("test")}"' if set, otherwise null
In your runner, only run tests that match BuildConfig.TEST_NAME (if null, run all tests)
Replace the testInstrumentationRunner with your custom runner
Run tests with ./gradlew connectedCheck -Ptest=com.example.Test.testSomething
Use Spoon
Spoon is an excellent test runner extension that, among other things (like beautiful multi-device test reports), lets you run individual tests. Since you're using gradle, I recommend the Spoon Gradle Plugin, which has an example of exactly what you want to do in its README.
Update: Run Individual Unit Tests with Android Gradle Plugin
With the addition of unit testing support, Android now supports running individul unit tests.
This is just an anchor task, actual test tasks are called testDebug and testRelease etc. If you want to run only some tests, using the gradle --tests flag, you can do it by running ./gradlew testDebug --tests='*.MyTestClass'.
Source
Edit: I should also add that Spoon is a drop-in replacement for running tests. You will not have to modify anything but a few lines in your build script to use it.

As answered in https://stackoverflow.com/a/32603798 there is now
android.testInstrumentationRunnerArguments.class