Develop Reference

Test Orchestrator Sample

Is anyone aware of a sample project that shows how to get test orchestrator working? I checked the google samples and there doesn't seem to be a good sample project that shows test orchestrator.
https://github.com/googlesamples/android-testing
I have been attempting to get android tests running in the test orchestrator but have been struggling to get it to work correctly. I tried both running the tests through Android Studio (latest 3.2.1) as well as the command line (https://developer.android.com/training/testing/junit-runner#ato-command-line). I used the Android developer document for reference.
https://developer.android.com/training/testing/junit-runner
Here are the steps I followed.
1) Create an empty activity application using the wizard in Android
Studio
2) Enable the test orchestrator using the steps provided here
(https://developer.android.com/training/testing/junit-runner).
3) Run the unit tests from within the IDE and from the command line.
When I do this, I get an error indicating that my "test suite is empty". I get the same error running from command-line.
Note that if I run the test without test orchestrator, then the test runs successfully.
Also note that I am using the latest test orchestrator versions
test-orchestrator (https://maven.google.com/androidx/test/orchestrator/1.1.0/orchestrator-1.1.0.apk)
test-services (https://maven.google.com/androidx/test/services/test-services/1.1.0/test-services-1.1.0.apk)

The complete configuration to run test orchestrator:
Add dependencies:
androidTestImplementation "androidx.test:runner:$testRunner"
androidTestUtil "androidx.test:orchestrator:$testOrchestrator"
Add clear package instruction (within defaultConfig in app's build.gradle):
//allows run all tests on an isolated way. If we need to debug a test, should disable this and the orchestrator
testInstrumentationRunnerArguments clearPackageData: 'true'
Add to testOptions Android/AndroidX orchestrator:
testOptions {
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}

Questions about Android Unit Test with gradle or android studio

I am writing some test cases in my project and I have bunch questions about android unit test with android studio & gradle I met in recent days and cannot get the good answers after searching.
Here are situations I have met and they are really bothering me.
Situation 1:
When I put the test cases in androidTest folder and run graldew cAT and also use the annotation #AndroidJUnit4 for the test class. I can run the unit test and get the right results. But after I just changed the #AndroidJUnit4 to #MockitoJUnitRunner (I need mock the context). The android studio or gradle cannot find any test, only tell "empty suit".
Situation 2:
When I put the test cases in test folder and run graldew test and also use the annotation #MockitoJUnitRunner. It can find the tests successfully. But I guess it only run locally. If the test cases dependent some native lib, it will give the failure message like: java.lang.UnsatisfiedLinkError: no libxx in java.library.path
So here are questions:
Question 1:
It seems that we can put the test code in src/test or src/androidTest. So what are differences between these two folders?
Question 2:
What are differences between gradlew cAT and gradle test? Are these two commands related to folder (test/androidTest folder I mentioned) in projects?
Question 3:
In my situation, I need write some test cases, which dependent the Context and native so. What should I do for that?

As this is actually 3 questions, it might have been best to create 3 separate SO questions. In any event, here are the answers:
Answer 1:
The src/test folder is meant for "regular" JUnit unit tests. These are tests that run in a regular JVM.
The src/androidTest folder is meant for any tests that require an Android device or emulator to be running. This is where your Espresso tests would live.
Answer 2:
The command gradlew cAT or gradlew connectedAndroidTest runs any tests which require a connected device (cAT standing for Connected Android Test) that are in the src/androidTest directory, while the command gradle test runs just unit tests, in src/test directory.
Answer 3:
If your unit tests depend on Context, consider using Robolectric for your unit tests. This will give you access to Context. Good examples for how to use Robolectric can be found in Corey Latislaw's "Android Katas" repo.
EDIT
Situation 1:
I'm not sure if this is what you are experiencing, but when I have run into this "Empty test suite" error (when I clearly have tests in the directory), it was because I forgot to include the Android JUnit Test Instrumentation Runner in my module's build.gradle file. Include it in the defaultConfig section of your android section, like this:
android {
compileSdkVersion 23
buildToolsVersion "23.0.2"
defaultConfig {
applicationId "com.kiodev.example"
minSdkVersion 16
targetSdkVersion 23
versionCode 1
versionName "1.0"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}

How solve NoClassDefFoundError on unit test using new test support on Android Studio 1.1

Using Android Studio Beta 4 with the new android build plugin 1.1.0-rc
I make some test under src/test and if no dependencies with Android framework work fine, but just a simple Log.d() or Uri.parse throw a NoClassDefFoundError
How can i fix it?

Try running ./gradlew test from the terminal then running your tests in Android Studio again.

Finally I found the issue, i need to add those lines to my build.gradle
testOptions {
unitTests.returnDefaultValues = true
}

Android Studio Unit Testing: unable to find instrumentation OR class not found ex

I have a test suite within my Android studio that has the following dir structure:
-MainProject
-src
--com
--tests
--java.xx.xxx.xx.test
Within my AndroidManifest, I have the following:
<instrumentation
android:name="android.test.InstrumentationTestRunner"
android:targetPackage="tests.java.xxx.xxx.xxx.test" />
When I run my tests from the command line using: ./gradlew connectedInstrumentTest, I simply get:
Tests on Nexus 5 - 4.4.2 failed: Instrumentation run failed due to 'java.lang.ClassNotFoundException'
When I run the tests from inside Android Studio I get:
Running tests
Test running startedTest running failed: Unable to find instrumentation info for: ComponentInfo{tests.xxx.xxx.xxx.xxx.test/android.test.InstrumentationTestRunner}
Empty test suite.
I also have the testPackageName specified in my defaultConfig in the build.gradle file. Am running Android Studio 0.4.6 on Ubuntu 12.0.4.
Any ideas?

I had the same problem and could not find anything on the net. The answer was pretty simple: You have to edit the RunConfiguration of your test so that it is not a Android Test but a JUnit test.
So basically I did the following:
Clicked "Edit Configuration"
Clicked "Add new Configuration" and chose JUnit
Inserted a name and the and chose "All in package" at Test kind.
That's it.

I had the same problem, what happened to me was an error with the configurations of tests.
Be sure that you have the correct base configurations for testing like those below in the image:
If you don't have an Android Instrumented Tests configuration, you can create one from the yellow arrow.
PS: Moreover check that classes referenced by configurations really
exists and the package name is correct

I also get error like this and I added the following line in build.gradle file as shown in https://google.github.io/android-testing-support-library/docs/espresso/setup/index.html and it's working. I hope it will also help you :-)
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"

Basically three things to cross check:
Make sure your have added testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
in your defaultconfig of build.gradle of that project/ module (in case of sub module)
Make sure you have added following test compile dependencies:
androidTestCompile 'com.android.support.test:rules:0.5'
androidTestCompile 'com.android.support:support-annotations:25.2.0'
androidTestCompile 'com.android.support.test:runner:0.5'
Make sure you are adding your test files to the AndroidTest folders and not just to test folder.

I tried various methods but none fixed the issue for me. What I did is simply make a copy of the java test file into the same exact folder but with a different name and deleted the original.
I believe this issue was originally caused by moving my test class from the project test folder into the androidTest folder.

I removed the entry android.support.test.runner.AndroidJUnitRunner from the Android Test configuration and it started to run the tests for me

Android gradle test framework: single class

Is it possible to run a single test class using the new Android gradle build framework?
I have a test package that has multiple test classes (All of them are InstrumentationTestCase classes). I've been able to setup my build.gradle file to run the test package
defaultConfig{
testPackageName "com.company.product.tests"
testInstrumentationRunner "android.test.InstrumentationTestRunner"
}
But is there a way to test only one test case in that package? Otherwise I'll be using the age old adb shell am instrument -w .......
P.S. I don't have time right now to switch to Roboelectric, but I do see that its pretty much the defacto framework nowadays.

Using android.test.InstrumentationTestRunner no, this is not possible. You do, however, have options:
Custom Test Runner
Extend android.test.InstrumentationTestRunner
Add a buildConfigField 'String', 'TEST_NAME', '${testName}', where testName is '"${project.properties.get("test")}"' if set, otherwise null
In your runner, only run tests that match BuildConfig.TEST_NAME (if null, run all tests)
Replace the testInstrumentationRunner with your custom runner
Run tests with ./gradlew connectedCheck -Ptest=com.example.Test.testSomething
Use Spoon
Spoon is an excellent test runner extension that, among other things (like beautiful multi-device test reports), lets you run individual tests. Since you're using gradle, I recommend the Spoon Gradle Plugin, which has an example of exactly what you want to do in its README.
Update: Run Individual Unit Tests with Android Gradle Plugin
With the addition of unit testing support, Android now supports running individul unit tests.
This is just an anchor task, actual test tasks are called testDebug and testRelease etc. If you want to run only some tests, using the gradle --tests flag, you can do it by running ./gradlew testDebug --tests='*.MyTestClass'.
Source
Edit: I should also add that Spoon is a drop-in replacement for running tests. You will not have to modify anything but a few lines in your build script to use it.

As answered in https://stackoverflow.com/a/32603798 there is now
android.testInstrumentationRunnerArguments.class