Develop Reference

Coverage for android tests using orchestrator

I am working on a project where in we are trying to use the ANDROID TEST ORCHESTRATOR for it's obvious benefits of isolating crashes. But while executing the test suite, it appears to me that as the orchestrator initiates a new process for every test case, the test coverage report for the suite execution is always showing incomplete data (mostly the data for the last test case present in the test suite).
So I was wondering that is there a way to overcome this problem and generate a jacoco code coverage report for all instrumented tests existing in the test suite.

If you're using the android test orchestrator this is the problem. There is an open bug report: https://issuetracker.google.com/issues/72758547
The only solution that I know of is to disable the android test orchestration until a fix is released.
In my Android Java build.gradle I had to comment out the test orchestrator like so:
android {
...
testOptions {
// temporarily disable the orchestrator as this breaks coverage: https://issuetracker.google.com/issues/72758547
//execution 'ANDROID_TEST_ORCHESTRATOR'
...
}
}

The issue mentioned above is now fixed and the latest gradle version is working fine for all android devices.

Android Studio 2.3 doesn't have Test Instrumentation specifier in UI

After updating to Android Studio 2.3 when I try to run some espresso tests I get the following error:
Test running failed: Unable to find instrumentation info for: ComponentInfo{com.example.android/android.test.InstrumentationTestRunner}
Empty test suite.
This was easily fixable in the past where in the Run Configuration I could specify my own InstrumentationRunner. Now I can't seem to find this option so I can't really specify my runner class now.
Note that my build gradle does contain
defaultConfig {
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
resConfigs "en", "es"
}

I ran into this problem this morning. I deleted the old run configuration (the one that was created before you specified the new runner via build.gradle). I re-ran the tests and the new runner was picked up by Android Studio.

On Android Studio 2.3 this feature is not available as this was an optional to specify the Instrumentation Runner package/class in previous versions of Studio. But Android 2.3 is smart enough to pick this from your build.gradle file provided you have defined Instrumentation package runner there.

The following setup fixed the issue
productFlavors {
doTheTests {
minSdkVersion 18
testInstrumentationRunner "com.company.app.test.TestRunner"
if (System.getenv('CONTINUOUS_INTEGRATION').equals("true")) {
testInstrumentationRunnerArguments(package: "com.company.app.test")
}
}
}
Because Android Studio doesn't allow the testInstrumentationRunnerArguments but executing the tests through terminal indeed requires it!
Obviously on the terminal / CI system, set an environment variable like:
export CONTINUOUS_INTEGRATION=true

What helped to me - starting the test from command line then syncing the project.

Android Studio 2.1.2 Local Unit Test steps

When attempting to run a 'hello world' type 'local unit test' i get the error
Test running failed: Instrumentation run failed due to 'java.lang.RuntimeException'
Looking at documentation at https://developer.android.com/training/testing/unit-testing/local-unit-tests.html
i have verified the following is in the app/build.gradle
testCompile 'junit:junit:4.12'
I would like concise steps on how to setup 'local unit testing' from 'File>New>New Project' with an empty activity.
Testing the method addition_isCorrect or whole class .ExampleUnitTest
Additionally my setup starts an emulated device but I understand that local unit testing did not need a device or emulator?
Also it appears that the Test Artifact in the Build Variants view. is obsolete now.

The answer is to create a new JUnit Configuration.
Not a Android test that seems to work with ApplicationTest.
Run>Edit Configurations>Add a new configuration>
Choose your module and pick class .ExampleUnitTest

Android Studio 1.1, simple junit test setup

I have read around, there are a number of extensive answers (like this one) but the Android world evolves so fast that they seem to be a bit outdated and the official documentation still refers to Eclipse with ADT.
I am running AS 1.1 and I am trying to setup simple junit tests to run on the emulator, without Robolectric. If I don't include junit in my build.gradle, it can't find #After, #Before and #Test and I get package org.junit does not exist. Upon adding
// unit tests
androidTestCompile 'junit:junit:4.11'
the error becomes
Error:duplicate files during packaging of APK
[...]/app/build/outputs/apk/app-debug-test-unaligned.apk
Path in archive: LICENSE.txt
Origin 1: [...]/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar
Origin 2: [...]/.gradle/caches/modules-2/files-2.1/junit/junit/4.11/4e031bb61df09069aeb2bffb4019e7a5034a4ee0/junit-4.11.jar
You can ignore those files in your build.gradle:
android {
packagingOptions {
exclude 'LICENSE.txt'
}
}
Following the console suggestion of excluding LICENSE.txt, it then works but it feels like a hack. So I'm wondering, am I maybe missing something? Thanks.

Android Studio unit testing support comes in 1.1 Beta 4 (release announcement) with Gradle plugin version 1.1.0-rc1.
More info in official document.
However it is experimental feature for now. E.g. it breaks installDebug gradle task.
For using JUnit in instrumentation tests there is good guide for Espresso library and another covering new AndroidJUnitRunner.

If it's any use I set up a boiler plate project allowing the use of Unit tests and Espresso tests by the use of switching build variants. You won't need the use of any third party plugins with this.
https://github.com/hitherejoe/Android-Boilerplate

Android gradle test framework: single class

Is it possible to run a single test class using the new Android gradle build framework?
I have a test package that has multiple test classes (All of them are InstrumentationTestCase classes). I've been able to setup my build.gradle file to run the test package
defaultConfig{
testPackageName "com.company.product.tests"
testInstrumentationRunner "android.test.InstrumentationTestRunner"
}
But is there a way to test only one test case in that package? Otherwise I'll be using the age old adb shell am instrument -w .......
P.S. I don't have time right now to switch to Roboelectric, but I do see that its pretty much the defacto framework nowadays.

Using android.test.InstrumentationTestRunner no, this is not possible. You do, however, have options:
Custom Test Runner
Extend android.test.InstrumentationTestRunner
Add a buildConfigField 'String', 'TEST_NAME', '${testName}', where testName is '"${project.properties.get("test")}"' if set, otherwise null
In your runner, only run tests that match BuildConfig.TEST_NAME (if null, run all tests)
Replace the testInstrumentationRunner with your custom runner
Run tests with ./gradlew connectedCheck -Ptest=com.example.Test.testSomething
Use Spoon
Spoon is an excellent test runner extension that, among other things (like beautiful multi-device test reports), lets you run individual tests. Since you're using gradle, I recommend the Spoon Gradle Plugin, which has an example of exactly what you want to do in its README.
Update: Run Individual Unit Tests with Android Gradle Plugin
With the addition of unit testing support, Android now supports running individul unit tests.
This is just an anchor task, actual test tasks are called testDebug and testRelease etc. If you want to run only some tests, using the gradle --tests flag, you can do it by running ./gradlew testDebug --tests='*.MyTestClass'.
Source
Edit: I should also add that Spoon is a drop-in replacement for running tests. You will not have to modify anything but a few lines in your build script to use it.

As answered in https://stackoverflow.com/a/32603798 there is now
android.testInstrumentationRunnerArguments.class