I'm using the AndroidJUnitRunner in my project, but the unit tests are painfully slow to execute on a device and emulator both in Android Studio and from the command-line via gradlew.
I'm using the master branch of this open-source project, OneBusAway (as of this commit):
https://github.com/OneBusAway/onebusaway-android
I'm seeing execution time of all 142 tests that is upwards of 3 minutes in real elapsed time, but Android only registers a smaller portion of this in the execution time it shows under "Test Results". Before switching to the AndroidJUnitRunner all of these unit tests executed under 20 seconds consistently.
Here's what an example test looks like:
/**
* Tests to evaluate utility methods related to math conversions
*/
#RunWith(AndroidJUnit4.class)
public class MathUtilTest {
#Test
public void testOrientationToDirection() {
// East
double direction = MathUtils.toDirection(0);
assertEquals(90.0, direction);
}
}
Here's the build.gradle config:
android {
dexOptions {
preDexLibraries true
}
compileSdkVersion this.ext.compileSdkVersion
buildToolsVersion "27.0.3"
defaultConfig {
minSdkVersion 14
targetSdkVersion 21
versionCode 93
versionName "2.3.8"
multiDexEnabled true
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
// The following argument makes the Android Test Orchestrator run its
// "pm clear" command after each test invocation. This command ensures
// that the app's state is completely cleared between tests.
testInstrumentationRunnerArguments clearPackageData: 'true'
}
...
testOptions {
execution 'ANDROID_TEST_ORCHESTRATOR'
unitTests.includeAndroidResources true
}
...
}
dependencies {
...
// Unit tests
androidTestImplementation 'com.android.support.test:runner:1.0.2'
androidTestUtil 'com.android.support.test:orchestrator:1.0.2'
...
}
Why is this running unit tests so slow?
Apparently the slowdown is related to including a dependency on Android Test Orchestrator, which I don't need (even if I'm running tests on a device that require an Android context). It wasn't clear to me from the existing documentation that Orchestrator wasn't required for these tests.
I removed the following lines from build.gradle, and my total execution time via Android Studio dropped back down into the ~15 second range:
// The following argument makes the Android Test Orchestrator run its
// "pm clear" command after each test invocation. This command ensures
// that the app's state is completely cleared between tests.
testInstrumentationRunnerArguments clearPackageData: 'true'
...
execution 'ANDROID_TEST_ORCHESTRATOR'
...
androidTestUtil 'com.android.support.test:orchestrator:1.0.2'
So here's the new build.gradle that executes tests in ~15 seconds:
android {
dexOptions {
preDexLibraries true
}
compileSdkVersion this.ext.compileSdkVersion
buildToolsVersion "27.0.3"
defaultConfig {
minSdkVersion 14
targetSdkVersion 21
versionCode 93
versionName "2.3.8"
multiDexEnabled true
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
...
testOptions {
unitTests.includeAndroidResources true
}
...
}
...
dependencies {
...
// Unit tests
androidTestImplementation 'com.android.support.test:runner:1.0.2'
}
I thought that maybe the testInstrumentationRunnerArguments clearPackageData: 'true' was the primary culprit causing increased execution time to clear the package data, but removing that line alone didn't change the total execution time of tests - I had to completely remove the Orchestrator dependency.
Here's the commit on Github that removed Orchestrator:
https://github.com/OneBusAway/onebusaway-android/commit/a1657c443258ac49b1be83a75399cf2ced71080e
Related
I have defined these into my build.gradle, to enable Orchestrator in my Android Instrumented tests:
defaultConfig {
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArguments clearPackageData: 'true'
}
testOptions {
animationsDisabled = true
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}
dependencies {
androidTestImplementation 'androidx.test:runner:1.5.1'
androidTestUtil 'androidx.test:orchestrator:1.4.2'
}
Sadly, when I start tests, data is not cleared and it persists between test runs.
I tried both, start test from Android Studio and by gradle cli ./gradlew connectedStageDebugAndroidTest
Any idea what I am doing wrong?
I am developing an Android application using Kotlin programming language. I am writing instrumented tests for my application. I am also using the test orchestrator to run my instrumented tests. But after the installation of the test orchestrator modifying the Grandle file, I cannot run the test in the Android Studio.
I added the following configuration to the app.grandle file
android {
compileSdkVersion 29
buildToolsVersion "29.0.2"
defaultConfig {
applicationId "com.example.memento"
minSdkVersion 21
targetSdkVersion 29
versionCode 1
versionName "1.0"
// testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
testInstrumentationRunner "com.example.memento.MockTestRunner"
// Cleared between tests.
testInstrumentationRunnerArguments clearPackageData: 'true'
testOptions {
execution 'ANDROID_TEST_ORCHESTRATOR'
}
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
}
}
}
With this
androidTestImplementation 'com.android.support.test:runner:1.1.1'
androidTestUtil 'com.android.support.test:orchestrator:1.1.1'
This is the MockTestRunner class
import android.app.Application
import android.content.Context
import androidx.test.runner.AndroidJUnitRunner
import com.example.utils.MockApplicationController
class MockTestRunner: AndroidJUnitRunner()
{
override fun newApplication(
cl: ClassLoader?,
className: String?,
context: Context?
): Application {
return super.newApplication(cl, MockApplicationController::class.java!!.getName(), context)
}
}
Then, in the Android Studio, I tried to run a test by right-clicking on the code as follows.
This is what I got when I run the test.
$ adb shell CLASSPATH=$(pm path android.support.test.services) app_process / android.support.test.services.shellexecutor.ShellMain am instrument -r -w -e targetInstrumentation com.example.memento.test/com.example.memento.MockTestRunner -e debug false -e class 'com.example.memento.EventListTest#eventListIndexZeroTabRendersCurrentEvents' -e clearPackageData true android.support.test.orchestrator/android.support.test.orchestrator.AndroidTestOrchestrator
Waiting for process to come online...
Started running tests
Test running failed: No test results
Literally, not tests were run. What is wrong with my configuration and how can I fix it. If I remove the test orchestrator installation, I can run the tests and it is working as expected.
When I run multiple espresso tests, I find that the app does not restart but starts from the same location that it is left off. How to make it start from the beginning everytime
Consider Android Test Orchestrator. It will clear all data and restart Instrumentation between each test.
Add to app Gradle:
android {
defaultConfig {
...
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
testOptions {
execution 'ANDROID_TEST_ORCHESTRATOR'
}
}
dependencies {
androidTestImplementation 'com.android.support.test:runner:1.0.1'
androidTestUtil 'com.android.support.test:orchestrator:1.0.1'
}
And then run it as follows:
./gradlew connectedCheck
So I've set up a gradle project with android and tried to get some tests to run. Unfortunately they don't seem to. It's possible that I'm missing something obvious but here goes...
I am running gradle 1.11 and as I understand the documentation that's the new folder (since 0.9 I believe?) that should be used for tests.
So I have my testclass ::
package se.coinhunter.multigradle.test;
import org.junit.Test;
import static org.junit.Assert.*;
public class HelloAndroidTests {
#Test
public void testHelper() {
assertEquals(1,1);
}
}
}
That lives in src/androidTest and here is my build.gradle:
apply plugin: 'android'
android {
compileSdkVersion 19
buildToolsVersion "19.0.3"
defaultConfig {
minSdkVersion 11
targetSdkVersion 19
versionCode 1
versionName "1.0"
}
buildTypes {
release {
runProguard false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
}
}
}
dependencies {
compile 'com.android.support:appcompat-v7:+'
compile fileTree(dir: 'libs', include: ['*.jar'])
compile project(':MultiGradleSubmodule')
}
This is a multi-project build and the submodule mentioned in the dependencies block is a plain java project that has its' own scource and unit tests running quite smoothly. I was able to specify that it should tell me when it runs its' tests and give me some feedback and that works fine. That was achieved for that project using
test {
testLogging {
events 'started', 'passed'
}
}
in its' build.gradle. I havn't come across anything like this for android projects. The whole project builds and runs, but I either can't get the tests to run, or they're running but I'm not getting any output.
You're using jUnit 4 (package name "org.junit" with #Test annotation). Android gradle only works with jUnit 3 (package name "junit.framework" with no annotations).
Android tests run in the Dalvik virtual machine on a device or emulator so your test class should also extend "AndroidTestCase" (or one of the other junit subclasses in Android - depending on what you're testing).
UPDATED: also add the following to default config:
testInstrumentationRunner "android.test.InstrumentationTestRunner"
testFunctionalTest true
Run the test using
gradle connectedCheck
I'm trying to get the androidTest (instrumentation tests) working for the openScale Android app using the following build.gradle:
apply plugin: 'com.android.application'
android {
compileSdkVersion 27
defaultConfig {
applicationId "com.health.openscale"
testApplicationId "com.health.openscale.test"
minSdkVersion 18
targetSdkVersion 22 // don't set target sdk > 22 otherwise bluetooth le discovery need permission to ACCESS_COARSE_LOCATION
versionCode 22
versionName "1.7 (beta)"
javaCompileOptions {
annotationProcessorOptions { arguments = ["room.schemaLocation":"$projectDir/schemas".toString()] }
}
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
sourceSets {
androidTest.assets.srcDirs += files("$projectDir/schemas".toString())
}
lintOptions {
abortOnError false
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
}
}
}
ext {
supportLibVersion = '27.0.2'
}
dependencies {
implementation "com.android.support:design:${supportLibVersion}"
implementation "com.android.support:support-v4:${supportLibVersion}"
implementation "com.android.support:appcompat-v7:${supportLibVersion}"
// HelloCharts
implementation 'com.github.lecho:hellocharts-library:1.5.8#aar'
// Simple CSV
implementation 'com.j256.simplecsv:simplecsv:2.2'
// CustomActivityOnCrash
implementation 'cat.ereza:customactivityoncrash:2.2.0'
// Room
implementation 'android.arch.persistence.room:runtime:1.0.0'
annotationProcessor 'android.arch.persistence.room:compiler:1.0.0'
androidTestImplementation 'android.arch.persistence.room:testing:1.0.0'
// Local unit tests
testImplementation 'junit:junit:4.12'
// Instrumented unit tests
androidTestImplementation "com.android.support:support-annotations:${supportLibVersion}"
androidTestImplementation 'com.android.support.test:runner:1.0.1'
androidTestImplementation 'com.android.support.test:rules:1.0.1'
}
tasks.withType(Test) {
testLogging {
exceptionFormat "full"
events "started", "skipped", "passed", "failed"
showStandardStreams true
}
}
There's a single test file under android_app/app/src/androidTest/java/com.health.openscale/DatabaseTest.java
I'm using the command ./gradlew --no-daemon --no-build-cache -i connectedDebugAndroidTest to build and run the tests and it works fine the first time: both tests in the file are executed and pass.
But now, if I change the test file (small change to trigger a rebuild) and run the above command again I get the following output:
...
Putting task artifact state for task ':app:compileDebugAndroidTestJavaWithJavac' into context took 0.0 secs.
file or directory '/<path>/android_app/app/src/androidTestDebug/java', not found
Executing task ':app:compileDebugAndroidTestJavaWithJavac' (up-to-date check took 0.007 secs) due to:
Input property 'source' file /<path>/android_app/app/src/androidTest/java/com.health.openscale/DatabaseTest.java has changed.
Compiling with source level 1.7 and target level 1.7.
...
Putting task artifact state for task ':app:transformClassesWithDexBuilderForDebugAndroidTest' into context took 0.0 secs.
Executing task ':app:transformClassesWithDexBuilderForDebugAndroidTest' (up-to-date check took 0.024 secs) due to:
Input property '$3' file /<path>/android_app/app/build/intermediates/classes/androidTest/debug/com/health/openscale/DatabaseTest.class has been removed.
Transform inputs calculations based on following changes
/<path>/android_app/app/build/intermediates/classes/androidTest/debug/com/health/openscale/DatabaseTest.class:REMOVED
...
Starting 0 tests on Nexus_5X_API_26(AVD) - 8.0.0
[XmlResultReporter]: XML test result file generated at /<path>/android_app/app/build/outputs/androidTest-results/connected/TEST-Nexus_5X_API_26(AVD) - 8.0.0-app-.xml. Total tests 0,
com.android.builder.testing.ConnectedDevice > No tests found.[Nexus_5X_API_26(AVD) - 8.0.0] FAILED
No tests found. This usually means that your test classes are not in the form that your test runner expects (e.g. don't inherit from TestCase or lack #Test annotations).
[XmlResultReporter]: XML test result file generated at /<path>/android_app/app/build/outputs/androidTest-results/connected/TEST-Nexus_5X_API_26(AVD) - 8.0.0-app-.xml. Total tests 1, failure 1,
...
Does anyone know why the rebuild doesn't work? Why is the class removed from the build?
I have verified with the APK analyzer in Android studio that the DatabaseTest class is indeed missing from the APK after the rebuild.
It seems that moving the test case from .../com.health.openscale/DatabaseTest.java to .../com/health/openscale/DatabaseTest.java fixed the problem.