Testing an android app generating specific xml junit file - android

I'm using Android Studio and the gradle plugin to build apps from the command line.
Now, suppose I want to run tests on my connected devices, I run:
./gradlew app:connectedCheck
and it will output a junit report XML file on:
./app/build/outputs/androidTest-results/connected/flavors/latest/TEST-Galaxy Nexus - 4.3-app-Latest.xml
This is almost what I want. Just that I would like to specify for example this test output file. For example, I would like to output it to $HOME/junit.xml. And I would like to specify this $HOME/junit.xml from the command line.
How do I do this?

You can specify the output directory for test reports using
android {
testOptions {
reportDir = "$project.buildDir/foo/report"
}
}
http://tools.android.com/tech-docs/new-build-system/user-guide#TOC-Test-reports

Related

Run specific instrumentation test suite with Gradle Managed Devices - Android

So with the release of Android Studio Dolphin & Beta of Electric Eel, I wanted to try the instrumentation tests in gradle. I do however want to exclude some of the tests being run, in order to be able to run specific test suites one at a time.
So here is what I configured so far:
android {
testOptions {
managedDevices {
devices {
pixel2api30 (com.android.build.api.dsl.ManagedVirtualDevice) {
device = "Pixel 2"
apiLevel = 30
systemImageSource = "aosp-atd"
}
}
}
}
}
I know I can run my entire suite using
./gradlew device-nameBuildVariantAndroidTest
In my case that would be
./gradlew pixel2api30gaeDebugAndroidTest
gaeDebug being my build variant. This command is being run in my project root.
If I want to run the tests in the tests/large folder for example
How would I go about doing that? Thanks.
For this you could use 2 different approaches:
Create and run test suites.
For example, for each of these folders, create a TestSuite and define Test classes. For example:
#RunWith(Suite.class)
#Suite.SuiteClasses({
ExampleLargeTest.class,
ExampleTwoLargeTest.class,
ExampleThreeLargeTest.class
})
public class LargeTestsSuite {
}
This suite can be run using following command
./gradlew pixel2api30gaeDebugAndroidTest - Pandroid.testInstrumentationRunnerArguments.class=path.to.your.suite.class
Use Test Categories
Annotate your Test classes like this:
#Category("Large")
public class ExampleLargeTest { ... }
And then you could execute the following command for running all tests with same category:
./gradlew pixel2api30gaeDebugAndroidTest -PtestCategory=Large
Hopefully one of these two approaches will suite you.

Android: after Gradle update to 6.7.1 ClassLoader in JUnit test no longer lists all resources

I need to iterate over specific classes from main package in my android unit test, to check some of their properties.
For this I use standard approach, using ClassLoader:
val classLoader = Thread.currentThread().contextClassLoader
val resources: Enumeration<URL> = classLoader.getResources("com/models/package")
assert(resources.hasMoreElements()) // Fails from CL, works in AS
Before the Gradle update (had Gradle 5.6.4) that worked. Now the behaviour is as follows: it works when test is run from Android Studio, but fails (returns empty enumeration) when run from command line with gradlew.
I wonder what might be the difference in this respect between the two Gradle versions? And why it still works when run from Studio?
Some considerations and things I have tried:
Referencing these classes in unit test works ok, and also classLoader.findClass("com.models.package.MyModel") and
classLoader.loadClass("com.models.package.MyModel") from unit test is working. But even after that classLoader.getResources("com/models/package") returns empty enumeration.
Using other references to ClassLoader, like MyModel::class.java.classLoader and ClassLoader.getSystemClassLoader() didn't make any difference.
Gradle build from command line contains the warning "OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended", but as far as I can tell it's not connected to my issue.
If I put some of the classes from 'com/models/package' to the unit test /test folder, they are getting returned in enumeration.
This might be connected with some new optimisation setting that makes ClassLoaders omit registering some of the classes, in different root directories, but as it still works in AS there might be some setting to turn this optimisation off in a command line build also?
Thank you for any suggestions on this.
In Gradle 6.7.1 I had to include the directory with the code to the test sourceSets. Afterwards the classloader from junit started to see the classes and return them in Enumeration.
sourceSets {
test {
java.srcDirs += ['src/main']
}
}

Is it possible in to only run tests in a certain _directory_ via command line (in Android Studio/Gradle)?

I have a need to separate running "unit tests" from more specifically named unit tests called "requirements tests". Directory structure like:
src
|__main\
|__test\
|__testRequirements\
I added a separate srcSet so I can run these all together no problem.
sourceSets {
test {
java.srcDirs = ['src/test/java', 'src/testRequirements/java']
}
}
But, I need to be able to run ONLY the testRequirements by command so I can have ONLY them run in their own CI flow. I see that I can target them specifically by using the filtering by package/class/method a la:
./gradlew test --tests *RequirementsTest
But that relies on humans appending the same string to every class name.
I played around a little with creating a new flavor just for requirements so I could access with a command ./gradlew testRequirementsDebugUnitTest but that would run both the requirements AND all my other unit tests

Testing inconvenience: Android Studio JUnit vs Gradle based: testOptions ignored by Android Studio

The following was done with Android Studio 3.4, Android Gradle Plugin 3.3.2 and Gradle 4.10.3.
In the build.gradle file, I have configured some unit test options like this:
android {
testOptions {
unitTests.all {
systemProperty "debug","true"
}
}
}
I do have a test function that tries to read this property:
package com.demo;
public class SysPropTestDemo {
#Test
public static void dumpSysProps() {
System.out.println("sysprop(debug)=" + System.getProperty("debug"));
}
}
When run via command line gradlew test --test com.demo.SysPropTestDemo I will get the property debug set correctly to true. If I run the same test via Android Studio without setting any options, the value shown will be null.
In order to get the same result from Android Studio, I explicitly have to enter some values in the "Run/Debug Configurations" panel, i.e something like -Ddebug=true in the VM options.
Now this is a trivial example, but what I really want to do, is to add some path to the java.library.path property in order to be able to load a JNI library compiled within the project. (I do need to write some tests that make use a modified SQLite lib, so not using JNI is not an option here)
It does work when setting additional options, but I think this is very inconvenient, since I can't enter a variable based value in the configuration options (or at least, I don't know how to). To sum it up: when setting or changing values, I do have to go through a bunch of config screens where I would really prefer to have one place in a config file.
Shouldn't Android Studio somehow make use of the values specified in the build.gradle file? If not, the docs don't make it clear that the testOptions.unitTests.all settings can only be used via gradlew invocation.
Skybow,
I feel you have two questions
1. How to load jni lib for androidTest(not for 'test[non instrumented unit tests])
- copy your jni library in corresponding folder [JNI libraries: [app/src/androidTestFLAVORNAMEDebug/jniLibs]
- load your jni library
static {
try {
System.loadLibrary("xyzjni");
} catch (Exception e) {
Logger.error("Exception on loading the jni library : " + e.getMessage());
}
}
2. How to make android studio use your config variables defined for unitTests.
- It would have great if some text file is there which has all configs.
- Or it is part of build.gradle
- I don't have any detail on this.

Android Espresso Test Suites

I am interested in how to run Espresso tests from command line (gradle task) individually (run group/suite tests then close app and then run another group/suite of tests).
Found that it is feasible to implement JUnit Test Suites but do not really understand how does it looks like under the hood in a context of instrumentation tests. Does it starts separate processes per Test suite? There is sample application on Github but how to execute it from terminal?
Another interesting discovery is Sharding tests. However, it one sentence documentation.
May be somebody can share with any experience of running Espresso tests individually.
Most of this is documented as part of AndroidJUnitRunner: https://developer.android.com/reference/android/support/test/runner/AndroidJUnitRunner.html
The key piece that is missing is how to pass those parameters via Gradle. You can do that by specifying the options at the commandline as such:
./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=*The full name of your test suite goes here*
I would recommend using the Spoon library to run your tests individually or in parallel on multiple devices. You can either use the jar file or use the Spoon gradle plugin mentioned on the same page. Both of them have adequate documentation to help you set it up.
You can also use Spoon to run an individual test and the command would look something like this:
./gradlew yourSpoonTaskName -PspoonClassName=com.yourPackageName.blah.ClassName
-PspoonMethodName=methodName
In order to know what yourSpoonTaskName is run ./gradlew tasks.
Also, in your build.gradle file add the following spoon configuration:
spoon {
// for debug output
debug = true
// To grant permissions to Android M >= devices
grantAllPermissions = true
// for sharding
/*
this will execute tests in parallel on multiple devices.
*/
shard = true
// Add this to run a specific test class & method
if (project.hasProperty('spoonClassName')) {
className = project.spoonClassName
}
if (project.hasProperty('spoonMethodName')) {
methodName = project.spoonMethodName
}
}
If you are not interested in Spoon and just want a simple solution, then use the following command to run an individual test:
am instrument -w -r -e class com.packageName.blah.TestName#methodName com.packageName.blah.YourIntrumentationRunnerName
You can easily determine these values if you right click the test name in AndroidStudio and run it. In the console, you will see the entire command being printed when the test is bring run.

Categories

Resources