Develop Reference

Jenkins Android building pipeline from different Git repositories

Hi, I would like to create a job in Jenkins where I would like to build one apk from 2 Android repositories.
I tried it with both Jenkinsfile and Gradle wrapper in the Jenkins GUI, but both are giving the same error at the same point, at the verification Gradle commands.
Since the code in the first repository is depending on the code from the 2nd, the structure was designed that they have to be sibling directories to reach each other.
What went wrong:
Could not determine the dependencies of task ':app:testStaging_gakUnitTest'.
Could not resolve all task dependencies for configuration ':app:staging_gakUnitTestRuntimeClasspath'.
Could not resolve project :ticketingcommons.
Required by:
project :app
Unable to find a matching variant of project :ticketingcommons:
First one is a specific application code repository.
Second one contains the commons for the specific apps to run tests, validation etc...
In the configuration I only set the source-code management and building process fields so far.
I have been trying with pipelines, freestyle projects, multibranch pipelines and nothing seemed to be working.
In the Jenkinsfile, I have the following code, which is supposed to do the same I was doing from the Jenkins GUI:
pipeline {
agent {
// Run on a build agent where we have the Android SDK installed
label 'master'
}
options {
// Stop the build early in case of compile or test failures
skipStagesAfterUnstable()
}
stages {
stage('Compile') {
steps {
// Compile the app and its dependencies
sh 'chmod +x gradlew'
sh './gradlew compileDebugSources'
}
}
stage('Unit test') {
steps {
// Compile and run the unit tests for the app and its dependencies
sh './gradlew test'
// Analyse the test results and update the build result as appropriate
junit 'app/build/test-results/**/*.xml'
}
}
stage('Build APK') {
steps {
// Finish building and packaging the APK
sh './gradlew assembleDev'
// Archive the APKs so that they can be downloaded from Jenkins
archiveArtifacts '**/*.apk'
}
}
stage('Stage Archive') {
steps {
//tell Jenkins to archive the apks
archiveArtifacts artifacts: 'app/build/outputs/apk/*.apk', fingerprint: true
}
}
stage('Static analysis') {
steps {
// Run Lint and analyse the results
sh './gradlew lintDebug'
androidLint pattern: '**/lint-results-*.xml'
}
}
}
post {
failure {
// Notify developer team of the failure
mail to: 'mymail#whynotworking.com', subject: 'Oops!', body: "Build ${env.BUILD_NUMBER} failed; ${env.BUILD_URL}"
}
}
}
I don't know how to make Jenkins have them as sibling directories after cloning them, so the app can see the commons and run the commands. Now it is failing at the tests, but every validation Gradle command makes it fail.

A simple solution could be to create two jobs and customize their workspace directory (So you can use the same directory for both jobs). In your first job just load the repository from git and in the second load the repository and run whatever commands you want.

Using Gradle Spoon Plugin to launch tests in specific location

I am having problem to find documentation how could I solve that case.
I am capable of launching small/medium/large tests with:
./gradlew spoonSmall
./gradlew spoonMedium
./gradlew spoonLarge
Or launching specific tests with usage of this setup:
spoon {
(...)
if (project.hasProperty('spoonClassName')) {
className = project.spoonClassName
if (project.hasProperty('spoonMethodName')) {
methodName = project.spoonMethodName
}
}
}
I can launch specific file:
./gradlew spoon -PspoonClassName=com.package.tests.MyTest;
What I am interested in is a possibility to launch all tests located in:
./gradlew spoon -PspoonClassName=com.package.tests
package. Either method is fine. Some parameter to bash console or maybe way to create my own annotation and launch by something like ./gradlew spoonMyTests.
I am grateful for suggestions/help.

From the official docs:
There are numerous ways to run a specific test, or set of tests. You
can use the Spoon --size, --class-name or --method-name options, or
you can use the --e option to pass arguments to the instrumentation
runner, e.g.
--e package=com.mypackage.unit_tests
The following command should work when executing Spoon directly from command line (not from the gradle task)
java -jar spoon-runner-1.1.9-jar-with-dependencies.jar \
--apk ExampleApp-debug.apk \
--test-apk ExampleApp-debug-androidTest-unaligned.apk \
--e package=com.package.tests
You need to find a way to pass this extra param to the Spoon gradle plugin.
UPDATE
From the official doc of Spoon Gradle plugin:
Custom instrumentation arguments
Use the instrumentationArgs property on spoon extension to pass custom
parameters to your tests:
spoon { instrumentationArgs = ["foo=bar", "name=value"] }
In your case, this should look like the following:
spoon {
instrumentationArgs = ["package=com.package.tests"]
}

Gradle tests report for Android Tests

Can I have gradle tests report for android tests? I want to have the report directly in the terminal. Every time a test is passed, failed or skipped I want to have the report printed in the terminal with the name of the test and the status.
For Unit tests I can add this to the gradle file to have the reports:
tasks.withType(Test) {
testLogging {
exceptionFormat "full"
events "passed", "skipped", "failed"
}
}
I didn't find any solution to the android tests.

You can try this (add to app/library build.gradle)
tasks.withType(com.android.build.gradle.internal.tasks.AndroidTestTask) { task ->
task.doFirst {
logging.level = LogLevel.INFO
}
task.doLast {
logging.level = LogLevel.LIFECYCLE
}
}

There's a bug filed to the Platform tools team that is still not resolved:
https://code.google.com/p/android/issues/detail?id=182307
You can vote it to help. The workaround didn't work for me, but if you use Gradle Plugin >= 1.5.0 you can run the command with info logs enabled like this:
./gradlew cC --info
and then you'll see the tests being printed with no extra configuration.
Hope it helps!

Have you tried looking into app/build/reports/tests/debug/index.html or app/build/test-results/?
By using the new test support library you should find your reports there, with no extra-configuration needed.

I've made some investigation, and universal variant to show unitTests and AndroidTests in the terminal is put '--debug' option on the gradle terminal command.
Example:
./gradlew cleanTestDebugUnitTest testDebugUnitTest cleanConnectedDebugAndroidTest connectedDebugAndroidTest --debug

How can I ignore test failures with the gradle robolectric plugin?

I'm using the robolectric-gradle-plugin for robolectric unit tests. I don't want to fail a build on failed tests. Is there a way in DSL or a property not to fail a test on the build similar to -DtestFailureIgnore=true on the Surefire Maven plugin?
I've tried:
robolectric {
ignoreFailures = true
}
and
robolectric {
ignoreFailure = true
}
and -DignoreFailure=true on the command line.
I can't seem to find any documentation of how to do this, or any reference to ignoring tests in the source code.

answering very old question, so that it might help others who bump into here
testOptions {
unitTests.all {
setIgnoreFailures(true)
}
}

I would suggest not to continue building an APK if there are any failing tests. But if you want to build an APK without testing the only way right now is to use gradle build -x test[1]. This will run build and not run any tests.
[1]http://www.gradle.org/docs/current/userguide/userguide_single.html#sec:excluding_tasks_from_the_command_line

try without '='
robolectric {
ignoreFailures true
}

Running a specific instrumentation unit test with Gradle

Is there a way to run a specific Android instrumentation unit test using Gradle? I've tried
gradle -Dtest.single=UnitTestName connectedInstrumentTest
but it seems to run all the tests in the package.

Using test.single appears to be deprecated. The new correct way to do this is
./gradlew :<module>:test --tests <pattern>
where <pattern> could be something like:
com.example.MyTest to run all test methods in com.example.MyTest
*MyTest to match every method in every class whose name ends with MyTest
*.MyTest.myMethod to run a specific test method in class MyTest in any package
If you have a multi-project build, make sure to give the module path before the test task; otherwise you'll get a misleading error message when it searches for your test pattern in every subproject.
None of this is documented on the Gradle site anywhere I could find it.

This works if you're using an instrumentationTestRunner:
./gradlew test -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
Using gradle 2.10 and android gradle plugin 2.0.0-beta2.
Since you know what test(s) you want to run, you probably know which module / flavor to use too. You can help Gradle out by specifying the exact module and Gradle task. So if your test is in the app module and you want to test the debug flavor:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
You can get even more fancy with the tests_regex argument instead:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=PartialClassName*
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=partialMethodName*

The pattern is -D<testTaskName>.single=<TestClass> so in your example it should be:
gradle -DconnectedInstrumentTest.single=UnitTestName connectedInstrumentTest
NOTE: This answer is outdated. You should use the --tests switch in the latest versions of Gradle. (see other answers for an explanation)

Since Android gradle plugin 1.1.0-rc1, one can run single test class using --tests flag by executing:
./gradlew app:testDebug --tests=com.example.MyTest
See http://tools.android.com/tech-docs/unit-testing-support#TOC-Running-from-Gradle

You gotta check this out.
https://github.com/JCAndKSolutions/android-unit-test
I made an issue in this github repository, and this guy solved my problem and upload to maven, so in my build.gradle file I use this plugin.
Instructions are written in his repository. you can easily follow it.
After using this android-unit-test plugin, I can use like
../gradlew -Dtest.single=SomeTest test
or
../gradlew -Dtest.single=SomeTest clean check
Now it's working and I could only run the specific tests I want to

You should not forget to specify a build variant name after test property declaration like
-Dtest<buildVariantName>=<yourTestName>.
Like if you have a debug build type which gives you debug variant after compilation, then if you want to run a test only for this build variant you should declare a command like this:
./gradlew -DtestDebug=UnitTestName testDebug

Erdi's answer didn't work for me but I have a single parent for all my test classes so I was able to do this:
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if(getClass().getSimpleName().equals("MyTestClassName")) {
super.runTest();
}
}
//...
}
This executes only MyTestClassName. We can extend it further to execute only specific test method (or methods):
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if("MyTestClassName".equals(getClass().getSimpleName())
&& "testMethodName".equals(getName())) {
super.runTest();
}
}
//...
}

the Gradle command does not work for me.
I used below mentioened adb command.
for this you need to build your apk first.
adb shell am instrument -w -r -e package -e debug false .debug.test/android.support.test.runner.AndroidJUnitRunner