Develop Reference

Jenkins Android building pipeline from different Git repositories

Hi, I would like to create a job in Jenkins where I would like to build one apk from 2 Android repositories.
I tried it with both Jenkinsfile and Gradle wrapper in the Jenkins GUI, but both are giving the same error at the same point, at the verification Gradle commands.
Since the code in the first repository is depending on the code from the 2nd, the structure was designed that they have to be sibling directories to reach each other.
What went wrong:
Could not determine the dependencies of task ':app:testStaging_gakUnitTest'.
Could not resolve all task dependencies for configuration ':app:staging_gakUnitTestRuntimeClasspath'.
Could not resolve project :ticketingcommons.
Required by:
project :app
Unable to find a matching variant of project :ticketingcommons:
First one is a specific application code repository.
Second one contains the commons for the specific apps to run tests, validation etc...
In the configuration I only set the source-code management and building process fields so far.
I have been trying with pipelines, freestyle projects, multibranch pipelines and nothing seemed to be working.
In the Jenkinsfile, I have the following code, which is supposed to do the same I was doing from the Jenkins GUI:
pipeline {
agent {
// Run on a build agent where we have the Android SDK installed
label 'master'
}
options {
// Stop the build early in case of compile or test failures
skipStagesAfterUnstable()
}
stages {
stage('Compile') {
steps {
// Compile the app and its dependencies
sh 'chmod +x gradlew'
sh './gradlew compileDebugSources'
}
}
stage('Unit test') {
steps {
// Compile and run the unit tests for the app and its dependencies
sh './gradlew test'
// Analyse the test results and update the build result as appropriate
junit 'app/build/test-results/**/*.xml'
}
}
stage('Build APK') {
steps {
// Finish building and packaging the APK
sh './gradlew assembleDev'
// Archive the APKs so that they can be downloaded from Jenkins
archiveArtifacts '**/*.apk'
}
}
stage('Stage Archive') {
steps {
//tell Jenkins to archive the apks
archiveArtifacts artifacts: 'app/build/outputs/apk/*.apk', fingerprint: true
}
}
stage('Static analysis') {
steps {
// Run Lint and analyse the results
sh './gradlew lintDebug'
androidLint pattern: '**/lint-results-*.xml'
}
}
}
post {
failure {
// Notify developer team of the failure
mail to: 'mymail#whynotworking.com', subject: 'Oops!', body: "Build ${env.BUILD_NUMBER} failed; ${env.BUILD_URL}"
}
}
}
I don't know how to make Jenkins have them as sibling directories after cloning them, so the app can see the commons and run the commands. Now it is failing at the tests, but every validation Gradle command makes it fail.

A simple solution could be to create two jobs and customize their workspace directory (So you can use the same directory for both jobs). In your first job just load the repository from git and in the second load the repository and run whatever commands you want.

Android Robolectric Exception

I am facing a problem with Robolectric library. I am writing unit tests using Robolectric, it's working locally but when I merge my code, it crashes on pipeline(remotely).
I am using 'org.robolectric:robolectric:4.0.2'
It fails by only adding this line to my test class: #RunWith(RobolectricTestRunner.class)
And the exception is:
FAILED
org.apache.tools.ant.BuildException
Caused by: org.apache.maven.artifact.resolver.MultipleArtifactsNotFoundException

I had the same issue: Robolectric was working fine locally, but once pushed to Jenkins the gradle task to execute the tests fails.
You can execute the gradle task to run the tests with -i -d flags to see more debug output.
./gradlew -i -d test
For me this revealed that Jenkins was unable to download Robolectric dependencies:
13:58:43 13:58:42.904 [DEBUG] [TestEventLogger] com.my.package.Test > my_test_case STANDARD_ERROR
13:58:43 13:58:42.904 [DEBUG] [TestEventLogger] Downloading: org/robolectric/android-all/9-robolectric-4913185-2/android-all-9-robolectric-4913185-2.jar from repository sonatype at https://oss.sonatype.org/content/groups/public/
I could solve it by telling gradle to use our corporate proxy when running on Jenkins. One way to achieve this would be to add the following to your gradle.properties:
systemProp.http.proxyHost=http://proxy.host
systemProp.http.proxyPort=3128
systemProp.https.proxyHost=http://proxy.host
systemProp.https.proxyPort=3128
---- EDIT ----
Actually I found a cleaner solution for my use case then configuring a proxy: Robolectric offers a way to configure the repository it uses during runtime (see http://robolectric.org/configuring/). This way I was able to tell it to use our corporate repository.
android {
testOptions {
unitTests.all {
systemProperty 'robolectric.dependency.repo.url', 'https://local-mirror/repo'
systemProperty 'robolectric.dependency.repo.id', 'local'
}
}
}

In my case the issue was (./gradlew -i -d testDebug printed it):
Caused by:
org.apache.maven.artifact.resolver.MultipleArtifactsNotFoundException: Missing:
----------
1) org.robolectric:android-all:jar:10-robolectric-5803371
...
Robolectric tried to fetch a dependency from the remote repository, but it couldn't.
As the output suggests, I have:
Manually downloaded org.robolectric:android-all:jar:10-robolectric-5803371 jar from the Maven repository.
Located it in /home/user/jars
Installed Maven on my machine: sudo apt install maven
Linked the jar with Robolectic: mvn install:install-file -DgroupId=org.robolectric -DartifactId=android-all -Dversion=10-robolectric-5803371 -Dpackaging=jar -Dfile=/home/user/jars/android-all-10-robolectric-5803371.jar
Now ./gradlew testDebug works fine.
Also check the issue on Github, maybe there will be some more useful information for you.

As #Christian.D said, Robolectric tries to download dependencies from external repository, but we need that it uses internal one. Below the fix:
1) Create a custom RobolectricTestRunner:
public class CustomRobolectricRunner extends RobolectricTestRunner {
public CustomRobolectricRunner(Class<?> testClass) throws InitializationError {
super(testClass);
}
static {
RoboSettings.setMavenRepositoryId("my-nexus");
RoboSettings.setMavenRepositoryUrl("your_custom_url");
}
}
2) Annotate your test class:
#Config(manifest = Config.NONE)
#RunWith(CustomRobolectricRunner.class)
Good coding!

Splitting unit and integration tests for android

Currently I have a test/src/java folder where all of the tests for the android application are stored (tests are done using junit, mockito and robolectric).
And I can run those using ./gradlew test
What I'd like to achieve is having two folders:
integrationTest/src/java - for integration tests
test/src/java - for unit tests
And also I'd like to run them separately, like ./gradlew test and ./gradlew integrationTest.
I've managed to split directories with tests using sourceSets like this:
sourceSets {
test {
java {
srcDirs = ['src/test/java', 'src/integrationTest/java', 'src/commonTest/java']
}
resources {
srcDirs = ['src/test/resources', 'src/integrationTest/resources', 'src/commonTest/resources']
}
}
}
And I had googled many examples on how to create custom test tasks, but most of them are related to java instead of android and the others are out-of-date. I've spent on that the whole day now and so if someone can help me I would really appreciate that.

If your integration tests are instrumented tests, then you can just use the default folders test and androidTest and run them separately using ./gradlew test and ./gradlew connectedAndroidTest
Another way (if you can have the integration tests inside the test folder) would be to use separate packages inside the test folder and run the tests separately using:
./gradlew testDebug --tests="com.yourapplication.unittests.*"
./gradlew testDebug --tests="com.yourapplication.integrationtests.*"
...

I had the same problem a few days ago.
In order to solve it and be able to run each type of test independently, I separated my tests like this:
// run only unit tests
test{
include '**/unit/**'
exclude '**/integration/**'
doLast {
println 'Unit tests execution finished.'
}
}
// run only integration tests
task integrationTest(type: Test){
include '**/integration/**'
exclude '**/unit/**'
doLast {
println 'Integration tests execution finished.'
}
}
// run all tests (unit + integration)
task allTests(type: Test){
include '**/integration/**'
include '**/unit/**'
doLast {
println 'All tests execution finished.'
}
}
The include keyword indicates which files you want to include when executing the commands. If you want to run only your unit tests, you can only include the folder(s) that include your unit tests and exclude the folders that include your integration tests.
You can use a similar logic when creating a gradle command to run only your integration tests.
To execute your tests using this configuration and gradle, use:
./gradlew test to execute the unit tests only.
./gradlew integrationTests to execute the integration tests only.
./gradlew allTeststo execute both the integration and the unit tests.
NOTE: You can setup the paths in the includes / excludes in order to include / exclude tests or classes when executing your tests. It is also possible to include only one test and exclude the others and vice-versa.

Possibly something like
sourceSets {
integrationTest {
java {
compileClasspath += main.output
runtimeClasspath += main.output
srcDir 'src/integrationTest/java'
}
resources.srcDir 'src/integrationTest/resources'
}
}
configurations {
integrationTestCompile {
extendsFrom compile
}
integrationTestRuntime {
extendsFrom runtime
}
}
task integrationTest(type: Test) {
testClassesDir = sourceSets.integrationTest.output.classesDir
classpath = sourceSets.integrationTest.runtimeClasspath
}
check.dependsOn integrationTest
Then you could do
dependencies {
integrationTestCompile 'org.seleniumhq.selenium:selenium-java:3.0.1'
integrationTestRuntime 'com.foo:bar:1.0'
}

Jacoco Coverage Report issues

I am trying to define the location, where jacoco will create the coverage file for instrumentation tests running on real devices.
From the --debug run of the gradle task I see this log:
[DEBUG] [org.gradle.api.Task] DeviceConnector 'Nexus 5X - 6.0.1': installing /home/martin/workspace/lib/my-lib/build/outputs/apk/my-lib-debug-androidTest-unaligned.apk
[INFO] [org.gradle.api.Task] Starting 1 tests on Nexus 5X - 6.0.1
[INFO] [org.gradle.api.Task] de.my.lib.utils.UtilsTest testMyTest[Nexus 5X - 6.0.1] [32mSUCCESS [0m
[DEBUG] [org.gradle.api.Task] DeviceConnector 'Nexus 5X - 6.0.1': fetching coverage data from /data/data/de.my.lib.test/coverage.ec
[DEBUG] [org.gradle.api.Task] DeviceConnector 'Nexus 5X - 6.0.1': uninstalling de.my.lib.test 13:46:14.538
[DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':my-lib:connectedDebugAndroidTest'
I tried 3 ways to define the location:
Using the <instrumentation> tag in the manifest file didn't change anything.
<?xml version="1.0" encoding="utf-8"?>
<manifest
package="de.my.lib.test"
xmlns:android="http://schemas.android.com/apk/res/android">
<instrumentation
android:name="android.support.test.runner.AndroidJUnitRunner"
xmlns:tools="http://schemas.android.com/tools"
android:targetPackage="de.my.lib.test"
tools:replace="android:targetPackage">
<meta-data
android:name="coverage"
android:value="true" />
<meta-data
android:name="coverageFile"
android:value="/sdcard/coverage.ec" />
</instrumentation>
</manifest>
I tried it with gradle but the output was the same:
defaultConfig {
// unimportant stuff
testApplicationId "de.my.lib.test"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArgument('coverageFile', '/sdcard/coverage.ec')
}
And finally I tried it with adb command:
adb shell am instrument -w -e coverage true -e coverageFile /sdcard/coverage.ec de.my.lib.test/android.support.test.runner.AndroidJUnitRunner
But there I get 2 errors:
de.my.lib.utils.UtilsTest:.
Could not find class: org.jacoco.agent.rt.internal_773e439.CoverageTransformer
.
Time: 0,072
OK (1 test)
Error: Failed to generate emma coverage.
I am completely lost here. Any ideas?
Background Why I need it to have it stored in another place: There is a bug with adb shell run-as command on some devices and Android version so I have devices in my test device farm which return 0% coverage because the file can't be pulled. So I need the file to be stored in a publicly available location.

However, the coverage report is not generated yet. To enable this option, we need to add a property to our debug build variant. Using the Android plugin DSL, we can enable the coverage through the testCoverageEnabled property:
android {
...
buildTypes {
debug {
testCoverageEnabled true
}
...
}
}
To enable the coverage report for local tests when using version 2.2.+ of Android Gradle plugin, you need to enable it in your app’s build.gradle:
android {
//...
testOptions {
unitTests.all {
jacoco {
includeNoLocationClasses = true
}
}
}
}
The instrumentation performed by Jacoco produces execution files that contain the necessary data to create the report (HTML, XML, etc.). The problem here, is that Espresso generates .ec file, while the unit tests execution generates .exec file… we have different formats!
As we are not able to configure the coverage task of Espresso, we must ensure that it is executed first. Next, we need to run unit tests, and then create the coverage data with both files (ec and exec).
To enable this, we need to edit our task once more and add the coverage.ec file as a parameter in executionData property:
apply plugin: 'jacoco'
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {
reports {
xml.enabled = true
html.enabled = true
}
def fileFilter = ['**/R.class', '**/R$*.class', '**/BuildConfig.*', '**/Manifest*.*', '**/*Test*.*', 'android/**/*.*']
def debugTree = fileTree(dir: "${buildDir}/intermediates/classes/debug", excludes: fileFilter)
def mainSrc = "${project.projectDir}/src/main/java"
sourceDirectories = files([mainSrc])
classDirectories = files([debugTree])
executionData = fileTree(dir: "$buildDir", includes: [
"jacoco/testDebugUnitTest.exec",
"outputs/code-coverage/connected/*coverage.ec"
])
}
As Android Gradle plugin 2.2.+ now generates a coverage file for each execution, using the device / emulator in the file name, now we need to pass every file to execution data, as the file name is now dynamic. In addition, I added the createDebugCoverageReport task as a dependency of our custom task, so we don’t need to run it manually :)
Finally, when we execute both tasks, running the Espresso tests first, we will get the unified coverage report!
gradle clean jacocoTestReport
For detailed explanation see here.
Edit:
So, the command adb shell am instrument does not generates the report, for this you have to handle it using gradle solution. If you really want to test it your self, then you've to use a custom runner.
References:
1) How to Generate Android Testing Report in HTML Automatically
2) How to retrieve test results when using "adb shell am instrument"
3) https://github.com/jsankey/android-junit-report
Test from the command line
Updated to Command line method follow these steps.
Running the instrumented application
Once we have the EmmaInstrumentation class (is in ref link) in place we need a few more adjustments to be able to get the coverage report of the running application.
Firstly, we need to add the new Activity to the manifest. Secondly, we should allow our application to write to the sdcard if this is where we decided to generate the coverage report. To do it you should grant the android.permission.WRITE_EXTERNAL_STORAGE permission.
Then, it's time to build and install the instrumented apk:
$ ant clean
$ ant instrument
$ ant installi
Everything is ready to start the instrumented application
$ adb shell am instrument -e coverage true \
-w com.example.i2at.tc/\
com.example.instrumentation.EmmaInstrumentation
Here is Source code.

the missing CoverageTransformer class hints for a missing dependency to the JaCoCo agent runtime. one simple solution approach might be, to provide it with what it demands (to be available on device):
testImplementation "org.jacoco:org.jacoco.agent:0.8.3"
in case you get a FileNotFoundException, when the ClassNotFoundException was fixed, make sure to have the permission to write to SD card, which appears to be another possible pitfall:
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
as I've just seen, there is also an org.jacoco.report package, which could be optionally added:
testImplementation "org.jacoco:org.jacoco.report:0.8.3"
package org.jacoco.agent is in every case the one which contains class CoverageTransformer.

For us it was an AOSP app. We were generating the test and module apks with disabling Jack compiler which was causing this further issue. Because for us Jack compiler was needed to generate coverage.em. But the error was not quite relevant. So had no clue to go any further.
So the fix was to run the jack compiler server and get the apks generated using it.

Test running startedTest running failed: Unable to find instrumentation info for: ComponentInfo{ro.vst.test/android.test.InstrumentationTestRunner}

here is my full code, when i run following code,
public class KernelTest {
#Test
public void testM() {
assertEquals(1, 1);
}
}
It raise:
Test running failed: Unable to find instrumentation info for: ComponentInfo{ro.vst.test/android.test.InstrumentationTestRunner}

I've found the solution: in android studio 1.5, whether I specify runner or not, avd side will generate a package name with "my_pkg_name.test/runner_name", the solution steps are:
run android test once
I get "Test running failed: Unable to find instrumentation info for: ComponentInfo{ro.vst.test/android.test.InstrumentationTestRunner}"
run adb shell pm list instrumentation, find pkg relative instrument
I get "instrumentation:ro.vst.test/com.google.android.apps.common.testing.testrunner.GoogleInstrumentationTestRunner"
the two TestRunner are different, so I specify TestRunner in build.gradle like following
build.gradle
defaultConfig {
testInstrumentationRunner 'android.support.test.runner.AndroidJUnitRunner'
}
now run again, it should work