How to get code coverage report for android instrumentation tests - android

How do I find the code coverage for instrumentation test case, the run tests with code coverage button is disabled.
Please help.

Before that you need to enable Coverage in gradle file at app level .
buildTypes {
debug
{
testCoverageEnabled = true
}
}
Once done you can run below command in Terminal
gradlew createDebugCoverageReport
You can check it at below path
YoutProjectName\app\build\reports\coverage\debug
and check index.html file in your browser.

Related

Access project related data from Gradle command

I need project related data like project name,app version and its main module from gradle based android project. I have tried various tasks like project,properties but none of it giving me specific information i need.
Is there a way to find version code,app name and main android module using gradle in command line?
Using "BuildConfig" global variable you will get
boolean DEBUG
String APPLICATION_ID
String BUILD_TYPE
String FLAVOR
int VERSION_CODE
String VERSION_NAME
eg :- BuildConfig.APPLICATION_ID
and if you defined any global data in gradle like
debug {
buildConfigField "String", "BASE_URL", '"http://172.16.1.175:8080/api/"'
debuggable true
}
you will get this details also
BuildConfig.BASE_URL
You can probably write your own custom gradle task for doing that. Add this code snippet in your app build.gradle, where you define your android plugin and run it from console. You can format output like you need it and use other data from build script.
task hello<<{
println("versionCode = ${android.defaultConfig.versionCode}")
println("applicationId = ${android.defaultConfig.applicationId}")
println("minSDK = ${android.defaultConfig.minSdkVersion}")
}
you can use resValue for that to get value
Gradle
defaultConfig {
//other config
resValue "String","versionCode","1"
}
your class
context.getString(R.string.versionCode);
I don't know if it suits, you can create one common init gradle file, which you run from command line, so it is not a source code manipulation, where you print out all necessary data. But gradle output is dirty.
This is snippet of init.gradle which is in /Users/username
allprojects{
afterEvaluate({//listen for project evaluation
println(project.name)//it is supposed to be 2 projects "ProjName" and "app"
if(project.name.equalsIgnoreCase("app")){//or any other condtion to check if it is inner android project
project.task("getVersion",{
println("versionCode = ${android.defaultConfig.versionCode}")
})
}
});
}
you start this script like ./gradlew --I /Users/username/init.gradle
This is what I have as an output
music
app
versionCode = 1
:help
Welcome to Gradle 2.4.
To run a build, run gradlew <task> ...
To see a list of available tasks, run gradlew tasks
To see a list of command-line options, run gradlew --help
To see more detail about a task, run gradlew help --task <task>
BUILD SUCCESSFUL
Total time: 6.929 secs
This build could be faster, please consider using the Gradle Daemon: http://gradle.org/docs/2.4/userguide/gradle_daemon.html
So this is what could be done, another available option is to parse build.gradle file or manifest.xml in bash, or write own console utility that will do it with a cleaner output.
I hope I helped.

Jacoco Coverage Report issues

I am trying to define the location, where jacoco will create the coverage file for instrumentation tests running on real devices.
From the --debug run of the gradle task I see this log:
[DEBUG] [org.gradle.api.Task] DeviceConnector 'Nexus 5X - 6.0.1': installing /home/martin/workspace/lib/my-lib/build/outputs/apk/my-lib-debug-androidTest-unaligned.apk
[INFO] [org.gradle.api.Task] Starting 1 tests on Nexus 5X - 6.0.1
[INFO] [org.gradle.api.Task] de.my.lib.utils.UtilsTest testMyTest[Nexus 5X - 6.0.1] [32mSUCCESS [0m
[DEBUG] [org.gradle.api.Task] DeviceConnector 'Nexus 5X - 6.0.1': fetching coverage data from /data/data/de.my.lib.test/coverage.ec
[DEBUG] [org.gradle.api.Task] DeviceConnector 'Nexus 5X - 6.0.1': uninstalling de.my.lib.test 13:46:14.538
[DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':my-lib:connectedDebugAndroidTest'
I tried 3 ways to define the location:
Using the <instrumentation> tag in the manifest file didn't change anything.
<?xml version="1.0" encoding="utf-8"?>
<manifest
package="de.my.lib.test"
xmlns:android="http://schemas.android.com/apk/res/android">
<instrumentation
android:name="android.support.test.runner.AndroidJUnitRunner"
xmlns:tools="http://schemas.android.com/tools"
android:targetPackage="de.my.lib.test"
tools:replace="android:targetPackage">
<meta-data
android:name="coverage"
android:value="true" />
<meta-data
android:name="coverageFile"
android:value="/sdcard/coverage.ec" />
</instrumentation>
</manifest>
I tried it with gradle but the output was the same:
defaultConfig {
// unimportant stuff
testApplicationId "de.my.lib.test"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
testInstrumentationRunnerArgument('coverageFile', '/sdcard/coverage.ec')
}
And finally I tried it with adb command:
adb shell am instrument -w -e coverage true -e coverageFile /sdcard/coverage.ec de.my.lib.test/android.support.test.runner.AndroidJUnitRunner
But there I get 2 errors:
de.my.lib.utils.UtilsTest:.
Could not find class: org.jacoco.agent.rt.internal_773e439.CoverageTransformer
.
Time: 0,072
OK (1 test)
Error: Failed to generate emma coverage.
I am completely lost here. Any ideas?
Background Why I need it to have it stored in another place: There is a bug with adb shell run-as command on some devices and Android version so I have devices in my test device farm which return 0% coverage because the file can't be pulled. So I need the file to be stored in a publicly available location.
However, the coverage report is not generated yet. To enable this option, we need to add a property to our debug build variant. Using the Android plugin DSL, we can enable the coverage through the testCoverageEnabled property:
android {
...
buildTypes {
debug {
testCoverageEnabled true
}
...
}
}
To enable the coverage report for local tests when using version 2.2.+ of Android Gradle plugin, you need to enable it in your app’s build.gradle:
android {
//...
testOptions {
unitTests.all {
jacoco {
includeNoLocationClasses = true
}
}
}
}
The instrumentation performed by Jacoco produces execution files that contain the necessary data to create the report (HTML, XML, etc.). The problem here, is that Espresso generates .ec file, while the unit tests execution generates .exec file… we have different formats!
As we are not able to configure the coverage task of Espresso, we must ensure that it is executed first. Next, we need to run unit tests, and then create the coverage data with both files (ec and exec).
To enable this, we need to edit our task once more and add the coverage.ec file as a parameter in executionData property:
apply plugin: 'jacoco'
task jacocoTestReport(type: JacocoReport, dependsOn: ['testDebugUnitTest', 'createDebugCoverageReport']) {
reports {
xml.enabled = true
html.enabled = true
}
def fileFilter = ['**/R.class', '**/R$*.class', '**/BuildConfig.*', '**/Manifest*.*', '**/*Test*.*', 'android/**/*.*']
def debugTree = fileTree(dir: "${buildDir}/intermediates/classes/debug", excludes: fileFilter)
def mainSrc = "${project.projectDir}/src/main/java"
sourceDirectories = files([mainSrc])
classDirectories = files([debugTree])
executionData = fileTree(dir: "$buildDir", includes: [
"jacoco/testDebugUnitTest.exec",
"outputs/code-coverage/connected/*coverage.ec"
])
}
As Android Gradle plugin 2.2.+ now generates a coverage file for each execution, using the device / emulator in the file name, now we need to pass every file to execution data, as the file name is now dynamic. In addition, I added the createDebugCoverageReport task as a dependency of our custom task, so we don’t need to run it manually :)
Finally, when we execute both tasks, running the Espresso tests first, we will get the unified coverage report!
gradle clean jacocoTestReport
For detailed explanation see here.
Edit:
So, the command adb shell am instrument does not generates the report, for this you have to handle it using gradle solution. If you really want to test it your self, then you've to use a custom runner.
References:
1) How to Generate Android Testing Report in HTML Automatically
2) How to retrieve test results when using "adb shell am instrument"
3) https://github.com/jsankey/android-junit-report
Test from the command line
Updated to Command line method follow these steps.
Running the instrumented application
Once we have the EmmaInstrumentation class (is in ref link) in place we need a few more adjustments to be able to get the coverage report of the running application.
Firstly, we need to add the new Activity to the manifest. Secondly, we should allow our application to write to the sdcard if this is where we decided to generate the coverage report. To do it you should grant the android.permission.WRITE_EXTERNAL_STORAGE permission.
Then, it's time to build and install the instrumented apk:
$ ant clean
$ ant instrument
$ ant installi
Everything is ready to start the instrumented application
$ adb shell am instrument -e coverage true \
-w com.example.i2at.tc/\
com.example.instrumentation.EmmaInstrumentation
Here is Source code.
the missing CoverageTransformer class hints for a missing dependency to the JaCoCo agent runtime. one simple solution approach might be, to provide it with what it demands (to be available on device):
testImplementation "org.jacoco:org.jacoco.agent:0.8.3"
in case you get a FileNotFoundException, when the ClassNotFoundException was fixed, make sure to have the permission to write to SD card, which appears to be another possible pitfall:
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
as I've just seen, there is also an org.jacoco.report package, which could be optionally added:
testImplementation "org.jacoco:org.jacoco.report:0.8.3"
package org.jacoco.agent is in every case the one which contains class CoverageTransformer.
For us it was an AOSP app. We were generating the test and module apks with disabling Jack compiler which was causing this further issue. Because for us Jack compiler was needed to generate coverage.em. But the error was not quite relevant. So had no clue to go any further.
So the fix was to run the jack compiler server and get the apks generated using it.

Gradle tests report for Android Tests

Can I have gradle tests report for android tests? I want to have the report directly in the terminal. Every time a test is passed, failed or skipped I want to have the report printed in the terminal with the name of the test and the status.
For Unit tests I can add this to the gradle file to have the reports:
tasks.withType(Test) {
testLogging {
exceptionFormat "full"
events "passed", "skipped", "failed"
}
}
I didn't find any solution to the android tests.
You can try this (add to app/library build.gradle)
tasks.withType(com.android.build.gradle.internal.tasks.AndroidTestTask) { task ->
task.doFirst {
logging.level = LogLevel.INFO
}
task.doLast {
logging.level = LogLevel.LIFECYCLE
}
}
There's a bug filed to the Platform tools team that is still not resolved:
https://code.google.com/p/android/issues/detail?id=182307
You can vote it to help. The workaround didn't work for me, but if you use Gradle Plugin >= 1.5.0 you can run the command with info logs enabled like this:
./gradlew cC --info
and then you'll see the tests being printed with no extra configuration.
Hope it helps!
Have you tried looking into app/build/reports/tests/debug/index.html or app/build/test-results/?
By using the new test support library you should find your reports there, with no extra-configuration needed.
I've made some investigation, and universal variant to show unitTests and AndroidTests in the terminal is put '--debug' option on the gradle terminal command.
Example:
./gradlew cleanTestDebugUnitTest testDebugUnitTest cleanConnectedDebugAndroidTest connectedDebugAndroidTest --debug

continue jacoco code coverage report after fail test case

Code coverage report not generated when Test case failed in android studio using Jacoco plugin.How to skip failed test case and generate code coverage report.
Try adding this flag when running your test
-Djacoco.haltonfailure=false
Use the following code in Build.gradle(Module:app):
android {
testOptions {
unitTests.all {
setIgnoreFailures(true)
}
}
buildTypes {
debug {
testCoverageEnabled true
}
}
}
If you just need to get the report you can add the #Ignore above the tests in order to temporarily skip them.
When you take care of the reason these tests fail then you just remove the annotation.
This annotation will not run any of the marked tests, you can use it either at class file or at the method/test.

Testing an android app generating specific xml junit file

I'm using Android Studio and the gradle plugin to build apps from the command line.
Now, suppose I want to run tests on my connected devices, I run:
./gradlew app:connectedCheck
and it will output a junit report XML file on:
./app/build/outputs/androidTest-results/connected/flavors/latest/TEST-Galaxy Nexus - 4.3-app-Latest.xml
This is almost what I want. Just that I would like to specify for example this test output file. For example, I would like to output it to $HOME/junit.xml. And I would like to specify this $HOME/junit.xml from the command line.
How do I do this?
You can specify the output directory for test reports using
android {
testOptions {
reportDir = "$project.buildDir/foo/report"
}
}
http://tools.android.com/tech-docs/new-build-system/user-guide#TOC-Test-reports

Categories

Resources