I am using make for building and Robolectric as a framework for running Android tests. I would like to calculate coverage of my app. For instrumentation tests I used to use emmalib. What is the best way for me to set up coverage calculation in this case? I can't migrate to gradle or maven.
Are you bound to Emma? How about using RoboElectric + Cobertura code coverage? (I think you could just use a CLI for the above combination)
So options
1.) RoboElectric + Cobertura - CLI alone probably for someone not on ANT
2.) JaCoCo might have some useful options
3.) Pure Android Testing + Emma/EclEmma
Useful Link trails to follow
Generating android code coverage though changes in build.xml and ant.properties
Android Gradle Code Coverage
https://intellectualcramps.wordpress.com/2013/08/18/code-coverage-of-robolectric-tests-using-jacoco/
https://bitbucket.org/ravidsrk/androidstarter
EDIT:
Well most of the tutorials I have come across use ant unfortunately, and I don't think it would be a bad idea for you to move to a recommended build system like Gradle so it opens up a lot options. But for JaCoCo you could take a look at here: https://intellectualcramps.wordpress.com/2013/08/18/code-coverage-of-robolectric-tests-using-jacoco/
UPDATE:
Moved this from comments to the answer section for information to anyone wanting to see this and because this is a bounty question
One solution would be to use Cobertura to generate code coverage, which can be integrated in eclipse and also run by an ant build script.
A template project of such an integration can be found here: https://github.com/adgllorente/android-cobertura-boilerplate
Note that all of the magic happens in the build.xml of the Test project. Theses tasks should probably be generalised to a custom_rules.xml file so you can still use android to update your projects.
Finally, for Gradle you have many different options:
http://raptordigital.blogspot.nl/2014/08/code-coverage-reports-using-robolectric.html
http://chrisjenx.com/gradle-robolectric-jacoco-dagger/
https://stackoverflow.com/a/25037742/2771851
Note that you can always use Gradle as a secondary build system just to generate the coverage reports. (but a second build system will introduce a lot of overhead)
jacoco sometimes does not work with Robelectric and powermock runner you can use clover an atlassian tools it is now open source tool.
Related
gradle-ndk-gtest-sample
Android NDK with
README.NDK
I have been trying to configure my Android NDK project for use with Google Test. I started with native-activity sample as the base project and began following the guides above. I inserted the code from the first link into my root build.gradle. I am really not having any success. It looks like i was able to generate the test library:
./libs/googleTest/1.7.0/lib/osx/libgtest.a
./libs/googleTest/1.7.0/lib/linux/libgtest.a
./app/.externalNativeBuild/cmake/debug/x86_64/libnative_app_glue.a
./app/.externalNativeBuild/cmake/debug/x86_64/libFOO.a
./app/.externalNativeBuild/cmake/debug/x86_64/libgtest.a
./app/.externalNativeBuild/cmake/debug/arm64-v8a/libnative_app_glue.a
./app/.externalNativeBuild/cmake/debug/arm64-v8a/libFOO.a
./app/.externalNativeBuild/cmake/debug/arm64-v8a/libgtest.a
But I don't know where to go from here. How do I actually run the tests and see if they passed?
Try something like https://github.com/DanAlbert/GTestJNI. It's an alternative test runner for JUnit and GTest that exposes GTests as if they were JUnit, so you can just run your normal androidTests and they'll run your GTests for you.
Some day I want to get this either added to the NDK or at least made trivial to add to a project as a git submodule or something, but for now you can merge the pieces of that into your project.
I have configured a Jenkins item that, after gradle building and testing, calls the "Execute SonarQube Scanner" with the following specifications:
sonar.projectKey=my_project_key
sonar.projectName=my_Android
sonar.projectVersion=1.0
sonar.sources=app/src
sonar.java.binaries=app/build/intermediates/classes/debug
sonar.coverage.exclusions=app/src/test/**,app/src/androidTest/**,app/src/main/res/**
This successfully runs SonarQube and generates a report to IPaddress:9000 that measures: Bugs, Vulnerabilities, Debt, Code Smells, and Duplications just like it is supposed to. However there is no place in this report that defines the code covered by tests. SonarQube is running the Kotlin plugin successfully and sees the codebase itself, but is not determining coverage. Is this possible to gather this information with SonarQube when using Kotlin Android?
I did try installing Jacoco, but did not see any new data on the SonarQube results.
You can install a 3rd-party Sonar plugin here that will measure code quality for Kotlin projects. Unfortunately, as of this posting, SonarQube does not have its own plugin for this.
Once that is set up, take a look here for setting up gradle and sonar-project.properties. The post is a little dated, but contains a link to an updated version and is still a good starting point.
TL;DR for #2: The information SonarQube needs for Kotlin is located in 2 places. One is already identified by the OP. The other is located in app/build/tmp/kotlin-classes.
If you are using gradle and Jacoco.
You can use Jacoco for the code coverage with the xmlReportPaths. With that Sonar will trust what Jacoco output.
sonarqube {
properties {
property("sonar.java.coveragePlugin", "jacoco")
property("sonar.coverage.jacoco.xmlReportPaths", "./build/reports/jacoco/test/jacocoTestReport.xml")
}
}
Documentation from Sonar + Jacoco Plugin.
It also works with mixed source code (Java, Kotlin).
I abandoned attempting to get Kotlin code coverage with Sonar shortly after posting the question.
However, I have gotten many projects to work with Java and JaCoCo.
Retrospectively looking back at my sonar.properties file I may have been able to fix this by referencing the output of the JaCoCo task by adding the following line to the sonar.properties file:
sonar.jacoco.reportPath=build/jacoco/test.exec
I am not sure that this will work with Kotlin, but it helped with Java coverage and solved my use case.
Newer versions of Sonarqube (6+) now support Kotlin natively, and do not require any custom configuration in your project.
Just ensure the Kotlin and the Jacoco plugins are installed, and it should just work for you.
--- OLD ANSWER---------------------------------
My project is using Jacoco for generating coverage reports.
To have SonarQube pick up the results, try setting the following:
sonar.java.binaries={$buildDir}/classes/kotlin
In other words let SonarQube know where the Kotlin class files are.
This configuration assumes a 100% Kotlin project. If your project is a mix of Java and Kotlin, then you'll have to determine how to configure both directories.
I have an Android library (jar file) containing some APIs and I want to do code coverage test for these APIs using Jacoco.
For example, I have a calculator library (may name it calc.jar), having some APIs to do add, subtract, multiple and divide (via some static method such as Calc.doAdd(a,b), Calc.doSubtract(a,b) etc...). Now I have another Android test application (created by Android Studio) using this calculator library. I would like to do code coverage for this calculator library (surely I have calculator library source code).
I am also successfully do code coverage test with EMMA and ant.
As Android Studio now uses Gradle for building, so I would like to know if we can do code coverage (I searched and see that Gradle can work with Jacoco).
I do searched and see that there are some topic relating to Jacoco code coverage for Android Application. But I am looking a way for doing code coverage for JAR library using Jacoco and Gradle. Please give me a help.
If there are topics relating to this, please help to show me.
Many thanks in advance,
According to Issue 76373 you can't get code coverage for libraries. Some people seems to have been successful with some hacks but I've not found anything that works for me.
I have an Android Maven project (let's call it parent_project) that contains various submodules: my_library_project, app_using_my_library_project, test_project and extra_lib.
So, the structure would be like this:
parent_project
* my_library_project (Android Library Project)
* app_using_my_library_project (Demo app that uses the Android Library Project)
* test_project (Project containing the tests instrumented against app_using_my_library_project)
* extra_lib
What I would like is to generate test coverage for my Android project using Maven (and not Ant, I am already able to generate code coverage reports using Ant, following these instructions: https://wiki.jenkins-ci.org/display/JENKINS/Building+an+Android+app+and+test+project).
I have no strong preference for the code coverage tool used but I would prefer EMMA, since seems the most common in the Android development world.
I am using android-maven-plugin (http://code.google.com/p/maven-android-plugin/) in its 3.0.0-alpha-12 version and I have already tried to put in the configuration of my parent's pom.xml the next:
<test>
<coverage>true</coverage>
<createreport>true</createreport>
</test>
But that does not produce the desired code coverage report.
So:
Is there any difference between the pom configuration for getting code coverage for a standard Java project and an Android project?
Do you know any example Android project using Maven that has code coverage?
Any hints on how to do this?
If you're going to stick with maven, and want a plugin for maven that will do the code-coverage job, I think Cobertura is a better choice, as Emma stable last build is from 2005.
Although in "Android Application Testing Guide" (a recent book from June this year) they talk about Emma and demonstrate how to use it for testing, I think people stick to it, because it's needed to build Android from source (and if Google use it for their own OS development, it should be the best, right?).
If you're not fanatically bound to Maven, I strongly recommend to try Robotium.
Robotium has full support for Activities, Dialogs, Toasts, Menus, and Context Menus.
It also supports code coverage (Ant based though, for now) and some people recognize it as one of the leading testing platforms for Android.
Edit:
According to the Cobertura site, it supports code coverage in Maven 1 and Maven 2 environments. Although, you can find examples with Maven 3 also.
A problem exists between pom configurations of Maven 2 and Maven 3. It seems for the reporting to work you have to basically move your old reporting plugins into the configuration section of the new maven-site-plugin. (See the article for details).
Another option is to try and use Sonar with Maven. Sonar has cobertura embedded (also options to embedd EMMA) and some people state that they had successfully reported code coverage, despite they had problems using the "stand-alone" cobertura plugin.
I could generate code coverage reported using emma maven plugin and display reports in sonar for an android application project. Just follow the configuration in https://code.google.com/p/maven-android-plugin/wiki/EmmaMaven.
But for library project I get a 0% coverage. It doesn't generate the metadata files. However as soon I change the library project to package as an apk, it works like a charm. It runs the tests in a emulator(configured in jenkins) and shows the coverage reports.
If you are looking out specifically for library project, this wont help. I will post if I find some solution which deals with library projects.
I'm a long time Java developer with many years of Java EE, Ant, Maven, unit testing, mocks, etc. I'm currently using gradle to build android apps and am looking at unit testing them. And it's got me tearing my hair out!
My reading indicates that to test an app, I have to create another app inside it in a test directory. But I'm not sure how this embedded app can see the main apps classes. I presume that google came up with this because of something to do with the manifests which are different. I'm not sure why.
Without doing this embedded app system, I've been able to get unit tests to run by including the test classes with the main classes in the APK, adding the instrumentation declarations to the manifest, deploying it and running the test runners. But I don't want to be compiling test classes with app classes and including all the dependencies so that's not really an option and I'm not really sure of the effects of the changes to the manifest as I cannot find any documentation about the effects.
None of this is understood by gradle which follows the maven system of building. Also note that the android way seems to be that the embedded sub-project (test) is dependant on the main parent project, something that is totally contray to gradle and maven.
Another option seems to be separate the test project so that it's completely outside the app project. But really, I'd like to use a similar convention to maven and simply have tests in a separate directory, along with the manifest in test resources.
Has anyone managed to get testing on the emulators running unit tests following a more maven like directory structure?
You can try Robotium. It provides lots of features for a better testcase. You can have a look at it here.
Do you have to run the unit tests in the emulator? Isn't that too slow? I've been using robolectric ( http://pivotal.github.com/robolectric/ ) which creates shadow objects that work similar to mocks. I use progaurd ( http://proguard.sourceforge.net/ ) to strip out the tests for the release build.