I am working on a project where in we are trying to use the ANDROID TEST ORCHESTRATOR for it's obvious benefits of isolating crashes. But while executing the test suite, it appears to me that as the orchestrator initiates a new process for every test case, the test coverage report for the suite execution is always showing incomplete data (mostly the data for the last test case present in the test suite).
So I was wondering that is there a way to overcome this problem and generate a jacoco code coverage report for all instrumented tests existing in the test suite.
If you're using the android test orchestrator this is the problem. There is an open bug report: https://issuetracker.google.com/issues/72758547
The only solution that I know of is to disable the android test orchestration until a fix is released.
In my Android Java build.gradle I had to comment out the test orchestrator like so:
android {
...
testOptions {
// temporarily disable the orchestrator as this breaks coverage: https://issuetracker.google.com/issues/72758547
//execution 'ANDROID_TEST_ORCHESTRATOR'
...
}
}
The issue mentioned above is now fixed and the latest gradle version is working fine for all android devices.
Related
I want to generate unit and integration test coverage html report for my Android project. This project is composed of 3 modules:
app module (I don't really want to make a report for this module)
sdk module (real target of the tests)
sdk-integration-tests (containing the sdk module integration tests)
I am following exactly this implementation
https://blog.mindorks.com/generate-global-code-coverage-report-in-android-development-using-jacoco-plugin
All tests pass.
But:
The report displays 0% of coverage. It does not match the actual test coverage. Classes and methods widely used in the tests are still marked as not covered.
I would like to merge the reports of two modules (sdk and sdk-integration-tests)
EDIT: One important remark: the tests use Mockito and Robolectric. I really think it could play an important part in the erroneous analysis of code coverage.
Do you have any idea?
Could you share your junitJacoco configuration?
Last time I also face the same issue, until find the solution here:
tasks.withType(Test) {
jacoco.includeNoLocationClasses = true
jacoco.excludes = ['jdk.internal.*']
}
I have written test cases using JUnit and PowerMockito. IntelliJ IDEA is showing 100% coverage for lot many classes but Sonar is showing just 19%
I am using following versions:
PowerMockito: 2.0.2
JUnit: 4.12
Jacoco: 0.8.5
Sonar: 7.3
i have gone through multiple posts but nothing solved this issue.
This sounds like you're running into the issue that Powermock replaces the Jacoco annotation during its instrumentation, so Jacoco has nothing to report against. You may be able to change the instrumentation type to overcome this, but it's currently a know limitation when using Powermock.
Is anyone aware of a sample project that shows how to get test orchestrator working? I checked the google samples and there doesn't seem to be a good sample project that shows test orchestrator.
https://github.com/googlesamples/android-testing
I have been attempting to get android tests running in the test orchestrator but have been struggling to get it to work correctly. I tried both running the tests through Android Studio (latest 3.2.1) as well as the command line (https://developer.android.com/training/testing/junit-runner#ato-command-line). I used the Android developer document for reference.
https://developer.android.com/training/testing/junit-runner
Here are the steps I followed.
1) Create an empty activity application using the wizard in Android
Studio
2) Enable the test orchestrator using the steps provided here
(https://developer.android.com/training/testing/junit-runner).
3) Run the unit tests from within the IDE and from the command line.
When I do this, I get an error indicating that my "test suite is empty". I get the same error running from command-line.
Note that if I run the test without test orchestrator, then the test runs successfully.
Also note that I am using the latest test orchestrator versions
test-orchestrator (https://maven.google.com/androidx/test/orchestrator/1.1.0/orchestrator-1.1.0.apk)
test-services (https://maven.google.com/androidx/test/services/test-services/1.1.0/test-services-1.1.0.apk)
The complete configuration to run test orchestrator:
Add dependencies:
androidTestImplementation "androidx.test:runner:$testRunner"
androidTestUtil "androidx.test:orchestrator:$testOrchestrator"
Add clear package instruction (within defaultConfig in app's build.gradle):
//allows run all tests on an isolated way. If we need to debug a test, should disable this and the orchestrator
testInstrumentationRunnerArguments clearPackageData: 'true'
Add to testOptions Android/AndroidX orchestrator:
testOptions {
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}
I have read around, there are a number of extensive answers (like this one) but the Android world evolves so fast that they seem to be a bit outdated and the official documentation still refers to Eclipse with ADT.
I am running AS 1.1 and I am trying to setup simple junit tests to run on the emulator, without Robolectric. If I don't include junit in my build.gradle, it can't find #After, #Before and #Test and I get package org.junit does not exist. Upon adding
// unit tests
androidTestCompile 'junit:junit:4.11'
the error becomes
Error:duplicate files during packaging of APK
[...]/app/build/outputs/apk/app-debug-test-unaligned.apk
Path in archive: LICENSE.txt
Origin 1: [...]/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar
Origin 2: [...]/.gradle/caches/modules-2/files-2.1/junit/junit/4.11/4e031bb61df09069aeb2bffb4019e7a5034a4ee0/junit-4.11.jar
You can ignore those files in your build.gradle:
android {
packagingOptions {
exclude 'LICENSE.txt'
}
}
Following the console suggestion of excluding LICENSE.txt, it then works but it feels like a hack. So I'm wondering, am I maybe missing something? Thanks.
Android Studio unit testing support comes in 1.1 Beta 4 (release announcement) with Gradle plugin version 1.1.0-rc1.
More info in official document.
However it is experimental feature for now. E.g. it breaks installDebug gradle task.
For using JUnit in instrumentation tests there is good guide for Espresso library and another covering new AndroidJUnitRunner.
If it's any use I set up a boiler plate project allowing the use of Unit tests and Espresso tests by the use of switching build variants. You won't need the use of any third party plugins with this.
https://github.com/hitherejoe/Android-Boilerplate
I have a question about running TestSuites with gradle.
So under androidTest/java I have two packages, the first one contains instrumentation tests (ActivityInstrumentationTestCase2) and the other is a TestSuite which contains those instrumentation tests. If I run the test suite in Android Studio, it works fine. How can I run it from gradle? I mean, I would like to run it from Jenkins. What should be that gradle task? I have tried with connectedAndroidTest, and it runs the instrumentation test in non-specific order, instead of the test suite.
You can add filters to choose your tests. If you only chose your test suite, you can use the normal test task provided by gradle.
test {
filter {
includeTestsMatching "*MyTestSuite"
}
}