I have a test project running successfully on Jenkins using Ant. I see the test results in the console output but how do I generate a report?
I use Jenkins and Espresso also. I use Spoon by Jake Wharton to generate my reports. Take a look! They are clean and very easy to use.
Without using Espresso this can be done easily by using the JUnitReportTestRunner from Zutubi. The report can then be digested e.g. via the Jenkins xUnit Plugin
Espresso though requires its own testrunner that has to be based on GoogleInstrumentation. Combining both the JUnitReportTestRunner and the GoogleInstrumentationTestRunner is quite hacky at the moment, because GoogleInstrumentationTestRunner does not expose a method to attach a test listener. This google groups post shows how to attach the listener from the Zutubi project anyway.
Related
Colleagues,
I am having hard time to find a library/framework/etc... which could help me to generate unit tests in Android.
I know that in Java there are multiple solutions, which could generate unit tests based on the source code. Are you using anything like this for Android?
(My project if fully written in Java).
Use just JUNIT - this should help :
https://developer.android.com/training/testing/unit-testing/local-unit-tests
Use Unit Test Architect
I am in the same boat as you were/are. Although TDD should be the approach for writing tests, but there may be a lot of untested code already written many times in larger projects.
One day I got frustrated with writing test cases of existing, older codebase. Hence, I thought of auto generating all the unit test cases.
I have created an Open-source Gradle Plugin which can be used for the above task. It is already hosted on mavenCentral. I have used it to generate test cases for my projects. But it can be used in any gradle project, (android, java, kotlin, kotlin+java). It may be rough around the edges but it has done it's job well for me.
BuildScript Dependency:
classpath "io.github.orange-3:unit-test-architect:$PLUGIN_VERSION"
gradle-ndk-gtest-sample
Android NDK with
README.NDK
I have been trying to configure my Android NDK project for use with Google Test. I started with native-activity sample as the base project and began following the guides above. I inserted the code from the first link into my root build.gradle. I am really not having any success. It looks like i was able to generate the test library:
./libs/googleTest/1.7.0/lib/osx/libgtest.a
./libs/googleTest/1.7.0/lib/linux/libgtest.a
./app/.externalNativeBuild/cmake/debug/x86_64/libnative_app_glue.a
./app/.externalNativeBuild/cmake/debug/x86_64/libFOO.a
./app/.externalNativeBuild/cmake/debug/x86_64/libgtest.a
./app/.externalNativeBuild/cmake/debug/arm64-v8a/libnative_app_glue.a
./app/.externalNativeBuild/cmake/debug/arm64-v8a/libFOO.a
./app/.externalNativeBuild/cmake/debug/arm64-v8a/libgtest.a
But I don't know where to go from here. How do I actually run the tests and see if they passed?
Try something like https://github.com/DanAlbert/GTestJNI. It's an alternative test runner for JUnit and GTest that exposes GTests as if they were JUnit, so you can just run your normal androidTests and they'll run your GTests for you.
Some day I want to get this either added to the NDK or at least made trivial to add to a project as a git submodule or something, but for now you can merge the pieces of that into your project.
I am using make for building and Robolectric as a framework for running Android tests. I would like to calculate coverage of my app. For instrumentation tests I used to use emmalib. What is the best way for me to set up coverage calculation in this case? I can't migrate to gradle or maven.
Are you bound to Emma? How about using RoboElectric + Cobertura code coverage? (I think you could just use a CLI for the above combination)
So options
1.) RoboElectric + Cobertura - CLI alone probably for someone not on ANT
2.) JaCoCo might have some useful options
3.) Pure Android Testing + Emma/EclEmma
Useful Link trails to follow
Generating android code coverage though changes in build.xml and ant.properties
Android Gradle Code Coverage
https://intellectualcramps.wordpress.com/2013/08/18/code-coverage-of-robolectric-tests-using-jacoco/
https://bitbucket.org/ravidsrk/androidstarter
EDIT:
Well most of the tutorials I have come across use ant unfortunately, and I don't think it would be a bad idea for you to move to a recommended build system like Gradle so it opens up a lot options. But for JaCoCo you could take a look at here: https://intellectualcramps.wordpress.com/2013/08/18/code-coverage-of-robolectric-tests-using-jacoco/
UPDATE:
Moved this from comments to the answer section for information to anyone wanting to see this and because this is a bounty question
One solution would be to use Cobertura to generate code coverage, which can be integrated in eclipse and also run by an ant build script.
A template project of such an integration can be found here: https://github.com/adgllorente/android-cobertura-boilerplate
Note that all of the magic happens in the build.xml of the Test project. Theses tasks should probably be generalised to a custom_rules.xml file so you can still use android to update your projects.
Finally, for Gradle you have many different options:
http://raptordigital.blogspot.nl/2014/08/code-coverage-reports-using-robolectric.html
http://chrisjenx.com/gradle-robolectric-jacoco-dagger/
https://stackoverflow.com/a/25037742/2771851
Note that you can always use Gradle as a secondary build system just to generate the coverage reports. (but a second build system will introduce a lot of overhead)
jacoco sometimes does not work with Robelectric and powermock runner you can use clover an atlassian tools it is now open source tool.
I want to run some JUnit4 tests. The code relies on some Android libraries(Android XML parser), but does not create any activites and so on. When I try to run tests I got that an Android class that I need was not found. Is there any way to run JUnit4 tests with Android code, not to test activity but to test code with some libraries.
I had the same problem and tried to adapt JUnit4 to Android's existing TestRunner - without success. Therefore I created a new project called JUnit4Android. It's a TestRunner application library for JUnit4 and JUnit3 tests and test suites. So you can run your existing JUnit4 tests with it. Please find more information on GitHub:
https://github.com/dthommes/JUnit4Android/wiki
There is no way (that I'm aware of) to use JUnit4 on Android. It does support JUnit3 though, if that's an option for you?
Alternatively, you could use Robolectric and run your tests on your development machine (where you'll be able to use whichever unit test framework you like). Whether this will work for you depends on exactly what you're testing, but it might be worth a go?
It might be little bit late, but there's finally an official update from Google about junit4:
based on Android-test-kit project and some other sources it's clear that:
The AndroidJUnitRunner is a new unbundled test runner for Android, which is part of the
Android Support Test Library and can be downloaded via the Android
Support Repository. The new runner contains all improvements of
GoogleInstrumentationTestRunner and adds more features:
- JUnit4 support
- Instrumentation Registry for accessing Instrumentation, Context and Bundle Arguments
- Test Filters #SdkSupress and #RequiresDevice
- Test timeouts
- Sharding of tests
- RunListener support to hook into the test run lifecycle
- Activity monitoring mechanism ActivityLifecycleMonitorRegistry
Actually, it's already presented in Support Repository. If You go to
%ANDROID_HOME%\extras\android\m2repository\com\android\support\test\testing-support-lib\
it's possible to find testing-support-lib in there (aar, jars etc.) which allows to use JUnit4. Even more, it contains espresso library same location which is handy for UI testing. Seems Android sites and support lib official references will be updated soon with that info.
I'm a long time Java developer with many years of Java EE, Ant, Maven, unit testing, mocks, etc. I'm currently using gradle to build android apps and am looking at unit testing them. And it's got me tearing my hair out!
My reading indicates that to test an app, I have to create another app inside it in a test directory. But I'm not sure how this embedded app can see the main apps classes. I presume that google came up with this because of something to do with the manifests which are different. I'm not sure why.
Without doing this embedded app system, I've been able to get unit tests to run by including the test classes with the main classes in the APK, adding the instrumentation declarations to the manifest, deploying it and running the test runners. But I don't want to be compiling test classes with app classes and including all the dependencies so that's not really an option and I'm not really sure of the effects of the changes to the manifest as I cannot find any documentation about the effects.
None of this is understood by gradle which follows the maven system of building. Also note that the android way seems to be that the embedded sub-project (test) is dependant on the main parent project, something that is totally contray to gradle and maven.
Another option seems to be separate the test project so that it's completely outside the app project. But really, I'd like to use a similar convention to maven and simply have tests in a separate directory, along with the manifest in test resources.
Has anyone managed to get testing on the emulators running unit tests following a more maven like directory structure?
You can try Robotium. It provides lots of features for a better testcase. You can have a look at it here.
Do you have to run the unit tests in the emulator? Isn't that too slow? I've been using robolectric ( http://pivotal.github.com/robolectric/ ) which creates shadow objects that work similar to mocks. I use progaurd ( http://proguard.sourceforge.net/ ) to strip out the tests for the release build.