Colleagues,
I am having hard time to find a library/framework/etc... which could help me to generate unit tests in Android.
I know that in Java there are multiple solutions, which could generate unit tests based on the source code. Are you using anything like this for Android?
(My project if fully written in Java).
Use just JUNIT - this should help :
https://developer.android.com/training/testing/unit-testing/local-unit-tests
Use Unit Test Architect
I am in the same boat as you were/are. Although TDD should be the approach for writing tests, but there may be a lot of untested code already written many times in larger projects.
One day I got frustrated with writing test cases of existing, older codebase. Hence, I thought of auto generating all the unit test cases.
I have created an Open-source Gradle Plugin which can be used for the above task. It is already hosted on mavenCentral. I have used it to generate test cases for my projects. But it can be used in any gradle project, (android, java, kotlin, kotlin+java). It may be rough around the edges but it has done it's job well for me.
BuildScript Dependency:
classpath "io.github.orange-3:unit-test-architect:$PLUGIN_VERSION"
Related
I like the Kotlin REPL in Idea / Android-Studio - but as an Android Developer I often run into Stub! problems here. When writing unit-tests I am using unmock to work around this problem. Is there a way to use the same method used there for the Kotlin REPL plugin?
[
All android (and java.lang.*) classes are placeholders in an Android project. This is because android does not use standard java class files to store the compiled code and there is no way to directly run this code on a computer.
You simply can't use the REPL with android classes, they will only exist on an actual device or emulator.
If you do not care about correctness, then you can use Robolectric's implementation of Android by adding it as a dependency to the project.
To make sure it does not collide with the actual implementation you should probably do this with a separate module dedicated to the REPL.
Robolectic's dependency used by unmock is: org.robolectric:android-all:7.1.0_r7-robolectric-0
The problem is that the Kotlin REPL in IDEA is provided by the Kotlin IDEA plugin, which has no notion of Android per se, but only looks at what's in the classpath, which in this case is the android.jar containing the stubs that throw the exception you mentioned.
Using unmock or even the integrated Android support for removing exceptions from the stubs in tests (see here at the end of "Mock Android dependencies") won't work as that only affects your Gradle build.
The only solution I can think of is to either open an issue on the Kotlin tracker or dig through the source code of the REPL function in the Kotlin plugin and send a Pull Request.
I am using make for building and Robolectric as a framework for running Android tests. I would like to calculate coverage of my app. For instrumentation tests I used to use emmalib. What is the best way for me to set up coverage calculation in this case? I can't migrate to gradle or maven.
Are you bound to Emma? How about using RoboElectric + Cobertura code coverage? (I think you could just use a CLI for the above combination)
So options
1.) RoboElectric + Cobertura - CLI alone probably for someone not on ANT
2.) JaCoCo might have some useful options
3.) Pure Android Testing + Emma/EclEmma
Useful Link trails to follow
Generating android code coverage though changes in build.xml and ant.properties
Android Gradle Code Coverage
https://intellectualcramps.wordpress.com/2013/08/18/code-coverage-of-robolectric-tests-using-jacoco/
https://bitbucket.org/ravidsrk/androidstarter
EDIT:
Well most of the tutorials I have come across use ant unfortunately, and I don't think it would be a bad idea for you to move to a recommended build system like Gradle so it opens up a lot options. But for JaCoCo you could take a look at here: https://intellectualcramps.wordpress.com/2013/08/18/code-coverage-of-robolectric-tests-using-jacoco/
UPDATE:
Moved this from comments to the answer section for information to anyone wanting to see this and because this is a bounty question
One solution would be to use Cobertura to generate code coverage, which can be integrated in eclipse and also run by an ant build script.
A template project of such an integration can be found here: https://github.com/adgllorente/android-cobertura-boilerplate
Note that all of the magic happens in the build.xml of the Test project. Theses tasks should probably be generalised to a custom_rules.xml file so you can still use android to update your projects.
Finally, for Gradle you have many different options:
http://raptordigital.blogspot.nl/2014/08/code-coverage-reports-using-robolectric.html
http://chrisjenx.com/gradle-robolectric-jacoco-dagger/
https://stackoverflow.com/a/25037742/2771851
Note that you can always use Gradle as a secondary build system just to generate the coverage reports. (but a second build system will introduce a lot of overhead)
jacoco sometimes does not work with Robelectric and powermock runner you can use clover an atlassian tools it is now open source tool.
I've needed to recently introduce ProGuard on Android because of issues with Scala on Android. I need ProGuard for its shrinking feature, which removes classes presumed to be unused. I'm very concerned about the impact of removing classes on testability.
As it stands, I write unit tests that run on the host and acceptance tests that run the fully integrated application on the Android platform.
Normally, I would be comfortable with relatively complete unit test coverage and spotty acceptance test coverage. However, given that in my code I use Guice dependency injection heavily, so far it's been my experience that ProGuard removes code in a manner that's difficult for me to predict. Because of this it's very likely to cause me to introduce bugs.
This leads me to believe that I need to write acceptance/platform tests that achieve full coverage because at any point, there may be a missing class.
Do others have this experience? If so, what has been your testing strategy? Or with experience, do you become more confident that the classes that ProGuard is removing are truly unneeded?
ProGuard will not break your application until it attempts to use reflection or Class#forName on removed classes and/or obfuscated members.
From my experience (with obfuscated Scala on Android too) it is really easy to spot problems caused by ProGuard to your Android application using the simple smoke tests. You know what libraries you include in your project. If some of them uses reflection or Class#forName - perform smoke test on them. Then exclude the necessary classes/members from the ProGuard configuration.
Remember also that you can automatize testing of your obfuscated project using the ActivityInstrumentationTestCase2 and emulator. If you plan to use ProGuard on your project, always perform instrumentation testing on obfuscated APK.
In conclusion - fear not. ProGuard-related problems are relativity easy to spot.
We've been both unit testing and "fully" testing our ProGuard-ed application for quite a while now, and we've had no "real" problems. The only issues we run into is when we use some library methods in our tests that aren't used in the main application; in these cases ProGuard will remove the code from the libraries and we would have to manually add the specific methods to proguard.cfg.
Oh, and we also use Guice :)
I want to run some JUnit4 tests. The code relies on some Android libraries(Android XML parser), but does not create any activites and so on. When I try to run tests I got that an Android class that I need was not found. Is there any way to run JUnit4 tests with Android code, not to test activity but to test code with some libraries.
I had the same problem and tried to adapt JUnit4 to Android's existing TestRunner - without success. Therefore I created a new project called JUnit4Android. It's a TestRunner application library for JUnit4 and JUnit3 tests and test suites. So you can run your existing JUnit4 tests with it. Please find more information on GitHub:
https://github.com/dthommes/JUnit4Android/wiki
There is no way (that I'm aware of) to use JUnit4 on Android. It does support JUnit3 though, if that's an option for you?
Alternatively, you could use Robolectric and run your tests on your development machine (where you'll be able to use whichever unit test framework you like). Whether this will work for you depends on exactly what you're testing, but it might be worth a go?
It might be little bit late, but there's finally an official update from Google about junit4:
based on Android-test-kit project and some other sources it's clear that:
The AndroidJUnitRunner is a new unbundled test runner for Android, which is part of the
Android Support Test Library and can be downloaded via the Android
Support Repository. The new runner contains all improvements of
GoogleInstrumentationTestRunner and adds more features:
- JUnit4 support
- Instrumentation Registry for accessing Instrumentation, Context and Bundle Arguments
- Test Filters #SdkSupress and #RequiresDevice
- Test timeouts
- Sharding of tests
- RunListener support to hook into the test run lifecycle
- Activity monitoring mechanism ActivityLifecycleMonitorRegistry
Actually, it's already presented in Support Repository. If You go to
%ANDROID_HOME%\extras\android\m2repository\com\android\support\test\testing-support-lib\
it's possible to find testing-support-lib in there (aar, jars etc.) which allows to use JUnit4. Even more, it contains espresso library same location which is handy for UI testing. Seems Android sites and support lib official references will be updated soon with that info.
I'm a long time Java developer with many years of Java EE, Ant, Maven, unit testing, mocks, etc. I'm currently using gradle to build android apps and am looking at unit testing them. And it's got me tearing my hair out!
My reading indicates that to test an app, I have to create another app inside it in a test directory. But I'm not sure how this embedded app can see the main apps classes. I presume that google came up with this because of something to do with the manifests which are different. I'm not sure why.
Without doing this embedded app system, I've been able to get unit tests to run by including the test classes with the main classes in the APK, adding the instrumentation declarations to the manifest, deploying it and running the test runners. But I don't want to be compiling test classes with app classes and including all the dependencies so that's not really an option and I'm not really sure of the effects of the changes to the manifest as I cannot find any documentation about the effects.
None of this is understood by gradle which follows the maven system of building. Also note that the android way seems to be that the embedded sub-project (test) is dependant on the main parent project, something that is totally contray to gradle and maven.
Another option seems to be separate the test project so that it's completely outside the app project. But really, I'd like to use a similar convention to maven and simply have tests in a separate directory, along with the manifest in test resources.
Has anyone managed to get testing on the emulators running unit tests following a more maven like directory structure?
You can try Robotium. It provides lots of features for a better testcase. You can have a look at it here.
Do you have to run the unit tests in the emulator? Isn't that too slow? I've been using robolectric ( http://pivotal.github.com/robolectric/ ) which creates shadow objects that work similar to mocks. I use progaurd ( http://proguard.sourceforge.net/ ) to strip out the tests for the release build.