After much trial and error I've got emma (code coverage tool) with ant for Android dev working purely on command line. Using Eclipse to try and run the test suite I get:
Conversion to Dalvik format failed with error 1
Which could possibly be part of the problem.
Any I've ran all the commands I've found such as
ant emma debug clean
ant emma debug
I use to run the test suite:
ant emma debug install test
Problem
However the report states that code is still untested even though I've written many new tests. IMO, I believe a new debug version isn't being complied, and I'm wondering if anyone has had similar problems. This could be due the the error I linked above, so currently I'm trying various remedies to resolve this first.
Many thanks
EDIT: could this be due to the generated build.xml files?
In my case, the problem was the Robotium tests whilst working perfectly fine, were was it not to compile. To be honest I'm not entirely sure why but removing the tests using Robotium library worked.
Related
I currently have various AndroidTestCases which use Mockito and the module dexmaker-mockito. I am now trying to add Robolectric test cases using the robolectric plugin.
I have everything running fine except that when I try to mock methods with no arguements I get an IllegalArgumentException from Dexmaker. Removing the dexmaker-mockito dependency and using mockito version 1.9.0 instead of 1.9.5 causes the error to go away but causes all of my AndroidTestCases to fail. Is there anyway in gradle to have dexmaker-mockito exclude from my robolectric test cases? Or does anyone know of any other way to run both android unit test and robolectric unit test from android studio?
Error:
java.lang.IllegalArgumentException at com.google.dexmaker.mockito.InvocationHandlerAdapter.invoke
Update
I tried gradle-android-test-plugin and robolectric-gradle-plugin but was unable to to get rid of the IllegalArguementException without changing the version of mockitio to 1.9.0 and removing dexmaker, dexmaker-mockito from my dependencies.
I tried the guide but was unable to get Android Studio to recognize my source directories without adding them to main or test source sets which break my build because robolectric is not compiled for those gradle tasks and I do not want it to be.
I ended up following the guide here this time the error disappeared when I ran the tests from command line but persisted when running from android studio which I found odd. Again changing the version of mockitio to 1.9.0 and removing dexmaker, dexmaker-mockito from my dependencies in my Android App removed the error. For now I will settle for not being able to run the unit tests from android studio until I find a better answer.
If I understand correctly what dexmaker-mockito is supposed to do, it's meant to be used with Android Unit Tests that run on device, so it's good for your Android Test Cases.
Robolectric tests run on the JVM and should be run as JUnit test, not Android tests. They should not be using dexmaker-mockito as they never get deployed to device, they can just use the normal Mockito libraries, with scope set to test.
I think you may not be able to mix the two, you may have two keep your robo tests separate from your Android unit tests.
I've written some unit tests in my Android projects with Robolectric, and some of them even don't use Android.
I've tried first this: robolectric-gradle-plugin, following this template.
The result is that I've faced this bug in IDEA, that I reported myself.
Then I've tried another possibility: the gradle-android-test-plugin, which separates tests in another submodule, that is a regular Java project. I've tried to follow this other template. I've open this bug report.
Then I've faced again with other bug.
What recommendations can you provide me, that does work?
Finally I've had to upgrade IDEA to 14 EAP, and know it works ok.
gradle-android-test-plugin seems to work better than robolectric plugin, as I can run JUnit and ScalaTest tests easily. I've had to install with a script, the non-published com.novoda:gradle-android-test-plugin:0.9.9-SNAPSHOT.
It's a hell the combination of versions of:
Android Build Tools
Gradle
com.android.tools.build:gradle
IDEA
robolectric
robolectric plugin
....
Any change in one of them, can make your project fail.
Luckily, by using Gradle you can fix the versions of every component, and do Continous integration.
BREAKING NEWS: Android Studio 1.1.0 now has built-in support for unit testing. See more here: https://sites.google.com/a/android.com/tools/tech-docs/unit-testing-support
At first I know that there are very, very, very much similar questions (even here on SO). Most of them are terribly outdated so I'm asking a fresh new question which should be up to date for Android Studio 0.8.x.
I know there are some libraries which work somehow, but in almost each case I had no evidence that the information were up to date. I know there is e.g. roboelectric, there are also some deprecated Jake Wharton like gradle-android-test-plugin or double-espresso, I also found the library RoboSpock and Deckard. But none of them seems to have any Android Studio integration.
After long reseach I found this two bugs in all implementations:
The classpath is broken and needs to been hacked to get junit running
The junit tests cannot been debugged
What I'm looking for:
I want to automatically test an algorithm (which is plain java)
I want to check the integration in my Android app works
I want an integration in Android Studio so that I can jump into the code out of a stacktrace
I want to step thrue the test code
I found also somewhere a nice hack which run the tests in gradle as an external task and pushed the results into AS so that the results could been displayed, but unfortunately I cannot find that link again (and if I remember correctly it did also not work for me).
Do you guys have some fresh references? Does it work for you?
You have to edit the .iml file that Android Studio generates to point to your test-classes directory and also to fix the Stub error from Junit. There is currently no work around for this.
Gradlectric is a sample that uses the Robolectric Gradle Plugin to run unit tests in Android Studio.
Here you miss one example project setup https://github.com/nenick/android-gradle-template
But none of them seems to have any Android Studio integration.
This project is maintained for android studio
The classpath is broken and needs to been hacked to get junit running
gradle scripts will fix the classpath
The junit tests cannot been debugged
tests a running inside AS so they are easy to debug
This project of mine does not cover all your requests, but a fair lot of them.
It covers a default Java test case, and some tests interacting with Views.
The root project uses Travis CI, which uses gradle connectedCheck to run the tests. To be able to debug the tests, you can just create a test run configuration in AS and run it in debugging mode.
Note that this does not use any special Android testing frameworks like robolectric.
You can find a fork of Jake Wharton's gradle-android-test-plugin here. This is compatible with AS 0.6 and is regularly maintained by Robolectric. You can look at this project to set up Robolectric in Android Studio with help of this plugin. You can achieve fair bit of functionality you mentioned with the help of robolectric-gradle-plugin. You can also successfully debug your test code using AS.
We're trying to get code coverage analysis of a set of manual tests run on an Android application running on a device. We build our application using maven, so the cool emma and ant solution isn't viable.
Does anybody have explicit experience getting emma or cobertura (or any other open source coverage tool) working for manual testing of Android applications? We can't afford to purchase a solution (so no Clover, or Java Test Coverage Tool, etc.).
I have seen several post regarding making a build in Jenkins fail if the unit test execution fail (e.g. this one). It turns out that by default Jenkins reports builds with failing tests as unstable and some people do not like that. This, however, will be perfectly fine for me. I just want to be able to easily differentiate builds with passing tests from such with failing tests.
And here is the catch: I am developing for Android so my build is configured following this page. Basically the tests are run with the following command:
ant all clean emma debug install test
As result coverage report is generated and published in Jenkins.
All posts I have read about configuring the Jenkins result according to tests results
were dealing with ant task manipulation. However, if we look at the android build.xml the Android tests are run with adb command: adb shell am instrument .... I don't know how to configure this command to print the tests results. It can be configured to print the coverage report. I have already done that, but was never able to make the build fail according to the coverage report.
I hope somebody else also faced the same problem and managed to solve it. Any guidance will be most appreciated.
Consider using 'the-missing-android-xml-junit-test-runner' - the idea being to produce jUnit reports that Jenkins understands. Disclaimer: I do have no personal experience with adb. However, I do know from experience that Jenkins deals with properly generated jUnit reports very gracefully.
This is possible, configure the server using the following documentation:
https://wiki.jenkins-ci.org/display/JENKINS/Android+Emulator+Plugin
See "Parsing monkey output" in the link.
Addition, Jenkins can test instrumentation, below have a link of a good place to start in the learning of instrumentation:
http://developer.android.com/resources/tutorials/testing/helloandroid_test.html
I hope it is useful.