Android Unit Test Not Reporting Failing with fail() - android

I've written a unit test that simply extends TestCase and I have the following:
public class MetricParserTests extends TestCase {
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public void testFailure() {
fail("This needs to fail");
}
}
When I run my tests using ant test or adb shell am instrument I get the following results:
... [exec] OK (1 tests) ...
I'd expect to see a failure on the command line.

I believe I know what the issue is. I was able to reproduce the issue and solve it. The command you use does not rebuild and re-install your test project onto a device. When you call ant test it will just execute the tests which are already installed on that device.
What you need to call is the three commands in your test project's directory:
ant debug
ant installd
ant test
Then all tests will be rebuild and re-installed and latest tests will be executed. If you don't call debug and installd, the changes you did to the tests do not get applied.

I haven't had recent experience in Android testing, but here is what I have found...
You can use normal JUnit tests if your code is totally decoupled from Android (see here
for an example). This would run on your JVM using the JUnit runner.
However, if you are trying to run these tests on an Android device (either via ant, or the command line tools) then you need to create a full android test project (See here).
To test "on device" your test cases need to extend one of the Android test classes like ActivityInstrumentationTestCase2<T>
and are run using the InstrumentationTestRunner in the Dalvik VM on the Android device.
Using an IDE or the command-line tools to create a test project should create a sample test for you to work from.
This blog post linked from the comments of the post above is a good source of information, as is the Android Testing Fundamentals doc.

The method testFailure() does not have a #Test annotation. Is that correct?

Related

Running test that was created by Android Studio

When creating a new project in Android Studio, I notice that it automatically creates an /androidTest directory under /src, where there is "ApplicationTest.java" class with the following code:
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
}
I'm guessing this is what Google wants us to use, but after searching for hours, I couldn't figure out how to use this class that was generated for me. Google's official doc seems to only list how to run on Eclipse IDE (not Android Studio), and I couldn't find any code that would let me perform a simple test (say like assertEquals(1,2)). Can someone show me how to write a simple test code using the above default template, and steps on how to run it, preferably from the command line?
EDIT:
I was able to write a simple test that is intended to fail.
ApplicationTest.java in /androidTest/java/path/to/package/
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
#Override
protected void setUp() throws Exception {
createApplication();
}
#SmallTest
public void testMultiply() {
assertEquals("This should not pass ", 50, 49);
}
}
I am able to run this from Android Studio, but I just cannot figure out how to run that from the command line. Any help?
As Jared already said, this is an example setup for an instrumentation test.
I guess, you've already taken a look on this: Testing Fundamentals
Simpliest way to run these tests in android studio is to right click the class and click run.
It is also possible to add tests in your run/debug configurations.
UPDATE:
To run the instrumentation tests on command line, use ./gradlew assembleAndroidTest. This command will run all tests in your src/androidTest (instrumentation test) folder.
As njzk2 mentioned, there also is a ./gradlew assembleTest command. This command is for running all unit tests (which should be placed in the src/test folder). For more information about unit testing in android take a look on this: Android Unit Testing Support
EXAMPLE:
Here an example for an instrumentaion test in android:
#Override
public void setUp() throws Exception {
super.setUp();
InputStream is = getContext().getAssets().open("test.xml");
XmlParser parser = new XmlParser(getContext());
parser.parse(is);
...
}
#MediumTest
public void testSomething() throws Exception {
// test some data your parser extracted from the xml file
...
}
As you can see, i need the context for creating the input stream, therefor i have to go for an instrumentation test. The #MediumTest signals e.g. i'm accessing files from storage (see Test Annotations).

setup and write demo test the robolectric in Android Studio

Do anyone know how to integrate robolectric into android studio?
How to write sample test?
How to launch it?
I am working with android studio not to long, and I am too bad with gradle.
Searching the net didn't give me a result - I even could not launch official demo - https://github.com/robolectric/robolectric-samples . My android studio do not saw the test class.
Please give me simpliest step by step gide, thanks
Since robolectric runs in a JVM (i.e. not on a device or emulator), it is just a library and adding the test runner is all that's needed.
Make sure that the android SDK is later in the classpath than robolectric or junit - otherwise you'll get the stubbed methods from the android SDK.
#RunWith(RobolectricTestRunner.class)
public class MyActivityTest {
#Test
public void shouldHaveApplicationName() throws Exception {
String appName = new MyActivity().getResources().getString(R.string.app_name);
assertThat(appName, equalTo("MyActivity"));
}
}
See http://robolectric.org/quick-start/

Android Studio - test configuration starts up app

I've got a couple of AndroidTestCase subclasses in separate packages of my project:
However, whenever I run Android Tests configuration from Android Studio, I see that my regular app is starting as well. I see that the onCreate method is fired up inside my Application class (which is really bad since I am loading some additional resources there).
Why is Android Studio/gradle running my app as well? Can I programatically detect if I am inside test or regular configuration? Can I stop my regular app from being booted before running tests?
In addition, when I am running tests in debug mode it doesn't stop on breakpoints placed inside the application's onCreate method. Why is this happening?
Edit:
Body of test class doesn't really matter, it can be something like:
public class SimpleTest extends AndroidTestCase {
public void testSample()
{
assertEquals(true, false);
}
}
Executing only this simple test fires up onCreate method inside application class.
Gradle console prints out:
Executing tasks: [:app:assembleDebug, :app:assembleDebugTest]
I guess that first task creates instance of my Application class - is it expected behavior?
Why is Android Studio/gradle running my app as well?
Can I stop my regular app from being booted before running tests?
AndroidTestCase is extension of JUnit TestCase which is aware of your android application. In case you don't need to test your android application and want to test plain java only you should use JUnit framework. Create regular JUnit tests, do not use android classes there and run JUnit test configuration like this:
You should treat AndroidTestCase as instrumentation tests which will build android app and run that tests on it. This is usefull with combination of Espresso and Robotium. Both are working on top of base android test classes and both will build and run your application before testing it. Real device or emulator is needed.
Use plain JUnit tests or Robolectric to test java on your desktop JVM.
Can I programatically detect if I am inside test or regular configuration?
You can use power of gradle to provide such info with autogenerated BuildConfig file.
At your build.gradle
android {
defaultConfig {
testPackageName "com.foo.test"
}
}
At your code:
BuildConfig.PACKAGE_NAME.equals("com.foo.test")
The AndroidTestCase is an unit test that unfortunately runs on the device (either virtual or real).
I think what you want to have is a UnitTestFramework that runs in the JVM (local on your machine). The TestFramework Robolectric can do this.
I have started a gitHub project to show how to setup the gradle test file and the project structure if you want to have UnitTests and InstrumentationTests side by side. If you want to look its AndroidGradleTests

Running a specific instrumentation unit test with Gradle

Is there a way to run a specific Android instrumentation unit test using Gradle? I've tried
gradle -Dtest.single=UnitTestName connectedInstrumentTest
but it seems to run all the tests in the package.
Using test.single appears to be deprecated. The new correct way to do this is
./gradlew :<module>:test --tests <pattern>
where <pattern> could be something like:
com.example.MyTest to run all test methods in com.example.MyTest
*MyTest to match every method in every class whose name ends with MyTest
*.MyTest.myMethod to run a specific test method in class MyTest in any package
If you have a multi-project build, make sure to give the module path before the test task; otherwise you'll get a misleading error message when it searches for your test pattern in every subproject.
None of this is documented on the Gradle site anywhere I could find it.
This works if you're using an instrumentationTestRunner:
./gradlew test -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
Using gradle 2.10 and android gradle plugin 2.0.0-beta2.
Since you know what test(s) you want to run, you probably know which module / flavor to use too. You can help Gradle out by specifying the exact module and Gradle task. So if your test is in the app module and you want to test the debug flavor:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=<pkg>.YourClassName
You can get even more fancy with the tests_regex argument instead:
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=PartialClassName*
./gradlew app:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.tests_regex=partialMethodName*
The pattern is -D<testTaskName>.single=<TestClass> so in your example it should be:
gradle -DconnectedInstrumentTest.single=UnitTestName connectedInstrumentTest
NOTE: This answer is outdated. You should use the --tests switch in the latest versions of Gradle. (see other answers for an explanation)
Since Android gradle plugin 1.1.0-rc1, one can run single test class using --tests flag by executing:
./gradlew app:testDebug --tests=com.example.MyTest
See http://tools.android.com/tech-docs/unit-testing-support#TOC-Running-from-Gradle
You gotta check this out.
https://github.com/JCAndKSolutions/android-unit-test
I made an issue in this github repository, and this guy solved my problem and upload to maven, so in my build.gradle file I use this plugin.
Instructions are written in his repository. you can easily follow it.
After using this android-unit-test plugin, I can use like
../gradlew -Dtest.single=SomeTest test
or
../gradlew -Dtest.single=SomeTest clean check
Now it's working and I could only run the specific tests I want to
You should not forget to specify a build variant name after test property declaration like
-Dtest<buildVariantName>=<yourTestName>.
Like if you have a debug build type which gives you debug variant after compilation, then if you want to run a test only for this build variant you should declare a command like this:
./gradlew -DtestDebug=UnitTestName testDebug
Erdi's answer didn't work for me but I have a single parent for all my test classes so I was able to do this:
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if(getClass().getSimpleName().equals("MyTestClassName")) {
super.runTest();
}
}
//...
}
This executes only MyTestClassName. We can extend it further to execute only specific test method (or methods):
public abstract class BaseEspressoTest<T extends Activity> extends ActivityInstrumentationTestCase2<T> {
//...
#Override
protected void runTest() throws Throwable {
if("MyTestClassName".equals(getClass().getSimpleName())
&& "testMethodName".equals(getName())) {
super.runTest();
}
}
//...
}
the Gradle command does not work for me.
I used below mentioened adb command.
for this you need to build your apk first.
adb shell am instrument -w -r -e package -e debug false .debug.test/android.support.test.runner.AndroidJUnitRunner

Parameterized JUnit tests in Android test project

When I create parameterized test cases in JUnit 3.x, I usually create a TestSuite with something like
public static Test suite() {
TestSuite s = new TestSuite();
for (int i = MIN; i < MAX; ++i) {
s.addTest(new MyTest(i));
}
}
This suite() method is called correctly when running JUnit from a desktop command-line. When I tried this with my Android test project, the tests don't run. How do I get my tests to run on the emulator? Or is there a different way to create parameterized tests for Android?
More thoughts:
Typically I run my tests with the command line:
adb shell am instrument -w [-e class <fully qualified test class name>[#<test method name>()]] <Android package name>/android.test.InstrumentationTestRunner
This allows me to select which tests to run from my test suite. Ideally, I want to run the the parameterized tests in this way as well. The link in the comment from #Appu describes building a separate app that runs JUnit tests. As part of that, this app has a custom TestRunner. I can very likely borrow these ideas to create a TestRunner which I can use in place of android.test.InstrumentationTestRunner. This seems like a lot of work for a not uncommon task. I prefer not to reinvent the wheel if there is already a similar solution in the Android API. Does anyone know of such a thing? Also, other alternative solutions will be helpful.
Nevermind, it looks like #dtmilano already posted this as an answer...
You can implement a test runner to be able to pass parameters to Android tests.
See the example at how to pass an argument to a android junit test (Parameterized tests).
Or is there a different way to create parameterized tests for Android?
We (Square) wrote a library called Burst for this purpose. If you add enum parameters in your test constructor, Burst's test runner will generate a test for each combination of enum values. For example:
public class ParameterizedTest extends TestCase {
enum Drink { COKE, PEPSI, RC_COLA }
private final Drink drink;
// Nullary constructor required by Android test framework
public ConstructorTest() {
this(null);
}
public ConstructorTest(Drink drink) {
this.drink = drink;
}
public void testSomething() {
assertNotNull(drink);
}
}
Quite a while after originally writing this question, I discovered that I can directly run a test class which contains a static suite() method:
adb shell am instrument -w -e class <fully qualified test class name> <Android package name>/android.test.InstrumentationTestRunner
However, the test suite doesn't run when I try to run all the tests in a given package.
Of course, this has been a while. Now I am using Android Studio instead of the command-line. I can still run the test class individually, but it still doesn't run when I select a package or try to run all of my tests.
A potential alternative is to write a master test class with a suite() method which adds all the tests to the returned TestCase. Unfortunately, this requires some manually editing every time I add a new test class to my suite.

Categories

Resources