When I create parameterized test cases in JUnit 3.x, I usually create a TestSuite with something like
public static Test suite() {
TestSuite s = new TestSuite();
for (int i = MIN; i < MAX; ++i) {
s.addTest(new MyTest(i));
}
}
This suite() method is called correctly when running JUnit from a desktop command-line. When I tried this with my Android test project, the tests don't run. How do I get my tests to run on the emulator? Or is there a different way to create parameterized tests for Android?
More thoughts:
Typically I run my tests with the command line:
adb shell am instrument -w [-e class <fully qualified test class name>[#<test method name>()]] <Android package name>/android.test.InstrumentationTestRunner
This allows me to select which tests to run from my test suite. Ideally, I want to run the the parameterized tests in this way as well. The link in the comment from #Appu describes building a separate app that runs JUnit tests. As part of that, this app has a custom TestRunner. I can very likely borrow these ideas to create a TestRunner which I can use in place of android.test.InstrumentationTestRunner. This seems like a lot of work for a not uncommon task. I prefer not to reinvent the wheel if there is already a similar solution in the Android API. Does anyone know of such a thing? Also, other alternative solutions will be helpful.
Nevermind, it looks like #dtmilano already posted this as an answer...
You can implement a test runner to be able to pass parameters to Android tests.
See the example at how to pass an argument to a android junit test (Parameterized tests).
Or is there a different way to create parameterized tests for Android?
We (Square) wrote a library called Burst for this purpose. If you add enum parameters in your test constructor, Burst's test runner will generate a test for each combination of enum values. For example:
public class ParameterizedTest extends TestCase {
enum Drink { COKE, PEPSI, RC_COLA }
private final Drink drink;
// Nullary constructor required by Android test framework
public ConstructorTest() {
this(null);
}
public ConstructorTest(Drink drink) {
this.drink = drink;
}
public void testSomething() {
assertNotNull(drink);
}
}
Quite a while after originally writing this question, I discovered that I can directly run a test class which contains a static suite() method:
adb shell am instrument -w -e class <fully qualified test class name> <Android package name>/android.test.InstrumentationTestRunner
However, the test suite doesn't run when I try to run all the tests in a given package.
Of course, this has been a while. Now I am using Android Studio instead of the command-line. I can still run the test class individually, but it still doesn't run when I select a package or try to run all of my tests.
A potential alternative is to write a master test class with a suite() method which adds all the tests to the returned TestCase. Unfortunately, this requires some manually editing every time I add a new test class to my suite.
Related
Hi I am trying to run a basic JUnit test in Android Studio. When I click run it is terminated every time before getting results. I never get pass or fail and the system does not try any other test cases.
Here are the pictures. I would like to know why it isn't working so I can keep practicing.
You can try a couple of things
Decorate your test class with #SmallTest
// #RunWith is required only if you use a mix of JUnit3 and JUnit4.
#RunWith(AndroidJUnit4::class)
#SmallTest
class ExampleInstrumentedTest {
}
Try to run your test from the command line:
./gradlew connectedAndroidTest
I am interested in how to run Espresso tests from command line (gradle task) individually (run group/suite tests then close app and then run another group/suite of tests).
Found that it is feasible to implement JUnit Test Suites but do not really understand how does it looks like under the hood in a context of instrumentation tests. Does it starts separate processes per Test suite? There is sample application on Github but how to execute it from terminal?
Another interesting discovery is Sharding tests. However, it one sentence documentation.
May be somebody can share with any experience of running Espresso tests individually.
Most of this is documented as part of AndroidJUnitRunner: https://developer.android.com/reference/android/support/test/runner/AndroidJUnitRunner.html
The key piece that is missing is how to pass those parameters via Gradle. You can do that by specifying the options at the commandline as such:
./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=*The full name of your test suite goes here*
I would recommend using the Spoon library to run your tests individually or in parallel on multiple devices. You can either use the jar file or use the Spoon gradle plugin mentioned on the same page. Both of them have adequate documentation to help you set it up.
You can also use Spoon to run an individual test and the command would look something like this:
./gradlew yourSpoonTaskName -PspoonClassName=com.yourPackageName.blah.ClassName
-PspoonMethodName=methodName
In order to know what yourSpoonTaskName is run ./gradlew tasks.
Also, in your build.gradle file add the following spoon configuration:
spoon {
// for debug output
debug = true
// To grant permissions to Android M >= devices
grantAllPermissions = true
// for sharding
/*
this will execute tests in parallel on multiple devices.
*/
shard = true
// Add this to run a specific test class & method
if (project.hasProperty('spoonClassName')) {
className = project.spoonClassName
}
if (project.hasProperty('spoonMethodName')) {
methodName = project.spoonMethodName
}
}
If you are not interested in Spoon and just want a simple solution, then use the following command to run an individual test:
am instrument -w -r -e class com.packageName.blah.TestName#methodName com.packageName.blah.YourIntrumentationRunnerName
You can easily determine these values if you right click the test name in AndroidStudio and run it. In the console, you will see the entire command being printed when the test is bring run.
I've got a couple of AndroidTestCase subclasses in separate packages of my project:
However, whenever I run Android Tests configuration from Android Studio, I see that my regular app is starting as well. I see that the onCreate method is fired up inside my Application class (which is really bad since I am loading some additional resources there).
Why is Android Studio/gradle running my app as well? Can I programatically detect if I am inside test or regular configuration? Can I stop my regular app from being booted before running tests?
In addition, when I am running tests in debug mode it doesn't stop on breakpoints placed inside the application's onCreate method. Why is this happening?
Edit:
Body of test class doesn't really matter, it can be something like:
public class SimpleTest extends AndroidTestCase {
public void testSample()
{
assertEquals(true, false);
}
}
Executing only this simple test fires up onCreate method inside application class.
Gradle console prints out:
Executing tasks: [:app:assembleDebug, :app:assembleDebugTest]
I guess that first task creates instance of my Application class - is it expected behavior?
Why is Android Studio/gradle running my app as well?
Can I stop my regular app from being booted before running tests?
AndroidTestCase is extension of JUnit TestCase which is aware of your android application. In case you don't need to test your android application and want to test plain java only you should use JUnit framework. Create regular JUnit tests, do not use android classes there and run JUnit test configuration like this:
You should treat AndroidTestCase as instrumentation tests which will build android app and run that tests on it. This is usefull with combination of Espresso and Robotium. Both are working on top of base android test classes and both will build and run your application before testing it. Real device or emulator is needed.
Use plain JUnit tests or Robolectric to test java on your desktop JVM.
Can I programatically detect if I am inside test or regular configuration?
You can use power of gradle to provide such info with autogenerated BuildConfig file.
At your build.gradle
android {
defaultConfig {
testPackageName "com.foo.test"
}
}
At your code:
BuildConfig.PACKAGE_NAME.equals("com.foo.test")
The AndroidTestCase is an unit test that unfortunately runs on the device (either virtual or real).
I think what you want to have is a UnitTestFramework that runs in the JVM (local on your machine). The TestFramework Robolectric can do this.
I have started a gitHub project to show how to setup the gradle test file and the project structure if you want to have UnitTests and InstrumentationTests side by side. If you want to look its AndroidGradleTests
I've written a unit test that simply extends TestCase and I have the following:
public class MetricParserTests extends TestCase {
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public void testFailure() {
fail("This needs to fail");
}
}
When I run my tests using ant test or adb shell am instrument I get the following results:
... [exec] OK (1 tests) ...
I'd expect to see a failure on the command line.
I believe I know what the issue is. I was able to reproduce the issue and solve it. The command you use does not rebuild and re-install your test project onto a device. When you call ant test it will just execute the tests which are already installed on that device.
What you need to call is the three commands in your test project's directory:
ant debug
ant installd
ant test
Then all tests will be rebuild and re-installed and latest tests will be executed. If you don't call debug and installd, the changes you did to the tests do not get applied.
I haven't had recent experience in Android testing, but here is what I have found...
You can use normal JUnit tests if your code is totally decoupled from Android (see here
for an example). This would run on your JVM using the JUnit runner.
However, if you are trying to run these tests on an Android device (either via ant, or the command line tools) then you need to create a full android test project (See here).
To test "on device" your test cases need to extend one of the Android test classes like ActivityInstrumentationTestCase2<T>
and are run using the InstrumentationTestRunner in the Dalvik VM on the Android device.
Using an IDE or the command-line tools to create a test project should create a sample test for you to work from.
This blog post linked from the comments of the post above is a good source of information, as is the Android Testing Fundamentals doc.
The method testFailure() does not have a #Test annotation. Is that correct?
I'm trying to test a method in a jar library, and was hoping to use Robolectric to do my unit testing, rather than running the tests in the Android emulator. I'm running into a problem though, where Robolectric needs an androidmanifest.xml file that doesn't exist, since I'm building a library...
Is there any way to run Robolectric tests without an app?
Here's what my test case and code under test look like:
public class ObjectUnderTest {
methodUnderTest(View v) {
...
}
}
#RunWith(RobolectricTestRunner.class)
public class Tests {
#Test
public void methodUnderTest_Test() {
...
}
}
When I run the test suite I get a FileNotFoundException from Robolectric looking for androidmanifest.xml. I've tried using the JUnit4 test runner instead, but then I get the "Stub!" exception when I create a View for the argument to methodUnderTest().
Is there a way to do this besides creating a stub application just for the unit tests? Thanks!
It depends which Robolectric you're using.
If you use 2.0 you could try to annotate your test class with #Config(manifest=Config.NONE).
If you use 1.x I think it's doable but will require more effort:
You should folder create structure similar to Android project and create dummy AndroidManiifest.xml inside
You should extend RobolectricTestRunner and pass through constructor path to this fake Android project