I have one test case file with around 20 methods (test cases) which extends ActivityInstrumentationTestCase2. I need to write a suite which will call only selected test case methods, I know in junit there is one method which accepts the methods to be executed
suite.addTest( new AllTestCases("testcase1"));
Is there a similar way to do stuff in android robotium? If yes, please help me out with a way to fix this. Thanks.
You can't make a call like new AllTestCases("testcase1"); because all Android related test classes inherit from either AndroidTestCase or InstrumentationTestCase and neither of these classes expose a constructor that takes a string as an argument.
You could take a look at android.test.suitebuilder.TestSuiteBuilder but even this class does not allow for the running of individual test methods, it accepts tests at the package level.
You might have some luck achieving your goal by using the Android test annotations such as #SmallTest, #MediumTest, #LargeTest etc. These will allow you to target only the specified annotated methods using the follwing command:
adb shell am instrument -w -e size <small|medium|large> com.youproject.test/android.test.InstrumentationTestRunner
Finally, its possible to target individual tests methods or classes directly from within eclipse.
To run an individual test case directly from command line:
adb shell am instrument -w -e class <Test-Class-With-Package-Name>#<Test-Method-Name> <Package-Name-Of-Test-App>/<Instrumentation-Name-Defined-In-Manifest>
Example:
adb shell am instrument -w -e class com.myapp.test.ActivityFragmentTest#testLogin com.myapp.test/android.test.InstrumentationTestRunner
You can run individual test cases programmatically with "-e" arguments to the "adb shell am instrument" command. For example, for a method 'testFoo()' in 'com.foo.bar.FooTest' you could run:
adb shell am instrument -w \
-e "class com.foo.bar.FooTest#testFoo" \
com.foo.bar.test/android.test.InstrumentationTestRunner
http://developer.android.com/guide/developing/testing/testing_otheride.html
Related
From what i read from these docs we can annotate any flakytests and then there should be a way to filter them out and run them alone.I was thinking firebase robo tests will know about this and re -tests the flaky ones only, but the following statements have confused me :
Can then be used to filter tests on execution using -e annotation or -e notAnnotation as desired.
What is this switch -e ? How can i filter tests ? the comment leads me to confusion on how to fitler tests on execution. is it done on the gradle command line ? Can i get an example ?
enter code hereI finally found out how to do this. When you put annotations on test methods you can run a group of methods that have the same annotation. Reading the docs we find out how to do this. so if i were to mark many tests as #FlakyTest then i can run all of the FlakyTest with adb like this:
adb shell am instrument -w -e annotation android.support.test.filters.FlakyTest
Here is an the part of the AndroidJUnitRunner docs explaining this:
Filter test run to tests with given annotation: adb shell am instrument -w -e annotation com.android.foo.MyAnnotation com.android.foo/android.support.test.runner.AndroidJUnitRunner
I'm experimenting with test sharding on Android and I'm getting pretty weird results:
+ adb -s emulator-5580 shell am instrument -e numShards 2 -e shardIndex 0 -e class com.package.etc.automation.Tests.SanityTest.SanityTest -w com.package.etc.test/android.support.test.runner.AndroidJUnitRunner
com.package.etc.automation.Tests.SanityTest.SanityTest:..........
Time: 306.578
OK (10 tests)
+ adb -s emulator-5582 shell am instrument -e numShards 2 -e shardIndex 1 -e class com.package.etc.automation.Tests.SanityTest.SanityTest -w com.package.etc.test/android.support.test.runner.AndroidJUnitRunner
com.package.etc.automation.Tests.SanityTest.SanityTest:......................
Time: 645.723
OK (22 tests)
As you can see, adb split the tests into two uneven groups. The second one has twice as many tests as the first one and executes twice as long. Not the best parallelism if you ask me.
Is there a possibility to control the distribution of tests, or at least force adb to split the tests evenly?
Let's trace it down.
When test suite is started, TestRequestBuilder is built upon JUnit Filters. ShardingFilter is one of them and is added. Adding it means that previously added Filter is "intersected" with new one - method public boolean shouldRun(Description description) is invoked. If you look at it, more likely at this fragment:
if (description.isTest()) {
return (Math.abs(description.hashCode()) % mNumShards) == mShardIndex;
}
And substituting with your numbers (numShards=2), you'll notice, that this is just a parity test. Statistically it might happen, that generated HashCode parity distribution is not 50%. Moreover, when some of the tests on your test class are ignored, disabled and are interweaving with enabled tests, you can even more disturb particular method hashcode (Junit Description uniqueId is generated from method and class name).
That is just statistics matter. As you might see in this answer:
How the groups are divided is arbitrary
I have a android test project setup to test my android project. In the android test project's manifest, I have an entry for instrumentation. It looks like:
<instrumentation
android:name="android.test.InstrumentationTestRunner"
android:targetPackage="com.company.android" />
I'm curious what the point of this entry is, and particuarly what the purpose of android:targetPackage="com.company.android" is for. I ask, because I refactored the old project and put classes into different packages, so I'm curious on what I need to update this value to... is it suppose to point the the package where the class that extends android.app.Application is at?
It tells the build system where to access the actual project you are going to test.
This is necessary because you need access to all your Activities and classes without having extra copies around.
Info about it is scattered around in: http://developer.android.com/tools/testing/testing_android.html
InstrumentationTestRunner is something you use to write Android unit tests.
From the documentation:
Typical Usage
Write TestCases that perform unit, functional, or performance tests against the classes in your package.
Typically these are subclassed from:
ActivityInstrumentationTestCase2
ActivityUnitTestCase
AndroidTestCase
ApplicationTestCase
InstrumentationTestCase
ProviderTestCase
ServiceTestCase
SingleLaunchActivityTestCase
In an appropriate AndroidManifest.xml, define the this instrumentation with the appropriate android:targetPackage set.
Run the instrumentation using "adb shell am instrument -w", with no optional arguments, to run all tests (except performance tests).
Run the instrumentation using "adb shell am instrument -w", with the argument '-e func true' to run all functional tests. These are tests that derive from InstrumentationTestCase.
Run the instrumentation using "adb shell am instrument -w", with the argument '-e unit true' to run all unit tests. These are tests that do notderive from InstrumentationTestCase (and are not performance tests).
Run the instrumentation using "adb shell am instrument -w", with the argument '-e class' set to run an individual TestCase.
I am using "am instrument" command to run a test case.
How do I pass arguments to testcase and run it using "am instrument" command?
I don't know of any way to pass command line arguments to a test case via am instrument.
What arguments do you need to pass to the test case? Why can't these arguments be defined in the testcase itself or in a resource file?
If you need fine grained control over your test instrumentation, you may want to create your own version of InstrumentationTestRunner.
I've created a test runner extending android.test.InstrumentationTestRunner. I'm looking for a way to define the set of tests to get executed based on a set of configurations.
I thought I may be able to override the below methods to return my custom test suite, however, these are not getting called! Just wondering whats the use of these:
public TestSuite getAllTests ()
public TestSuite getTestSuite ()
Any clues? Any other alternatives I can use to define a custom test suite at runtime?
Thanx
I also dont know... Another solution is to define your own annotation and annotate the test you want to run. Then you can run only the tests with that annotation using:
Filter test run to tests with given annotation: adb shell am instrument -w -e annotation com.android.foo.MyAnnotation com.android.foo/android.test.InstrumentationTestRunner
If used with other options, the resulting test run will contain the union of the two options. e.g. "-e size large -e annotation com.android.foo.MyAnnotation" will run only tests with both the LargeTest and "com.android.foo.MyAnnotation" annotations.
Filter test run to tests without given annotation: adb shell am instrument -w -e notAnnotation com.android.foo.MyAnnotation com.android.foo/android.test.InstrumentationTestRunner