This question already has answers here:
Why is the Android test runner reporting "Empty test suite"?
(31 answers)
Closed 6 years ago.
I have created an example test case that extends AndroidTestCase. When I run the test case,
it errors out by saying
Running tests
Test running startedTest running failed:
Instrumentation run failed due to 'java.lang.RuntimeException'
Empty test suite.
The test case
import android.test.AndroidTestCase;
import android.test.suitebuilder.annotation.SmallTest;
import static org.junit.Assert.assertEquals;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.lang.Exception;
import java.lang.Override;
public class DateFormatTest extends AndroidTestCase{
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public DateFormatTest(){
super(DateFormatTest.class);
}
#SmallTest
public void testMultiply() {
assertEquals("10 x 5 must be 50", 50, 10*5);
}
}
Since nobody else mentions it: methods in AndroidTestCase subclasses need to be public and have names starting with "test"!
The OP and the answers all got this right but I missed it and got the exact same error.
I got this error when running tests from Android Studio. Turned out I had placed my test cases in the wrong folder. When running Gradle/Android Studio, all Android tests should be in the folder src/instrumentTest/java.
Edit: In Gradle plugin version 0.9 or later the correct name of the folder is androidTest. (http://tools.android.com/tech-docs/new-build-system/migrating_to_09)
I understand that this question is old, and the development tools have changed significantly since this question has been asked.
However, I had a similar issue (in AndroidStudio 2.1.1) and it turned out that it was just the error message that was quite misleading. For me it said:
Test running started
Test running failed: Instrumentation run failed due to 'java.lang.IllegalStateException'
Empty test suite.
(Note: The difference to the original question is in IllegalStateException vs. RuntimeException)
It turned out that this IllegalStateException was actually thrown in the initialization of the application context as can be seen by inspecting the logcat output. As a result, no test-methods were run, and Android Studio writes this somewhat misleading "Empty test suite" error.
I.e. "Empty test suite" can mean what it actually says (you don't have any test methods declared, e.g. because you forgot to annotate them with #Test), but it can also mean that something (like a runtime exception thrown in your application initialization code) prevents the test-runner from reaching any test methods (which seems to be your case).
Check adb-logcat and search for RuntimeExceptions. This will probably find you the root-cause of your problem.
I got this error because one of my test methods didn't include "throws exception" after the method signature. might help somebody
I got the same issue. I was having testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
in my defaultConfig {...}. I have just removed that line, and now it's working fine (The IDE is picking the right runner config on build time).
I hope this will help someone.
My problem was I had default constructor generated by android studio, looked like this
public class SomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public SomeTest(Class<NewsActivity> activityClass) {
super(activityClass);
}
}
and I had to change it to this to get rid of the problem
public class SomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public SomeTest() {
super(ActivityYouWantToTest.class);
}
}
We use the AndroidTestCase class to define that we are testing components which are specific to Android. The main benefit of AndroidTestCase is that it gives us access to the application's Resources such as strings and layouts, etc.
The AndroidTestCase does not require you to overwrite the default constructor as it does not provide a particular Activity as the Context, but rather provides you a general one so you can still call getContext().
In your example, you are not using any Android components, so for you, the following would make sense:
import android.test.suitebuilder.annotation.SmallTest;
import junit.framework.TestCase;
public class DateFormatTest2 extends TestCase {
#SmallTest
public void testMultiply() {
assertEquals("10 x 5 must be 50", 50, 10 * 5);
}
}
Notice the use of TestCase rather than AndroidTestCase.
For AndroidTestCase to be applicable, a test that requires resources would be necessary:
import android.test.AndroidTestCase;
import android.test.suitebuilder.annotation.SmallTest;
public class DateFormatTest extends AndroidTestCase {
#SmallTest
public void testAppTitle() {
assertEquals("MyApp", getContext().getResources().getString(R.string.app_name));
}
}
Here we use the AndroidTestCase because we need to access the application's resources.
This guide might help -
http://www.slideshare.net/tobiaspreuss/how-to-setup-unit-testing-in-android-studio
On the latest gradle (0.9+) the test should be under androidTest dir
Also in your gradle.build:
dependencies {
androidTestCompile 'junit:junit:4.+'
}
also add those under defaultConfig {
testPackageName "test.java.foo"
testInstrumentationRunner "android.test.InstrumentationTestRunner"
}
Did you configure your testRunner in your gradleConfig?
We use different TestRunners for different tests (to speed things up. My config looks like this
android {
// Some stuff
defaultConfig {
// Some other stuff
//junit test
testInstrumentationRunner "de.qabel.qabelbox.QblJUnitRunner"
//ui tests
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}
If i disable one of this lines the corresponding test will also report "Empty Test Suite".
I just started learning about testing Android applications and I've been struggling with the same problem. You need to provide default constructor for your test class, for example:
package nilzor.myapp.tests;
public class NilzorSomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public NilzorSomeTest(){
super(ActivityYouWantToTest.class);
}
#SmallTest
public void testBlah(){
assertEquals(1,1);
}
}
I already have a default constructor in my test case but still it was giving me error "Empty test suite" and was stuck at "Instantiating tests...".
Tried creating new workspace, resetting Android Studio, but that didn't work.
Finally, close Android SDK and emulator.
Go to your android-sdks/platform-tools.
Clear all Android temp files with these commands:
a. rm -rf ~/Library/Application Support/AndroidStudio
b. rm -rf ~/Library/Caches/AndroidStudio
c. rm -rf ~/Library/Application Support/AndroidStudio
d. rm -rf ~/Library/Preferences/AndroidStudio
Run:
./adb kill-server
./adb start-server
Start Android and run test case.
Related
When creating a new project in Android Studio, I notice that it automatically creates an /androidTest directory under /src, where there is "ApplicationTest.java" class with the following code:
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
}
I'm guessing this is what Google wants us to use, but after searching for hours, I couldn't figure out how to use this class that was generated for me. Google's official doc seems to only list how to run on Eclipse IDE (not Android Studio), and I couldn't find any code that would let me perform a simple test (say like assertEquals(1,2)). Can someone show me how to write a simple test code using the above default template, and steps on how to run it, preferably from the command line?
EDIT:
I was able to write a simple test that is intended to fail.
ApplicationTest.java in /androidTest/java/path/to/package/
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
#Override
protected void setUp() throws Exception {
createApplication();
}
#SmallTest
public void testMultiply() {
assertEquals("This should not pass ", 50, 49);
}
}
I am able to run this from Android Studio, but I just cannot figure out how to run that from the command line. Any help?
As Jared already said, this is an example setup for an instrumentation test.
I guess, you've already taken a look on this: Testing Fundamentals
Simpliest way to run these tests in android studio is to right click the class and click run.
It is also possible to add tests in your run/debug configurations.
UPDATE:
To run the instrumentation tests on command line, use ./gradlew assembleAndroidTest. This command will run all tests in your src/androidTest (instrumentation test) folder.
As njzk2 mentioned, there also is a ./gradlew assembleTest command. This command is for running all unit tests (which should be placed in the src/test folder). For more information about unit testing in android take a look on this: Android Unit Testing Support
EXAMPLE:
Here an example for an instrumentaion test in android:
#Override
public void setUp() throws Exception {
super.setUp();
InputStream is = getContext().getAssets().open("test.xml");
XmlParser parser = new XmlParser(getContext());
parser.parse(is);
...
}
#MediumTest
public void testSomething() throws Exception {
// test some data your parser extracted from the xml file
...
}
As you can see, i need the context for creating the input stream, therefor i have to go for an instrumentation test. The #MediumTest signals e.g. i'm accessing files from storage (see Test Annotations).
I'm working on a uiautomator project recently and then the UiObject.getFromParent turn out a wrong element to me and I look into the source code of uiautomator and found out the answer is because by the UiSelector I used.
I found that the uiautomator is using the Instrumentation to get the UI element stuff just like :
getInstrumentation().getUiAutomation().getRootInActiveWindow();
I just want to get a AccessibilityNodeInfo node just like what uiautomator do but the uiautomator didn't exposed this.
I’m trying this way by a new class extends InstrumentationTestCase,by the getInstrumentation() always return a null to me.
i found an answer on
android instrumentation test case - getinstrumentation() returning null
that needs injectInstrumentation(InstrumentationRegistry.getInstrumentation());
and told InstrumentationRegistry is from the official Android testing-support-lib:0.1
I have download the Android Support Repository and import the testing-support-lib-0.1-source.jar into my project but I still can't see InstrumentationRegistry.
Anyone have any idea about my cast?
I'm using an extension of InstrumentationTestCase and it works just fine like this:
#Override
public void setUp() throws Exception{
super.setUp();
Instrumentation instrumentation = InstrumentationRegistry.getInstrumentation();
AccessibilityNodeInfo root = instrumentation.getUiAutomation().getRootInActiveWindow();
}
I imported
import android.test.InstrumentationTestCase;
import android.support.test.InstrumentationRegistry;
import android.view.accessibility.AccessibilityNodeInfo;
and installed
Android Support Repository (15)
Android Support Library (22.2)
Hope it helps.
well,this question has beep solved by
unpack testing-support-lib-0.1.aar and get the classes.jar instead of
testing-support-lib-0.1-source.jar
.and now the eclipse will not saying can't resolved anymore.By the way,u have to
add this lib to Ant's javac classpath by edit build.xml
or else u will get a lib not found while u trying ant build.
but now the NEW question has turns out
INSTRUMENTATION_STATUS: stack=java.lang.NoClassDefFoundError: Failed resolution of: Landroid/support/test/InstrumentationRegistry;
maybe it's another question so i am going to find out via google and i will update this if figure it out or i will post a new question.
Frist tried:
i have noticed that the point of this question is the uiautomator didn't expose the api i want,and when i look into the source and found the UiDevice.dumpWindowHierarchy that has these code:
AccessibilityNodeInfo root =getAutomatorBridge().getQueryController().getAccessibilityRootNode();
the getAutomatorBridge was implemented by UiDevice:
UiAutomatorBridge getAutomatorBridge() {
if (mUiAutomationBridge == null) {
throw new RuntimeException("UiDevice not initialized");
}
return mUiAutomationBridge;
}
the point is there is no modifier on it,so what i have done is modified it's modifier to public ,modified the byte code of
/system/framework/uiautomator.jar
and the other one in sdk(sorry i don't,ya it worked!i can use it like this.
AccessibilityNodeInfo root =getUidevice().getAutomatorBridge().getQueryController().getAccessibilityRootNode();
but there is a compatibility issue isn't it?
Second tried:
Let's think about the default modifier of java,we can access it under the same package,so why don't we just implement a class has the same Package name ?then we can call it without any unnomal patch.
that's what i have done and it really works well:
package com.android.uiautomator.core;
import android.view.Display;
import android.view.accessibility.AccessibilityNodeInfo;
public class AutomationUltilites
{
public AutomationUltilites()
{
}
public static AccessibilityNodeInfo getRootNode()
{
return UiDevice.getInstance().getAutomatorBridge().getRootInActiveWindow();
}
public static Display getDisplay()
{
return UiDevice.getInstance().getAutomatorBridge().getDefaultDisplay();
}
}
Hope it helps.
I want to start writing unit tests for my applications but I cannot get one simple test to run.
I have created a small application just to try how the unit test should be setup and run, but no test is actually run and I get 'Empty test suite'.
I am using Android Studio 0.6.1 with gradle 1.12
Here is my folder structure:
MyActivityTest.java
package com.vist.testableapp.tests;
import android.content.Intent;
import android.test.ActivityUnitTestCase;
import android.test.suitebuilder.annotation.SmallTest;
import android.widget.Button;
import com.vist.testableapp.MyActivity;
import com.vist.testableapp.R;
public class MyActivityTest extends ActivityUnitTestCase<MyActivity>
{
public MyActivityTest(Class<MyActivity> activityClass)
{
super(activityClass);
}
Button btn1;
#Override
public void setUp() throws Exception
{
super.setUp();
startActivity(new Intent(getInstrumentation().getTargetContext(), MyActivity.class), null, null);
btn1 = (Button)getActivity().findViewById(R.id.button1);
}
#SmallTest
public void testFirst()
{
assertEquals("Btn1",btn1.getText());
}
}
application's build.gradle
apply plugin: 'android'
android {
compileSdkVersion 19
buildToolsVersion "19.1.0"
defaultConfig {
applicationId "com.vist.testableapp"
minSdkVersion 15
targetSdkVersion 15
versionCode 1
versionName "1.0"
testApplicationId "com.vist.testableapp.tests"
}
buildTypes {
release {
runProguard false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
}
dependencies {
compile fileTree(dir: 'libs', include: ['*.jar'])
}
Could anyone point out what am I doing wrong or what I am missing?
I searched in SO but none of the answers helped me.
The constructor should look like this:
public MyActivityTest()
{
super(MyActivity.class);
}
I will need to learn not to rely so much on the IDE's code template that provided constructor with parameter.
This was resolved thanks to a colleague and http://siemprepreguntando.blogspot.de/2013/07/running-tests-test-running-startedtest.html
I also ran into "empty test suite" problem recently. After checking a few similar questions and answers, and my problem, I can possibly conclude that the problem results from an error preventing the tests being added to the test suite, such as an error in static initialization.
For example I'm using a popular approach to adding all tests as shown below, but it's the same scenario with different approaches to adding test cases to the suite:
public class FullTestSuite extends TestSuite {
public static Test suite() {
return new TestSuiteBuilder(FullTestSuite.class)
.includeAllPackagesUnderHere().build();
}
public FullTestSuite() {
super();
}
}
And apparently my test file had a problem in a static {} block which prevented .includeAllPackagesUnderHere() to run successfully.
So I would suggest anyone facing this error to first check your app logs to see if your test runs into a problem that prevents test cases being added to the test suite (like similar examples of wrong constructor being called or static initialization problems).
In my case, the "empty test suite" message was directly related to the target API Level of the Android Emulator I was running. I had set up an emulator with API level 19 and was using that while trying to run my instrumentation tests. I also had just recently migrated my codebase to use the JUnit4 framework along with the AndroidJUnitRunner instrumentation runner.
I was banging my head against the wall for a while before I started to look into issues with the actual emulator. Sure enough, as soon as I set up an emulator with API Level 23, the tests started to run fine.
Further experimentation found that my tests ran fine on API Level 22 & 23 emulators, but not on anything below that. I suspect it has something to do with my test dependencies and minimum API level requirements.
I'll update this answer if I discover more.
I'm wondering if it's possible to randomise the order in which instrumentation tests are run, i.e. those extending ActivityInstrumentationTestCase2. I tried following this blog post, but I can't work out how to tell the testing framework that I wish to use my test runner.
The problem is that I can't use the #RunWith annotation, as these are (as I understand it) JUnit3 tests, rather than JUnit4.
It's quite possible that this is pointless, as they don't need to be randomised, but it would be nice to prove the tests' independence in this way.
Ideally I'd like to get it running first using the command line and the gradle wrapper.
Then, it would be nice to have it working via Android Studio, if possible.
[Edit]
I can see that when you do "Edit Configurations . . ." in AS, it's possible to specify your own runner there, via the "Specific instrumentation runner (optional)" box. Unfortunately if I do that, I get the following error:
Test running startedTest running failed: Unable to find instrumentation info for: ComponentInfo{<path_to_class_here>.RandomizingTestRunner}
Empty test suite.
And I can't work out why.
You could use the following randomized runner:
package com.example.test.runners;
import android.test.InstrumentationTestRunner;
import android.test.suitebuilder.TestSuiteBuilder;
import junit.framework.Test;
import junit.framework.TestSuite;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class RandomizedInstrumentationTestRunner extends InstrumentationTestRunner {
#Override
public TestSuite getTestSuite() {
return buildTestSuite();
}
private TestSuite buildTestSuite() {
TestSuiteBuilder builder = new TestSuiteBuilder(getClass().getName(), getTargetContext().getClassLoader());
builder.includePackages("");
List<Test> tests = new ArrayList<Test>();
addTestsFromSuite(builder.build(), tests);
Collections.shuffle(tests);
TestSuite randomizedSuite = new TestSuite();
for (Test one : tests) {
randomizedSuite.addTest(one);
}
return randomizedSuite;
}
private void addTestsFromSuite(TestSuite suite, List<Test> out) {
List<Test> tests = Collections.list(suite.tests());
for (Test one : tests) {
if (one instanceof TestSuite) {
addTestsFromSuite((TestSuite) one, out);
}
else{
out.add(one);
}
}
}
}
and don't forget to set the runner in your build.gradle file:
android {
defaultConfig {
testInstrumentationRunner "com.example.test.runners.RandomizedInstrumentationTestRunner"
minSdkVersion 8
}
....
}
Finally run the following twice to verify the random order of execution:
./gradlew connectedCheck --info
I've written a unit test that simply extends TestCase and I have the following:
public class MetricParserTests extends TestCase {
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public void testFailure() {
fail("This needs to fail");
}
}
When I run my tests using ant test or adb shell am instrument I get the following results:
... [exec] OK (1 tests) ...
I'd expect to see a failure on the command line.
I believe I know what the issue is. I was able to reproduce the issue and solve it. The command you use does not rebuild and re-install your test project onto a device. When you call ant test it will just execute the tests which are already installed on that device.
What you need to call is the three commands in your test project's directory:
ant debug
ant installd
ant test
Then all tests will be rebuild and re-installed and latest tests will be executed. If you don't call debug and installd, the changes you did to the tests do not get applied.
I haven't had recent experience in Android testing, but here is what I have found...
You can use normal JUnit tests if your code is totally decoupled from Android (see here
for an example). This would run on your JVM using the JUnit runner.
However, if you are trying to run these tests on an Android device (either via ant, or the command line tools) then you need to create a full android test project (See here).
To test "on device" your test cases need to extend one of the Android test classes like ActivityInstrumentationTestCase2<T>
and are run using the InstrumentationTestRunner in the Dalvik VM on the Android device.
Using an IDE or the command-line tools to create a test project should create a sample test for you to work from.
This blog post linked from the comments of the post above is a good source of information, as is the Android Testing Fundamentals doc.
The method testFailure() does not have a #Test annotation. Is that correct?