Develop Reference

Android espresso console log Build.Serial onFail

I am running test on multiple devices at once using the adb test command. My pseudo shell script looks like this:
for each device
adb -s ${device} shell am instrument -w -e ${classOrPkg} ${androidTestPackage}${test_name} ${main_package}.${flavor}.test/android.support.test.runner.AndroidJUnitRunner &
The problem is when a test fails, I have no information on which device the failure occurred. I could use LogCat but it requires looking up logcat for each device. And also, System.out.println() does not work.
One possible solution I am trying right now is by extending TestWatcher class and overriding the failed() method like this,
public class TestWatcherRule extends TestWatcher {
#Override
protected void failed(Throwable e, Description description) {
Description d = Description.createTestDescription(e.getClass(), "<<<< Failed on Device: " + Build.SERIAL);
super.failed(e, d);
}
}
Implementation:
#Rule
public TestWatcher testWatcher = new TestWatcherRule();
assertThat("My message", true, is(false));
I cannot get the device serial yet on the terminal.
My expected output would be something like this:
com.myapp.mobile.tests.BenefitCardDBTest:
Error in addDeleteCard(com.myapp.mobile.tests.BenefitCardDBTest):
**<<<< Failed on Device: HTC10xwrtxe**
android.support.test.espresso.NoMatchingViewException: No views in hierarchy found matching: with id: com.myapp.mobile.qa:id/drawer_layout

Let's say this is my sample Espresso test:
#RunWith(AndroidJUnit4.class)
#FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SettingsActivityTest {
#Rule
public ActivityTestRule<SettingsActivity> mRule = new ActivityTestRule<>(SettingsActivity.class);
#Test
public void checkIfToolbarIsProperlyDisplayed() throws InterruptedException {
onView(withText(R.string.action_settings)).check(matches(withParent(withId(R.id.toolbar))));
onView(withId(R.id.toolbar)).check(matches(isDisplayed()));
}
}
To run it on multiple devices I'm using Gradle's connectedAndroidTest which extends connectedCheck, so it :
will run on all connected devices in parallel.
From:
http://tools.android.com/tech-docs/new-build-system/user-guide#TOC-Android-tasks
Just go use your terminal or console to go to your project's directory, then use:
./gradlew connectedAndroidTest.
This one is very useful as it would generate HTML test output which allows you to check which method on which device had failed.
It would look like this:
Hope it will help

Running test that was created by Android Studio

When creating a new project in Android Studio, I notice that it automatically creates an /androidTest directory under /src, where there is "ApplicationTest.java" class with the following code:
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
}
I'm guessing this is what Google wants us to use, but after searching for hours, I couldn't figure out how to use this class that was generated for me. Google's official doc seems to only list how to run on Eclipse IDE (not Android Studio), and I couldn't find any code that would let me perform a simple test (say like assertEquals(1,2)). Can someone show me how to write a simple test code using the above default template, and steps on how to run it, preferably from the command line?
EDIT:
I was able to write a simple test that is intended to fail.
ApplicationTest.java in /androidTest/java/path/to/package/
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
#Override
protected void setUp() throws Exception {
createApplication();
}
#SmallTest
public void testMultiply() {
assertEquals("This should not pass ", 50, 49);
}
}
I am able to run this from Android Studio, but I just cannot figure out how to run that from the command line. Any help?

As Jared already said, this is an example setup for an instrumentation test.
I guess, you've already taken a look on this: Testing Fundamentals
Simpliest way to run these tests in android studio is to right click the class and click run.
It is also possible to add tests in your run/debug configurations.
UPDATE:
To run the instrumentation tests on command line, use ./gradlew assembleAndroidTest. This command will run all tests in your src/androidTest (instrumentation test) folder.
As njzk2 mentioned, there also is a ./gradlew assembleTest command. This command is for running all unit tests (which should be placed in the src/test folder). For more information about unit testing in android take a look on this: Android Unit Testing Support
EXAMPLE:
Here an example for an instrumentaion test in android:
#Override
public void setUp() throws Exception {
super.setUp();
InputStream is = getContext().getAssets().open("test.xml");
XmlParser parser = new XmlParser(getContext());
parser.parse(is);
...
}
#MediumTest
public void testSomething() throws Exception {
// test some data your parser extracted from the xml file
...
}
As you can see, i need the context for creating the input stream, therefor i have to go for an instrumentation test. The #MediumTest signals e.g. i'm accessing files from storage (see Test Annotations).

Android studio says "Empty Test Suite" for AndroidTestCase [duplicate]

This question already has answers here:
Why is the Android test runner reporting "Empty test suite"?
(31 answers)
Closed 6 years ago.
I have created an example test case that extends AndroidTestCase. When I run the test case,
it errors out by saying
Running tests
Test running startedTest running failed:
Instrumentation run failed due to 'java.lang.RuntimeException'
Empty test suite.
The test case
import android.test.AndroidTestCase;
import android.test.suitebuilder.annotation.SmallTest;
import static org.junit.Assert.assertEquals;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.lang.Exception;
import java.lang.Override;
public class DateFormatTest extends AndroidTestCase{
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public DateFormatTest(){
super(DateFormatTest.class);
}
#SmallTest
public void testMultiply() {
assertEquals("10 x 5 must be 50", 50, 10*5);
}
}

Since nobody else mentions it: methods in AndroidTestCase subclasses need to be public and have names starting with "test"!
The OP and the answers all got this right but I missed it and got the exact same error.

I got this error when running tests from Android Studio. Turned out I had placed my test cases in the wrong folder. When running Gradle/Android Studio, all Android tests should be in the folder src/instrumentTest/java.
Edit: In Gradle plugin version 0.9 or later the correct name of the folder is androidTest. (http://tools.android.com/tech-docs/new-build-system/migrating_to_09)

I understand that this question is old, and the development tools have changed significantly since this question has been asked.
However, I had a similar issue (in AndroidStudio 2.1.1) and it turned out that it was just the error message that was quite misleading. For me it said:
Test running started
Test running failed: Instrumentation run failed due to 'java.lang.IllegalStateException'
Empty test suite.
(Note: The difference to the original question is in IllegalStateException vs. RuntimeException)
It turned out that this IllegalStateException was actually thrown in the initialization of the application context as can be seen by inspecting the logcat output. As a result, no test-methods were run, and Android Studio writes this somewhat misleading "Empty test suite" error.
I.e. "Empty test suite" can mean what it actually says (you don't have any test methods declared, e.g. because you forgot to annotate them with #Test), but it can also mean that something (like a runtime exception thrown in your application initialization code) prevents the test-runner from reaching any test methods (which seems to be your case).
Check adb-logcat and search for RuntimeExceptions. This will probably find you the root-cause of your problem.

I got this error because one of my test methods didn't include "throws exception" after the method signature. might help somebody

I got the same issue. I was having testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
in my defaultConfig {...}. I have just removed that line, and now it's working fine (The IDE is picking the right runner config on build time).
I hope this will help someone.

My problem was I had default constructor generated by android studio, looked like this
public class SomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public SomeTest(Class<NewsActivity> activityClass) {
super(activityClass);
}
}
and I had to change it to this to get rid of the problem
public class SomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public SomeTest() {
super(ActivityYouWantToTest.class);
}
}

We use the AndroidTestCase class to define that we are testing components which are specific to Android. The main benefit of AndroidTestCase is that it gives us access to the application's Resources such as strings and layouts, etc.
The AndroidTestCase does not require you to overwrite the default constructor as it does not provide a particular Activity as the Context, but rather provides you a general one so you can still call getContext().
In your example, you are not using any Android components, so for you, the following would make sense:
import android.test.suitebuilder.annotation.SmallTest;
import junit.framework.TestCase;
public class DateFormatTest2 extends TestCase {
#SmallTest
public void testMultiply() {
assertEquals("10 x 5 must be 50", 50, 10 * 5);
}
}
Notice the use of TestCase rather than AndroidTestCase.
For AndroidTestCase to be applicable, a test that requires resources would be necessary:
import android.test.AndroidTestCase;
import android.test.suitebuilder.annotation.SmallTest;
public class DateFormatTest extends AndroidTestCase {
#SmallTest
public void testAppTitle() {
assertEquals("MyApp", getContext().getResources().getString(R.string.app_name));
}
}
Here we use the AndroidTestCase because we need to access the application's resources.

This guide might help -
http://www.slideshare.net/tobiaspreuss/how-to-setup-unit-testing-in-android-studio
On the latest gradle (0.9+) the test should be under androidTest dir
Also in your gradle.build:
dependencies {
androidTestCompile 'junit:junit:4.+'
}
also add those under defaultConfig {
testPackageName "test.java.foo"
testInstrumentationRunner "android.test.InstrumentationTestRunner"
}

Did you configure your testRunner in your gradleConfig?
We use different TestRunners for different tests (to speed things up. My config looks like this
android {
// Some stuff
defaultConfig {
// Some other stuff
//junit test
testInstrumentationRunner "de.qabel.qabelbox.QblJUnitRunner"
//ui tests
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}
If i disable one of this lines the corresponding test will also report "Empty Test Suite".

I just started learning about testing Android applications and I've been struggling with the same problem. You need to provide default constructor for your test class, for example:
package nilzor.myapp.tests;
public class NilzorSomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public NilzorSomeTest(){
super(ActivityYouWantToTest.class);
}
#SmallTest
public void testBlah(){
assertEquals(1,1);
}
}

I already have a default constructor in my test case but still it was giving me error "Empty test suite" and was stuck at "Instantiating tests...".
Tried creating new workspace, resetting Android Studio, but that didn't work.
Finally, close Android SDK and emulator.
Go to your android-sdks/platform-tools.
Clear all Android temp files with these commands:
a. rm -rf ~/Library/Application Support/AndroidStudio
b. rm -rf ~/Library/Caches/AndroidStudio
c. rm -rf ~/Library/Application Support/AndroidStudio
d. rm -rf ~/Library/Preferences/AndroidStudio
Run:
./adb kill-server
./adb start-server
Start Android and run test case.

Randomise the order of instrumentation tests

I'm wondering if it's possible to randomise the order in which instrumentation tests are run, i.e. those extending ActivityInstrumentationTestCase2. I tried following this blog post, but I can't work out how to tell the testing framework that I wish to use my test runner.
The problem is that I can't use the #RunWith annotation, as these are (as I understand it) JUnit3 tests, rather than JUnit4.
It's quite possible that this is pointless, as they don't need to be randomised, but it would be nice to prove the tests' independence in this way.
Ideally I'd like to get it running first using the command line and the gradle wrapper.
Then, it would be nice to have it working via Android Studio, if possible.
[Edit]
I can see that when you do "Edit Configurations . . ." in AS, it's possible to specify your own runner there, via the "Specific instrumentation runner (optional)" box. Unfortunately if I do that, I get the following error:
Test running startedTest running failed: Unable to find instrumentation info for: ComponentInfo{<path_to_class_here>.RandomizingTestRunner}
Empty test suite.
And I can't work out why.

You could use the following randomized runner:
package com.example.test.runners;
import android.test.InstrumentationTestRunner;
import android.test.suitebuilder.TestSuiteBuilder;
import junit.framework.Test;
import junit.framework.TestSuite;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class RandomizedInstrumentationTestRunner extends InstrumentationTestRunner {
#Override
public TestSuite getTestSuite() {
return buildTestSuite();
}
private TestSuite buildTestSuite() {
TestSuiteBuilder builder = new TestSuiteBuilder(getClass().getName(), getTargetContext().getClassLoader());
builder.includePackages("");
List<Test> tests = new ArrayList<Test>();
addTestsFromSuite(builder.build(), tests);
Collections.shuffle(tests);
TestSuite randomizedSuite = new TestSuite();
for (Test one : tests) {
randomizedSuite.addTest(one);
}
return randomizedSuite;
}
private void addTestsFromSuite(TestSuite suite, List<Test> out) {
List<Test> tests = Collections.list(suite.tests());
for (Test one : tests) {
if (one instanceof TestSuite) {
addTestsFromSuite((TestSuite) one, out);
}
else{
out.add(one);
}
}
}
}
and don't forget to set the runner in your build.gradle file:
android {
defaultConfig {
testInstrumentationRunner "com.example.test.runners.RandomizedInstrumentationTestRunner"
minSdkVersion 8
}
....
}
Finally run the following twice to verify the random order of execution:
./gradlew connectedCheck --info

Simple Java class test with Android SDK

I would like to know how to test a simple Java class in an Android project which do not use any android SDK code, using the (JUnit based) test framework included with the android sdk.
Or if I would have to use an external (JUnit) test library?
Regards.

Thanks All, I get a clue with this link https://marakana.com/tutorials/android/junit-test-example.html, but i had to tweak it a little bit to fit my needs.
So if one want to write tests for a simple class in an android project, using the android test framework, let's say a Calculator class.
package com.example.application;
public class Calculator {
public static int add(int a, int b){
return a+b;
}
public static int sub(int a, int b){
return a-b;
}
}
You will have to create the following test class in your test project:
package com.example.application.tests;
import android.test.AndroidTestCase;
import com.example.application.Calculator;
public class CalculatorTest{
public void testAdd() throws Throwable{
assertEquals(3, Calculator.add(1,2);
}
public void testSub() throws Throwable{
assertEquals(-1, Calculator.add(1,2);
}
}
And with the command line, after compiling and installing both the main project and the test project, on the emulator (for instance), you can use the following command to test the class:
adb shell am instrument -w -e class com.example.appplication.tests.CalculatorTest com.example.application.tests/android.test.InstrumentationTestRunner
You can also use Eclipse to run the tests.
Hope this will help someone.

Use the test framework from your android sdk.
That way you can test your android code too if you want to do that later.
Testing Android