Develop Reference

Failing to run Instrumented tests on a new Android Kotlin project

I just created a new android application with Kotlin Support.
When I've tried to run the default instrumented tests it does not run and shows me this message:
Class not found: "oussaki.com.pos.ExampleInstrumentedTest"Empty test suite.
This the Instrumented test class that I'm trying to run:
#RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {
#Test
fun useAppContext() {
// Context of the app under test.
val appContext = InstrumentationRegistry.getTargetContext()
assertEquals("oussaki.com.pos", appContext.packageName)
}
}

This is a known issue: https://issuetracker.google.com/issues/38452937 which hopefully will be fixed in the next release.
For the time being you can manually go to 'Edit Configurations' and add the configuration for the specific class/method you want to run under 'Android Instrumented Tests'.
You could also try the latest canary build: https://developer.android.com/studio/preview/index.html but personally I've had trouble getting it to work with my project.

Espresso Instrumentation Tests - how to uninstall or delete the app after the test

I setup Espresso instrumentation framework to run my Android Functional Automation tests.
For every test, I want to login to the app and delete the app after I finish the test.
So, I setup something like below:
public class FirstSampleTest extends BaseTest {
private final BaseTest baseTest;
// private final ElementUtils elementUtils;
public FirstSampleTest() throws InterruptedException {
this.baseTest = new BaseTest();
}
#Before
public void initiate() throws InterruptedException {
//I have setup login method here to login to the app after it installs
}
#Rule
public ActivityTestRule<SplashScreenActivity> splashScreenActivityActivityTestRule = new ActivityTestRule(SplashScreenActivity.class);
#Test
public void testTheHomeScreen() throws InterruptedException {
//Some tests go here.
}
#After
public void teardown() throws InterruptedException {
//I want to uninstall the app or delete it from the emulator once the test is run
}
}

You can add a gradle task in the Android Studio Before launch section in Run -> Edit Configurations.
Click + -> Add a gradle-aware Make -> :app:uninstallAll
note: "app" in :app:uninstallAll depends on your main module name. So it can be :my_module:uninstallAll, or :company:uninstallAll

Uninstalling the app from the Instrumentation tests is not possible. However, once all the tests are run, the app is uninstalled automatically.
Note: The app is not uninstalled only when a single test is run. Please run the whole build using the command
./gradlew connectedAndroidTest

Running test that was created by Android Studio

When creating a new project in Android Studio, I notice that it automatically creates an /androidTest directory under /src, where there is "ApplicationTest.java" class with the following code:
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
}
I'm guessing this is what Google wants us to use, but after searching for hours, I couldn't figure out how to use this class that was generated for me. Google's official doc seems to only list how to run on Eclipse IDE (not Android Studio), and I couldn't find any code that would let me perform a simple test (say like assertEquals(1,2)). Can someone show me how to write a simple test code using the above default template, and steps on how to run it, preferably from the command line?
EDIT:
I was able to write a simple test that is intended to fail.
ApplicationTest.java in /androidTest/java/path/to/package/
public class ApplicationTest extends ApplicationTestCase<Application> {
public ApplicationTest() {
super(Application.class);
}
#Override
protected void setUp() throws Exception {
createApplication();
}
#SmallTest
public void testMultiply() {
assertEquals("This should not pass ", 50, 49);
}
}
I am able to run this from Android Studio, but I just cannot figure out how to run that from the command line. Any help?

As Jared already said, this is an example setup for an instrumentation test.
I guess, you've already taken a look on this: Testing Fundamentals
Simpliest way to run these tests in android studio is to right click the class and click run.
It is also possible to add tests in your run/debug configurations.
UPDATE:
To run the instrumentation tests on command line, use ./gradlew assembleAndroidTest. This command will run all tests in your src/androidTest (instrumentation test) folder.
As njzk2 mentioned, there also is a ./gradlew assembleTest command. This command is for running all unit tests (which should be placed in the src/test folder). For more information about unit testing in android take a look on this: Android Unit Testing Support
EXAMPLE:
Here an example for an instrumentaion test in android:
#Override
public void setUp() throws Exception {
super.setUp();
InputStream is = getContext().getAssets().open("test.xml");
XmlParser parser = new XmlParser(getContext());
parser.parse(is);
...
}
#MediumTest
public void testSomething() throws Exception {
// test some data your parser extracted from the xml file
...
}
As you can see, i need the context for creating the input stream, therefor i have to go for an instrumentation test. The #MediumTest signals e.g. i'm accessing files from storage (see Test Annotations).

Android Tests does not recognize my test method

I am trying to add some unit tests to my android app. Following is what I tried.
Created a directory for my tests and created a package for test classes inside it.
Created following test class in added package
public class MyFirstTest extends TestCase {
#Override
protected void setUp() throws Exception {
super.setUp();
}
#SmallTest
public void basicTest() {
assertEquals("abc", "abc");
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
}
Created a new Android Tests build configuration in Android Studio
Specified module in build configuration settings and choose to run test cases of my test class (specified class name MyFirstTest in build configuration settings)
But, when I run my build configuration, it says that no tests were found in MyFirstTest class.
junit.framework.AssertionFailedError: No tests found in my.package.tests.MyFirstTest
What should I do to make basicTest() method to get identified as a test case?

You must name your tests testSomething(), that is starting with test

Randomise the order of instrumentation tests

I'm wondering if it's possible to randomise the order in which instrumentation tests are run, i.e. those extending ActivityInstrumentationTestCase2. I tried following this blog post, but I can't work out how to tell the testing framework that I wish to use my test runner.
The problem is that I can't use the #RunWith annotation, as these are (as I understand it) JUnit3 tests, rather than JUnit4.
It's quite possible that this is pointless, as they don't need to be randomised, but it would be nice to prove the tests' independence in this way.
Ideally I'd like to get it running first using the command line and the gradle wrapper.
Then, it would be nice to have it working via Android Studio, if possible.
[Edit]
I can see that when you do "Edit Configurations . . ." in AS, it's possible to specify your own runner there, via the "Specific instrumentation runner (optional)" box. Unfortunately if I do that, I get the following error:
Test running startedTest running failed: Unable to find instrumentation info for: ComponentInfo{<path_to_class_here>.RandomizingTestRunner}
Empty test suite.
And I can't work out why.

You could use the following randomized runner:
package com.example.test.runners;
import android.test.InstrumentationTestRunner;
import android.test.suitebuilder.TestSuiteBuilder;
import junit.framework.Test;
import junit.framework.TestSuite;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class RandomizedInstrumentationTestRunner extends InstrumentationTestRunner {
#Override
public TestSuite getTestSuite() {
return buildTestSuite();
}
private TestSuite buildTestSuite() {
TestSuiteBuilder builder = new TestSuiteBuilder(getClass().getName(), getTargetContext().getClassLoader());
builder.includePackages("");
List<Test> tests = new ArrayList<Test>();
addTestsFromSuite(builder.build(), tests);
Collections.shuffle(tests);
TestSuite randomizedSuite = new TestSuite();
for (Test one : tests) {
randomizedSuite.addTest(one);
}
return randomizedSuite;
}
private void addTestsFromSuite(TestSuite suite, List<Test> out) {
List<Test> tests = Collections.list(suite.tests());
for (Test one : tests) {
if (one instanceof TestSuite) {
addTestsFromSuite((TestSuite) one, out);
}
else{
out.add(one);
}
}
}
}
and don't forget to set the runner in your build.gradle file:
android {
defaultConfig {
testInstrumentationRunner "com.example.test.runners.RandomizedInstrumentationTestRunner"
minSdkVersion 8
}
....
}
Finally run the following twice to verify the random order of execution:
./gradlew connectedCheck --info