Develop Reference

Android instrumented test no tests found

I am new to Android instrumented unit tests. I have a simple test that looks like this. This is in instrumented tests; before I really write an actual instrumented test, I just want to test a simple one:
#RunWith(AndroidJUnit4.class)
#SmallTest
public class AddressBookKeepingInstrumentedTest {
public static final String TEST_STRING = "This is a string";
public static final long TEST_LONG = 12345678L;
#Test
public void test_simple() {
assertEquals(2,1+1);
}
}
When I run this, I get the following error:
junit.framework.AssertionFailedError: No tests found in com.example.myfirstapp.AddressBookKeepingInstrumentedTest
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1729)
Tests ran to completion.
Gradle build passed with success before this.
Any ideas why this is happening?

Please add the following into your build.gradle and put your test classes into androidTest folder
android {
defaultConfig {
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}

I recently tried the accepted answer, but it gave me a process crash error when trying to run. If you get this you should use:
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
Source: Instrumentation run failed due to 'Process crashed.'

The typical cause of this can be seen as an exception in Logcat.
I had a specific issue regarding an instrumented multidex run on < API 21, I had to both install MultiDex on a subclass of androidx.test.runner.AndroidJUnitRunner, and override:
AndroidJUnitRunner::createTestRequestBuilder
TestRequestBuilder::createClassPathScanner
ClassPathScanner::getClassPathEntries with the implementation as:
Unzip and extract the dex files from the APK
Run dexlib2 to obtain the type names.
Convert from the type names to a readable format: Ljava/lang/String; -> java.lang.string
This is because ClassPathScanner uses dalvik.system.DexFile, which on API 16, only reads APKs, and then only reads classes.dex

Another reason to get this error message is when you have an Android App implemented with Java and test cases implemented with Kotlin. If that is the case, be sure you have the following lines in your gradle file:
apply plugin: 'kotlin-android'
apply plugin: 'kotlin-android-extensions'

This post helped me, just rename your tests methods with "test...", like testMethod()

Make sure you added the correct dependencies in the app's build.gradle file:
testImplementation 'junit:junit:4.13.2'
androidTestImplementation 'androidx.test.ext:junit:1.1.3'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
The Expresso one is required for any instrumented test even if you don't use it.

Android app build throws error for release, debug doesn't

I'm trying to build an Android app that I've been working on (it's a project I adopted so most of the code isn't mine - which isn't helping :P) and I'm running into an issue.
The app builds just fine in debug mode (building and installing it on a device for testing). But when I try to build a release it fails.
This is the error in the gradle console:
Execution failed for task ':app:lintVitalRelease'.
Lint found fatal errors while assembling a release target.
And this is what it says in Messages Gradle Build:
Error:Error: This fragment class should be public ([com.company.appname].fragments.create_dilemma.CreateDilemmaFragment1_2.UploadDialogFragment) [ValidFragment]
This is the offending line:
private class UploadDialogFragment extends DialogFragment implements View.OnClickListener
So I change private to public and then it complains that it should be a static class. Thing is, it looks like this class isn't supposed to be static because AS isn't happy with pretty much any of the code as soon as I make it static.
So I'm left with a few questions:
How is this only an issue for the release build and not for debug?
Is there a way to ignore this 'error' when building a release?
There must be a reason for this error, right? Why is it ignored for debug and not for release? What are the up/downsides of fixing this? Because the app works just fine as far as I can tell so I don't really see the problem..?
PS: My java skills are so-so. I know my way around the language but I have a lot to learn when it comes to knowing what a static class exactly is and what is allowed and what not, why it is(n't), etc. So plz be gentle, I'm trying to learn this stuff :)
Update: As per request here's the relevant part of my build.gradle:
android {
compileSdkVersion 24
buildToolsVersion '24.0.1'
useLibrary 'org.apache.http.legacy'
defaultConfig {
applicationId "[com.pany.appname]"
minSdkVersion 15
targetSdkVersion 22
multiDexEnabled true
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
}
}
productFlavors {
}
}

I'm summarizing some of the answers given in this post to collect them in one. Makes it easier to accept it, too :)
-- WHY THE QUICK FIX IS NOT A REAL FIX --
It's POSSIBLE to have this project build a release apk successfully (see below). But it's probably not a good idea to just do that and not think about it.
As pointed out by X3Btel:
Fragments needs to be public because systems recreates them on orientation change. The other warning is because non static inner classes holds reference to their outter class, hence it creates memmory leak. Best course of action would be move the fragment to his own class. Or make it public and ignore the lint warning (this may create memmory leak but keeping it private may crash the app)
In my case I can keep it non-static and circumvent memory leaks by making sure I finish the fragment whenever its parent activity is finished. It's not pretty but it'll fix the memory leak and I don't have to refactor A LOT of code I didn't write. That said: It apparently is bad practice to have an activity or fragment and declare another activity/fragment as an inner class because of the way the Android lifecycle works.
Here's some more reading on the topic which I found useful (and only found AFTER I posted this question):
Should an internal DialogFragment class be static or not?
-- THE QUICK FIXES --
1) Don't check for lint errors during build (as pointed out by Jay Shan)
Add lintOptions -> checkReleaseBuilds option to build.gradle
android {
// ..
lintOptions {
checkReleaseBuilds false
}
}
2) Check for errors but keep building even when they are found
This is probably a little bit safer than not checking for errors at all because at least you'll get a warning somewhere in the log output.
Add lintOptions -> abortOnError option to build.gradle
android {
// ..
lintOptions {
abortOnError false
}
}
3) Supress the error where it happens
I find this to be the preferred method because you can still use lint for finding other problems AND have it abort when that happens BUT at the same time you can ignore things you've checked.
In my case I had to add #SuppressLint("ValidFragment") before the offending line:
#SuppressLint("ValidFragment")
private class UploadDialogFragment extends DialogFragment implements View.OnClickListener
{
// ..
}
UPDATE 2018/01/04
If you use a recent version of Android Support Library (and its Fragment implementation instead of the OS's) your app will crash (IllegalStateException) if you try to initialize that Fragment. Suppressing the warning will not help you. You'll just have to fix the underlying problem, make the inner class public and static, or move the class to a separate file.

You can put this option in android block of build.gradle section
lintOptions {
checkReleaseBuilds false
}

Jay Shan`s answer should work. But better to understand what is the problem. Fragments needs to be public because systems recreates them on orientation change. The other warning is because non static inner classes holds reference to their outter class, hence it creates memmory leak.
Best course of action would be move the fragment to his own class. Or make it public and ignore the lint warning (this may create memmory leak but keeping it private may crash the app)

How to switch/change testInstrumentationRunner dynamically with gradle

My project has 2 different groups of tests. One group runs only with the default AndroidJUnitRunner the other has to be run with a custom implementation TestRunner extends MonitoringInstrumentation.
Currently I switch the testInstrumentationRunner by editing the build.gradle each time I need to run the other group of tests:
android{
defaultConfig {
//testInstrumentationRunner "my.custom.TestRunner"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}
I know that flavours can have their own testInstrumentationRunner but my current app already has 2 flavourDimensions. Using flavours is actually intended to have different versions of an app. I need 2 versions of the test application, both testing the same app with different testInstrumentationRunners.
I tried to change the testInstrumentationRunner by iterating over all test variants. There are actually multiple testInstrumentationRunner properties:
android.testVariants.all { TestVariant variant ->
//readonly
variant.variantData.variantConfiguration.instrumentationRunner
variant.variantData.variantConfiguration.defaultConfig.testInstrumentationRunner
}
But as soon as android.testVariants is called the build gets configured and all changes are not reflected in the build.
How can I change the testInstrumentationRunner (from a gradle plugin) dynamically?
I'd prefer to have 2 different gradle tasks, each using a different testInstrumentationRunner but testing the same variant. Because I intent to create a gradle plugin the solution should work as plugin too.

Have you considered using console parameter as a switch between two configurations? As simple as that:
android {
defaultConfig {
if (project.ext.has("customRunner")) {
testInstrumentationRunner "my.custom.TestRunner"
} else {
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}
}
And then for example run gradlew aDeb -PcustomRunner if you want to test using custom runner or gradlew aDeb to use default.
I know it's not rocket science but simpler is better, right? You can use it in your plugin too, just obtain the Project object and do the similar thing.

Since the android gradle plugin 1.3 it is possible to create separate test modules. Each of those test modules can have its own testInstrumentationRunner.
For a detailed example see the AndroidTestingBlueprint example project on github.
The solution from #johan-stuyts that got bounty works fine (or at least it did with the android gradle plugin 1.2). But it uses private APIs and creating a separate module is easier and future proof.

I had a similar issue, I used gradle's taskGraph. Based on your statement "My project has 2 different groups of tests." I'm going to assume you have different tasks defined, I'll call them testGroupOne and testGroupTwo:
task testGroupOne{
}
task testGroupTwo{
}
gradle.taskGraph.whenReady {taskGraph ->
if(taskGraph.hasTask(testGroupOne)){
testInstrumentationRunner "my.custom.TestRunner"
} else if (taskGraph.hasTask(testGroupTwo)){
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}
This enables you to set the value of testInstrumentationRunner after configuration but before execution.

This is a partial solution, but maybe this will make things a bit less cumbersome for you: It does not allow you to run the tests with both test runners in the same build. If you want that you would have to clone all task instances of DeviceProviderInstrumentTestTask, which, in my opinion, is too complex and fragile.
This works for me (Gradle 2.4 and Android build tools 1.2.3). It uses internal APIs, so it is possible that this no longer works with the next release of the Android build tools.
You should modify the tasks generated by the Android plug-in after the project has been evaluated. The changes will then be used by the test tasks. The property that actually changes the used test runner is task.testVariantData.variantConfiguration.testedConfig.mergedFlavor.testInstrumentationRunner. Instead of making these changes during the configuration phase (i.e. outside a task), the changes are applied using a task, so your test runner is only used when requested. By forcing the test tasks to run after useMyTestRunner (if the latter is part of the build), the class name of the test runner will have been changed when a test task starts:
project.afterEvaluate {
task useMyTestRunner << {
tasks.withType(com.android.build.gradle.internal.tasks.DeviceProviderInstrumentTestTask.class) { task ->
task.testVariantData.variantConfiguration.testedConfig.mergedFlavor.testInstrumentationRunner = 'com.mycompany.MyTestRunner'
}
}
tasks.withType(com.android.build.gradle.internal.tasks.DeviceProviderInstrumentTestTask.class) { task ->
task.mustRunAfter useMyTestRunner
}
}
You can now run the tests using the default test runner configured for the flavor(s) with:
gradle :myApp:connectedAndroidTest
When you want to run the tests with your test runner use:
gradle :myApp:connectedAndroidTest :myApp:useMyTestRunner
I did not add checks for null for any of the properties retrieved using task.testVariantData.variantConfiguration.testedConfig.mergedFlavor.testInstrumentationRunner. You should add them for robustness. I think at least testedConfig needs attention. See getInstrumentationRunner() at https://android.googlesource.com/platform/tools/build/+/master/builder/src/main/java/com/android/builder/VariantConfiguration.java
You can use an import for com.android.build.gradle.internal.tasks.DeviceProviderInstrumentTestTask at the top of your build file, so you only have to use the simple name of the class.

Android studio says "Empty Test Suite" for AndroidTestCase [duplicate]

This question already has answers here:
Why is the Android test runner reporting "Empty test suite"?
(31 answers)
Closed 6 years ago.
I have created an example test case that extends AndroidTestCase. When I run the test case,
it errors out by saying
Running tests
Test running startedTest running failed:
Instrumentation run failed due to 'java.lang.RuntimeException'
Empty test suite.
The test case
import android.test.AndroidTestCase;
import android.test.suitebuilder.annotation.SmallTest;
import static org.junit.Assert.assertEquals;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.lang.Exception;
import java.lang.Override;
public class DateFormatTest extends AndroidTestCase{
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public DateFormatTest(){
super(DateFormatTest.class);
}
#SmallTest
public void testMultiply() {
assertEquals("10 x 5 must be 50", 50, 10*5);
}
}

Since nobody else mentions it: methods in AndroidTestCase subclasses need to be public and have names starting with "test"!
The OP and the answers all got this right but I missed it and got the exact same error.

I got this error when running tests from Android Studio. Turned out I had placed my test cases in the wrong folder. When running Gradle/Android Studio, all Android tests should be in the folder src/instrumentTest/java.
Edit: In Gradle plugin version 0.9 or later the correct name of the folder is androidTest. (http://tools.android.com/tech-docs/new-build-system/migrating_to_09)

I understand that this question is old, and the development tools have changed significantly since this question has been asked.
However, I had a similar issue (in AndroidStudio 2.1.1) and it turned out that it was just the error message that was quite misleading. For me it said:
Test running started
Test running failed: Instrumentation run failed due to 'java.lang.IllegalStateException'
Empty test suite.
(Note: The difference to the original question is in IllegalStateException vs. RuntimeException)
It turned out that this IllegalStateException was actually thrown in the initialization of the application context as can be seen by inspecting the logcat output. As a result, no test-methods were run, and Android Studio writes this somewhat misleading "Empty test suite" error.
I.e. "Empty test suite" can mean what it actually says (you don't have any test methods declared, e.g. because you forgot to annotate them with #Test), but it can also mean that something (like a runtime exception thrown in your application initialization code) prevents the test-runner from reaching any test methods (which seems to be your case).
Check adb-logcat and search for RuntimeExceptions. This will probably find you the root-cause of your problem.

I got this error because one of my test methods didn't include "throws exception" after the method signature. might help somebody

I got the same issue. I was having testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
in my defaultConfig {...}. I have just removed that line, and now it's working fine (The IDE is picking the right runner config on build time).
I hope this will help someone.

My problem was I had default constructor generated by android studio, looked like this
public class SomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public SomeTest(Class<NewsActivity> activityClass) {
super(activityClass);
}
}
and I had to change it to this to get rid of the problem
public class SomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public SomeTest() {
super(ActivityYouWantToTest.class);
}
}

We use the AndroidTestCase class to define that we are testing components which are specific to Android. The main benefit of AndroidTestCase is that it gives us access to the application's Resources such as strings and layouts, etc.
The AndroidTestCase does not require you to overwrite the default constructor as it does not provide a particular Activity as the Context, but rather provides you a general one so you can still call getContext().
In your example, you are not using any Android components, so for you, the following would make sense:
import android.test.suitebuilder.annotation.SmallTest;
import junit.framework.TestCase;
public class DateFormatTest2 extends TestCase {
#SmallTest
public void testMultiply() {
assertEquals("10 x 5 must be 50", 50, 10 * 5);
}
}
Notice the use of TestCase rather than AndroidTestCase.
For AndroidTestCase to be applicable, a test that requires resources would be necessary:
import android.test.AndroidTestCase;
import android.test.suitebuilder.annotation.SmallTest;
public class DateFormatTest extends AndroidTestCase {
#SmallTest
public void testAppTitle() {
assertEquals("MyApp", getContext().getResources().getString(R.string.app_name));
}
}
Here we use the AndroidTestCase because we need to access the application's resources.

This guide might help -
http://www.slideshare.net/tobiaspreuss/how-to-setup-unit-testing-in-android-studio
On the latest gradle (0.9+) the test should be under androidTest dir
Also in your gradle.build:
dependencies {
androidTestCompile 'junit:junit:4.+'
}
also add those under defaultConfig {
testPackageName "test.java.foo"
testInstrumentationRunner "android.test.InstrumentationTestRunner"
}

Did you configure your testRunner in your gradleConfig?
We use different TestRunners for different tests (to speed things up. My config looks like this
android {
// Some stuff
defaultConfig {
// Some other stuff
//junit test
testInstrumentationRunner "de.qabel.qabelbox.QblJUnitRunner"
//ui tests
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
}
If i disable one of this lines the corresponding test will also report "Empty Test Suite".

I just started learning about testing Android applications and I've been struggling with the same problem. You need to provide default constructor for your test class, for example:
package nilzor.myapp.tests;
public class NilzorSomeTest extends ActivityUnitTestCase<ActivityYouWantToTest>{
public NilzorSomeTest(){
super(ActivityYouWantToTest.class);
}
#SmallTest
public void testBlah(){
assertEquals(1,1);
}
}

I already have a default constructor in my test case but still it was giving me error "Empty test suite" and was stuck at "Instantiating tests...".
Tried creating new workspace, resetting Android Studio, but that didn't work.
Finally, close Android SDK and emulator.
Go to your android-sdks/platform-tools.
Clear all Android temp files with these commands:
a. rm -rf ~/Library/Application Support/AndroidStudio
b. rm -rf ~/Library/Caches/AndroidStudio
c. rm -rf ~/Library/Application Support/AndroidStudio
d. rm -rf ~/Library/Preferences/AndroidStudio
Run:
./adb kill-server
./adb start-server
Start Android and run test case.

Randomise the order of instrumentation tests

I'm wondering if it's possible to randomise the order in which instrumentation tests are run, i.e. those extending ActivityInstrumentationTestCase2. I tried following this blog post, but I can't work out how to tell the testing framework that I wish to use my test runner.
The problem is that I can't use the #RunWith annotation, as these are (as I understand it) JUnit3 tests, rather than JUnit4.
It's quite possible that this is pointless, as they don't need to be randomised, but it would be nice to prove the tests' independence in this way.
Ideally I'd like to get it running first using the command line and the gradle wrapper.
Then, it would be nice to have it working via Android Studio, if possible.
[Edit]
I can see that when you do "Edit Configurations . . ." in AS, it's possible to specify your own runner there, via the "Specific instrumentation runner (optional)" box. Unfortunately if I do that, I get the following error:
Test running startedTest running failed: Unable to find instrumentation info for: ComponentInfo{<path_to_class_here>.RandomizingTestRunner}
Empty test suite.
And I can't work out why.

You could use the following randomized runner:
package com.example.test.runners;
import android.test.InstrumentationTestRunner;
import android.test.suitebuilder.TestSuiteBuilder;
import junit.framework.Test;
import junit.framework.TestSuite;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class RandomizedInstrumentationTestRunner extends InstrumentationTestRunner {
#Override
public TestSuite getTestSuite() {
return buildTestSuite();
}
private TestSuite buildTestSuite() {
TestSuiteBuilder builder = new TestSuiteBuilder(getClass().getName(), getTargetContext().getClassLoader());
builder.includePackages("");
List<Test> tests = new ArrayList<Test>();
addTestsFromSuite(builder.build(), tests);
Collections.shuffle(tests);
TestSuite randomizedSuite = new TestSuite();
for (Test one : tests) {
randomizedSuite.addTest(one);
}
return randomizedSuite;
}
private void addTestsFromSuite(TestSuite suite, List<Test> out) {
List<Test> tests = Collections.list(suite.tests());
for (Test one : tests) {
if (one instanceof TestSuite) {
addTestsFromSuite((TestSuite) one, out);
}
else{
out.add(one);
}
}
}
}
and don't forget to set the runner in your build.gradle file:
android {
defaultConfig {
testInstrumentationRunner "com.example.test.runners.RandomizedInstrumentationTestRunner"
minSdkVersion 8
}
....
}
Finally run the following twice to verify the random order of execution:
./gradlew connectedCheck --info