Develop Reference

registerIdlingResources deprecation replacement doesn't work

I'm trying to replace my Espresso registerIdlingResources and unregisterIdlingResources deprecated method by using IdlingRegistry method according to Android documentation.
Some of my tests worked before the update and no longer work now. These tests work unitarily, but not together.
I noticed that there is a little difference with the old version (of Espresso class), this line is not present in IdlingRegistry class :
baseRegistry.sync(IdlingRegistry.getInstance().getResources(), IdlingRegistry.getInstance().getLoopers());
I think this sync method is very important for my custom IdlingResource...
How can I sync my looper without this line?
Edit: I use EspressoCore 3.0.1 with runner/rules 1.0.1
Edit2: Link of documentation who has been specify deprecation : Here and Here.

Be sure to run the latest version of androidx.test
If your tests run one at a time, but fail when run together, Android Test Orchestrator ("ATO") can solve that problem.
ATO runs each test method in a new process, so any in memory state is cleared.
From the docs, the basic gradle setup is:
android {
defaultConfig {
...
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
// The following argument makes the Android Test Orchestrator run its
// "pm clear" command after each test invocation. This command ensures
// that the app's state is completely cleared between tests.
testInstrumentationRunnerArguments clearPackageData: 'true'
}
testOptions {
execution 'ANDROIDX_TEST_ORCHESTRATOR'
}
}
dependencies {
androidTestImplementation 'androidx.test:runner:1.3.0'
androidTestUtil 'androidx.test:orchestrator:1.3.0'
}
The docs also include setup for installing and using Android Test Orchestrator without gradle.
You can also use ATO on Firebase Test Lab:
via web UI
via CLI
If you still have problems with your IdlingResources you can try the busybee library which simplifies using IdlingResources and makes them easier to debug.
(disclaimer, I am a maintainer of that library)

Is there a way to write a unit test for a target API

I am in the process of writing Android instrumented tests in an area were Robolectric custom shadows fall short.
Ideally, we want to write code that is flexible across all versions of the Android SDK however I am in an edge case situation where I need to write a individual unit test for a method that works only Marshmallow.
I also need to write a unit test that only works for Lollipop and under because of differences in operating system dependencies (i.e. java.lang.NoClassDefFoundError's)
Is there anyway I can do this through Junit4-like annotations or something similar to what Robolectric does where it can ignore running the tests if the SDK is not a good fit?
I do not want to be writing code like this:
// MarshmallowWidgetTest.java
#Test
public void testPrintName()
{
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M)
{
// ...asserts...
}
}
// LollipopWidgetTest.java
#Test
public void testPrintName()
{
if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.LOLLIPOP)
{
// ...asserts...
}
}

I'm not very familiar with unit testing or Robolectric, but because at time of writing unit tests by me there was no support for API 23 I used that config:
#RunWith(RobolectricGradleTestRunner.class)
#Config(constants = BuildConfig.class, sdk = 21) //this guy
public class MainActivityTest {
MainActivity_ activity = Robolectric.setupActivity(MainActivity_.class);
}
So like you see there's a annotation which you can use to your test classes.
EDIT:
Sorry that I focused only on Robolectric test framework, not main problem.
For annotating instrumentation tests for specific API I would use:
1. Class with #Before annotation
Create a class with #Before annotation, where it would check the API of tested devices. If wrong, the tests would fail in this method. Use fail(); method.
2. Use #SdkSuppress annotation
Indicates that a specific test or class requires a minimum API Level to execute.
Test(s) will be skipped when executed on android platforms less than specified level.
From: http://developer.android.com/reference/android/support/test/filters/SdkSuppress.html
So if you would set #SdkSuppress(minSdkVersion=23) it would run only on Android Marshmallow devices and if ##SdkSuppress(minSdkVersion=20) it would run only on higher 5.0 API devices.
Read also: http://www.vogella.com/tutorials/AndroidTesting/article.html
3. Create your own annotation like #SdkOnly
Maybe this article would be useful: http://help.testdroid.com/customer/portal/articles/1256803-using-annotations-in-android-instrumentation-tests
4. Create suites for your specific instrumentation tests
For this purpose you would use #RunWith() and Suites.SuiteClasses() annotations.
To organize the execution of your instrumented unit tests, you can
group a collection of test classes in a test suite class and run these
tests together. Test suites can be nested; your test suite can group
other test suites and run all their component test classes together.
A test suite is contained in a test package, similar to the main
application package. By convention, the test suite package name
usually ends with the .suite suffix (for example,
com.example.android.testing.mysample.suite).
To create a test suite for your unit tests, import the JUnit RunWith
and Suite classes. In your test suite, add the #RunWith(Suite.class)
and the #Suite.SuitClasses() annotations. In the #Suite.SuiteClasses()
annotation, list the individual test classes or test suites as
arguments.
The following example shows how you might implement a test suite
called UnitTestSuite that groups and runs the
CalculatorInstrumentationTest and CalculatorAddParameterizedTest test
classes together.
import com.example.android.testing.mysample.CalculatorAddParameterizedTest;
import com.example.android.testing.mysample.CalculatorInstrumentationTest;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
// Runs all unit tests.
#RunWith(Suite.class)
#Suite.SuiteClasses({CalculatorInstrumentationTest.class,
CalculatorAddParameterizedTest.class})
public class UnitTestSuite {}
From: http://developer.android.com/training/testing/unit-testing/instrumented-unit-tests.html
5. Helpful resources
http://www.netmite.com/android/mydroid/development/pdk/docs/instrumentation_testing.html
https://github.com/googlesamples/android-testing-templates/blob/master/AndroidTestingBlueprint
Hope it help

setup and write demo test the robolectric in Android Studio

Do anyone know how to integrate robolectric into android studio?
How to write sample test?
How to launch it?
I am working with android studio not to long, and I am too bad with gradle.
Searching the net didn't give me a result - I even could not launch official demo - https://github.com/robolectric/robolectric-samples . My android studio do not saw the test class.
Please give me simpliest step by step gide, thanks

Since robolectric runs in a JVM (i.e. not on a device or emulator), it is just a library and adding the test runner is all that's needed.
Make sure that the android SDK is later in the classpath than robolectric or junit - otherwise you'll get the stubbed methods from the android SDK.
#RunWith(RobolectricTestRunner.class)
public class MyActivityTest {
#Test
public void shouldHaveApplicationName() throws Exception {
String appName = new MyActivity().getResources().getString(R.string.app_name);
assertThat(appName, equalTo("MyActivity"));
}
}
See http://robolectric.org/quick-start/

Android Studio - test configuration starts up app

I've got a couple of AndroidTestCase subclasses in separate packages of my project:
However, whenever I run Android Tests configuration from Android Studio, I see that my regular app is starting as well. I see that the onCreate method is fired up inside my Application class (which is really bad since I am loading some additional resources there).
Why is Android Studio/gradle running my app as well? Can I programatically detect if I am inside test or regular configuration? Can I stop my regular app from being booted before running tests?
In addition, when I am running tests in debug mode it doesn't stop on breakpoints placed inside the application's onCreate method. Why is this happening?
Edit:
Body of test class doesn't really matter, it can be something like:
public class SimpleTest extends AndroidTestCase {
public void testSample()
{
assertEquals(true, false);
}
}
Executing only this simple test fires up onCreate method inside application class.
Gradle console prints out:
Executing tasks: [:app:assembleDebug, :app:assembleDebugTest]
I guess that first task creates instance of my Application class - is it expected behavior?

Why is Android Studio/gradle running my app as well?
Can I stop my regular app from being booted before running tests?
AndroidTestCase is extension of JUnit TestCase which is aware of your android application. In case you don't need to test your android application and want to test plain java only you should use JUnit framework. Create regular JUnit tests, do not use android classes there and run JUnit test configuration like this:
You should treat AndroidTestCase as instrumentation tests which will build android app and run that tests on it. This is usefull with combination of Espresso and Robotium. Both are working on top of base android test classes and both will build and run your application before testing it. Real device or emulator is needed.
Use plain JUnit tests or Robolectric to test java on your desktop JVM.
Can I programatically detect if I am inside test or regular configuration?
You can use power of gradle to provide such info with autogenerated BuildConfig file.
At your build.gradle
android {
defaultConfig {
testPackageName "com.foo.test"
}
}
At your code:
BuildConfig.PACKAGE_NAME.equals("com.foo.test")

The AndroidTestCase is an unit test that unfortunately runs on the device (either virtual or real).
I think what you want to have is a UnitTestFramework that runs in the JVM (local on your machine). The TestFramework Robolectric can do this.
I have started a gitHub project to show how to setup the gradle test file and the project structure if you want to have UnitTests and InstrumentationTests side by side. If you want to look its AndroidGradleTests

Android Unit Test Not Reporting Failing with fail()

I've written a unit test that simply extends TestCase and I have the following:
public class MetricParserTests extends TestCase {
#Override
protected void setUp() throws Exception {
super.setUp();
}
#Override
protected void tearDown() throws Exception {
super.tearDown();
}
public void testFailure() {
fail("This needs to fail");
}
}
When I run my tests using ant test or adb shell am instrument I get the following results:
... [exec] OK (1 tests) ...
I'd expect to see a failure on the command line.

I believe I know what the issue is. I was able to reproduce the issue and solve it. The command you use does not rebuild and re-install your test project onto a device. When you call ant test it will just execute the tests which are already installed on that device.
What you need to call is the three commands in your test project's directory:
ant debug
ant installd
ant test
Then all tests will be rebuild and re-installed and latest tests will be executed. If you don't call debug and installd, the changes you did to the tests do not get applied.

I haven't had recent experience in Android testing, but here is what I have found...
You can use normal JUnit tests if your code is totally decoupled from Android (see here
for an example). This would run on your JVM using the JUnit runner.
However, if you are trying to run these tests on an Android device (either via ant, or the command line tools) then you need to create a full android test project (See here).
To test "on device" your test cases need to extend one of the Android test classes like ActivityInstrumentationTestCase2<T>
and are run using the InstrumentationTestRunner in the Dalvik VM on the Android device.
Using an IDE or the command-line tools to create a test project should create a sample test for you to work from.
This blog post linked from the comments of the post above is a good source of information, as is the Android Testing Fundamentals doc.

The method testFailure() does not have a #Test annotation. Is that correct?