I have the following Espresso test. It always passes if I run it by itself, but always fails when I run all the tests in the class together.
What's also a bit strange is that it used to work even as part of the suite. I'm not sure why now it stopped working. It must be something I've done but I don't know what.
#Test
public void itemHasImage_ShowsImage() {
closeSoftKeyboard();
if (mItem.getImageUrl() != null) {
onView(withId(R.id.edit_item_image)).perform(scrollTo())
.check(matches(isDisplayed()));
}
}
The error I'm getting is:
Error performing 'scroll to'...
...
Caused by: java.lang.RuntimeException: Action will not be performed
because the target view does not match one or more of the following
constraints:(view has effective visibility=VISIBLE and is descendant
of a: (is assignable from class: class android.widget.ScrollView...
The view is visible and a descendant of a scroll view, as evidenced by it passing when run on it's own.
When it gets to this test (in the suite) it just doesn't scroll. But when I run it by itself it scrolls just fine.
In another stack overflow question I asked recently Espresso not starting Activity the same for second iteration in parameterised test, I found out that onDestroy from the previous test was getting called after onResume in the current test, causing it to set a value to null and fail the test. Again in that situation, the problem was that the test passed by itself but not in the suite. I now have a similar problem but no obvious way to fix it. (Edit: the workaround for the other question can no longer be applied).
Any ideas anyone? Could it be reading the wrong Activity somehow? Like maybe it's looking at the one from the previous test. That sounds ridiculous but after that last problem I had it seems possible.
Edit: Ok it turns out that when running this test as part of the suite, the image is in fact not visible causing the test to fail. I found this using the debugger and scrolling the view manually. But why?
I think it's a bug and have logged an issue here:
https://code.google.com/p/android/issues/detail?id=235247
It can able to solve using Orchestrator android testing utility library, It will executing each test case independently, so there is no chance of breaking in test suite
When using AndroidJUnitRunner version 1.0 or higher, you have access to a tool called Android Test Orchestrator, which allows you to run each of your app's tests within its own invocation of Instrumentation.
Enabling using build.gradle
android {
defaultConfig {
...
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
}
testOptions {
execution 'ANDROID_TEST_ORCHESTRATOR'
}
}
dependencies {
androidTestImplementation 'com.android.support.test:runner:1.0.1'
androidTestUtil 'com.android.support.test:orchestrator:1.0.1'
}
For more info:https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator
I had a similar issue but the cause was different, #Rakshith-Kumar suggestion of using Orchestrator might work, also putting each test method in a TestSuite could also be a solution, since the flakiness could be addressed by running each test individually.
Also want to mention that if you're testing component that's using RxJava schedulers (io scheduler for example) to run some tasks in a background thread, the flakiness could happen because Espresso will not be able to synchronize with the background threads.
A solution can be injecting a the ObservableTransformer to do the subscribeOn and observeOn operations. And for Espresso tests substitute the ObservableTransformer instance with another one that uses Schedulers.from(AsyncTask.THREAD_POOL_EXECUTOR) for the io scheduler.
Something like that:
public <T> ObservableTransformer<T, T> ioTransformer() {
return observable -> observable.subscribeOn(AsyncTask.THREAD_POOL_EXECUTOR).observeOn(AndroidSchedulers.mainThread());
}
This way Espresso can synchronize with background threads and avoid flakiness.
More info:
https://blog.danlew.net/2015/03/02/dont-break-the-chain/
https://developer.android.com/training/testing/espresso/#sync
Related
did anyone else faced the similar problem, details are:
in earlier version of androidx.fragment:fragment-testing:1.4.1 library the mockito based tests passes but after upgrading the library to 1.5.3 version the tests fails by giving this error.
code which is used to test the fragment:
activity = Robolectric.buildActivity(AppCompatActivity::class.java).setup().get()
activity.supportFragmentManager.beginTransaction().add(android.R.id.content, fragment).commit()
if (idleLooper) {
shadowOf(getMainLooper()).idle()
}
here If I pass the normal fragment object, the tests runs and it continue to run the test, but if I pass spy(fragment) and then try to run code (test) it breaks with the above error.
gone through the change documentation for library as well, but nothing specific is mentioned regarding fragmentmanager.
https://developer.android.com/jetpack/androidx/releases/fragment#1.5.4
release version documentation
I tried:
passing the fragment object which does pass the test cases, but spy(fragment) gives error in the 1.5.3 version of library, and same code passes till 1.4.3 library.
tried fragementScenario bases test case but that also fails to load the fragment and same error appears while running tests.
I'm having error in the tests only (Roboelectric & mockito based tests)
here is the fragement object I'm passing
MyFragment(contact, viewmodel, fragmentManger)
and the viewmodel and fragement are mock objects, and contact is just data object)
I have a few "connected" tests that are only relevant to run on a specific device model or on a specific brand and should be skipped on other brands/models.
I may be missing something, but this kind of filtering seems not possible out-of-the-box with AndroidJUnitRunner (by using annotation and/or passing appropriate arguments to it).
So, I was thinking to extend the AndroidX test framework to support this kind of filtering. In the end, I would like to be able to filter test with something like this
#TargetDeviceFilter(brand="SAMSUNG",model="XCover3")
#Test
public void myTestToRunOnSamsungXCover3DeviceOnly(){
...
}
My question: is there any way to accomplish this kind of filtering without extending AndroidX test framework? And if writing my own AndroidJUnitRunner and/or my own annotations is required, how should I start ?
I found a few interesting base classes that I may need to extend like :
androidx.test.internal.runner.TestRequestBuilder
androidx.test.internal.runner.TestRequestBuilder.DeviceBuild
but as those classes are in a "internal" package: attempting to extend them is probably not a good idea?
Any advice on how to deal with that problem is welcome.
I think, you may use org.junit.Assume.
Create a helper class DeviceHelper to detect mobile device informations for convenience.
Your test logic will be executed only if the assumption is correct.
#Test
public void myTestToRunOnSamsungXCover3DeviceOnly() {
// adapt this part to your business need
org.junit.Assume.assumeTrue(
DeviceHelper.isBrand("SAMSUNG") &&
DeviceHelper.isModel("XCover3")
);
// i.e. you can filter whatever you want test's according to device sdk_int
assumeTrue(SomeHelper.getDeviceSdk() >= 21);
// your test code
}
Have the following doubts about testing View (Activity, Fragment). The first approach is to test view using robolectric with mocking Android components like the following code do:
private void setUp() {
fragment = new CustomFragment();
activity = Robolectric.setupActivity(MainActivity.class);
}
#Test
public void testOnCreateView() {
fragment.onCreateView(LayoutInflater.from(activity), (ViewGroup) activity.findViewById(R.id.container), null);
verify(repoInfoPresenter).onCreateView(null);
}
Don't like this approach, cause this tests executing takes long time. In my opinion unit tests should be executed on each merge request, so they shouldn't be as slow as tests like that.
Another variant is to write Instrumentation tests using Espresso. I know that it's already takes long time and real device, but I don't think that this test should executed as often as tests running on jvm.
Or maybe there is magical variant to mock(make stubs) for fragments or Activities which I don't know about?
So i have a an android application project and unit test project to it. I am a beginner in Eclipse and maybe i don't understand properly how to work with JUnit.
I have written two simple test methods:
#Test
public void searchTest() {
fail("test is failed");
}
#Test
public void getTest() {
assertTrue(10 > 1);
}
So my problems are
Firstly: It is obvoius that searchTest must fail but when i run it doesn't happend :
Secondly: in some reason i can not debug my test methods. I click right button on serachTest in JUnit dialogue and select "Debug" but debugging is not starting.
Thirdly: Only one test method is showing in in JUnit dialogue however i have two methods with #Test annotation in my class. Why does it happend?
Could anyone help me? Thanks in advance
Think about what you had selected when you said to Run it as a Test. You need to select the class itself rather than a method within it. The JUnit View will only show you the tests that were run.
And you need to set a Breakpoint in your test code to have any observable difference from just Running the test.
I have been having quite a bit of trouble implementing unit testing on the Android. As a simple test, I've been trying to match a string retrieved from string resources:
String myString = myActivity.getResources().getString(R.string.testString));
However, when unit testing this invariably results in a null pointer exception. This includes robolectric as well as the Junit implementation delivered with the Android sdk.
One possible solution is to approach the retrieval of resources in a manner similar to a data access object. That is, create an interface through which string resources would be accessed. This would allow me to mock access the string resource. Similarly, I could separate the non-android dependent behavior of, say, an Activity, into a separate pojo class. This would allow me to run unit tests using standard Java testing tools. In fact, I could potentially delegate any Android infrastructure related activity to an interface.
This seems like a lot of jumping through hoops to get to unit testing. Is it worth it? Is there a more viable approach?
It turned out, the problem was that the activity has to be gotten in the actual test method. So, for example, my method now looks like this:
public void testGetActivityResourceString() {
Activity myActivity = this.getActivity();
String myString = myActivity.getResources().getString(R.string.hello);
Assert.assertNotNull(myString);
}
Whereas before I was creating activity in setup. This giveaway was in the docs:
"For each test method invocation, the Activity will not actually be created until the first time this method is called."
This was a real hassle to figure out. The example for HelloWorldTest doesn't work for the same reason.
Here's the full entry:
Public T getActivity ()
Since: API Level 3
Get the Activity under test, starting it if necessary.
For each test method invocation, the Activity will not actually be created until the first time this method is called.
If you wish to provide custom setup values to your Activity, you may call setActivityIntent(Intent) and/or setActivityInitialTouchMode(boolean) before your first call to getActivity(). Calling them after your Activity has started will have no effect.
NOTE: Activities under test may not be started from within the UI thread. If your test method is annotated with UiThreadTest, then your Activity will be started automatically just before your test method is run. You still call this method in order to get the Activity under test.
This works correctly:
public void testGetResourceString() {
assertNotNull(mActivity.getResources()
.getString(com.example.pkg.R.string.testString));
}
Because you haven't provided any of your code but only the getReousrces() line, I will guess what you are doing wrong:
you are not using the correct base class for your test, use ActivityInstrumentationTestCase2 because you need the system infrastructure
you are using the resources of your test project instead of your project under test, that's why in my example the id is com.example.pkg.R.string.testString.