I need to collect some data on my current app in order to analyse performance speed by checking the average ellapsed time during Activity start up. I would like to run a test battery where the activity is started 10, 100, 1000 and 5000 times. For each test, it should remain open for at least 10 seconds (time needed to collect all data that happens asynchronously). What I want is exactly this behaviour without having to write these many methods:
#RunWith(AndroidJUnit4.class)
#LargeTest
public class TestStreamLoadingPerformance {
private static final long TIME_OUT = 2;
private static final long WAITING_TIME = 10000;
#Rule
public ActivityTestRule mActivityRule = new ActivityTestRule(HomepageActivity.class);
private ElapsedTimeIdlingResource mIdleRes = new ElapsedTimeIdlingResource(WAITING_TIME);
#Before
public void setUp() {
IdlingPolicies.setMasterPolicyTimeout(TIME_OUT, TimeUnit.HOURS);
IdlingPolicies.setIdlingResourceTimeout(TIME_OUT, TimeUnit.HOURS);
Espresso.registerIdlingResources(mIdleRes);
}
#After
public void tearDown() {
Espresso.unregisterIdlingResources(mIdleRes);
}
#Test
public void test01() {
}
#Test
public void test02() {
}
#Test
public void test03() {
}
#Test
public void test04() {
}
#Test
public void test05() {
}
#Test
public void test06() {
}
#Test
public void test07() {
}
#Test
public void test08() {
}
#Test
public void test09() {
}
}
With the help of #Be_negative comments, this blog post and this answer, I was able to solve the problem with the code below:
#RunWith(AndroidJUnit4.class)
#LargeTest
public class TestStreamLoadingPerformance {
#Rule
public ActivityTestRule mActivityRule = new ActivityTestRule(HomepageActivity.class, false, false);
#Rule
public RepeatRule mRepeatRule = new RepeatRule();
#After
public void tearDown() {
closeActivity();
}
private void closeActivity() {
final int N = 10;
try {
for (int i = 0; i < N; i++) {
Espresso.pressBack();
}
} catch (NoActivityResumedException e) {
Log.e(TestStreamLoadingPerformance.class.getSimpleName(), "Unable to close activities", e);
}
}
#Test
#RepeatRule.Repeat(times = 10)
public void collectData() {
mActivityRule.launchActivity(null);
}
}
import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
public class RepeatRule implements TestRule {
#Retention(RetentionPolicy.RUNTIME)
#Target({
java.lang.annotation.ElementType.METHOD
})
public #interface Repeat {
public abstract int times();
}
private static class RepeatStatement extends Statement {
private final int times;
private final Statement statement;
private RepeatStatement(int times, Statement statement) {
this.times = times;
this.statement = statement;
}
#Override
public void evaluate() throws Throwable {
for (int i = 0; i < times; i++) {
statement.evaluate();
}
}
}
#Override
public Statement apply(Statement statement, Description description) {
Statement result = statement;
Repeat repeat = description.getAnnotation(Repeat.class);
if (repeat != null) {
int times = repeat.times();
result = new RepeatStatement(times, statement);
}
return result;
}
}
The easiest (as in least amount of new code required) way to do this is to run the test as a parametrized test (annotate with an #RunWith(Parameterized.class) and add a method to provide 10 empty parameters). That way the framework will run the test 10 times.
This test would need to be the only test in the class, or better put all test methods should need to be run 10 times in the class.
Here is an example:
#RunWith(Parameterized.class)
public class RunTenTimes {
#Parameterized.Parameters
public static List<Object[]> data() {
return Arrays.asList(new Object[10][0]);
}
public RunTenTimes() {
}
#Test
public void runsTenTimes() {
System.out.println("run");
}
}
With the above, it is possible to even do it with a parameter-less constructor, but I'm not sure if the framework authors intended that, or if that will break in the future.
If you are implementing your own runner, then you could have the runner run the test 10 times. If you are using a third party runner, then with 4.7, you can use the new #Rule annotation and implement the MethodRule interface so that it takes the statement and executes it 10 times in a for loop. The current disadvantage of this approach is that #Before and #After get run only once. This will likely change in the next version of JUnit (the #Before will run after the #Rule), but regardless you will be acting on the same instance of the object (something that isn't true of the Parameterized runner). This assumes that whatever runner you are running the class with correctly recognizes the #Rule annotations. That is only the case if it is delegating to the JUnit runners.
If you are running with a custom runner that does not recognize the #Rule annotation, then you are really stuck with having to write your own runner that delegates appropriately to that Runner and runs it 10 times.
Note that there are other ways to potentially solve this (such as the Theories runner) but they all require a runner. Unfortunately JUnit does not currently support layers of runners. That is a runner that chains other runners.
I had a very similar issue and as a result I've created a library to run Android UI tests multiple times. Might be useful in your case: https://github.com/stepstone-tech/AndroidTestXRunner
Related
I have some unit tests using androidx Espresso Every time I run the test I get the following error.
androidx.test.espresso.IdlingResourceTimeoutException: Wait for [WorkTrackerIdlingResource] to become idle timed out
Why am I getting timed out?
Its timing out on this line of code in my unittests,
onView(withId(R.id.map_container)).check(matches(isDisplayed()));
Unit Test Class:
#RunWith(AndroidJUnit4.class)
#LargeTest
public class CanvassFragmentTest extends TopLevelNavFragmentTest<CanvassFragment> {
#NonNull
#Override
protected TopLevelNavFragment getTargetTopLevelNavFragment(#NonNull MainActivity activity) {
return activity.mCanvassFragment;
}
#Test
public void mapContainer_isDisplayed() {
onView(withId(R.id.id1_container)).check(matches(isDisplayed()));
}
}
And the xml which has the view we are trying to test.
<androidx.coordinatorlayout.widget.CoordinatorLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context="com.company.MapContainerFragment">
<FrameLayout
android:id="#+id/id1_container"
android:layout_width="match_parent"
android:layout_height="match_parent">
</FrameLayout>
.....
app:layout_behavior="com.google.android.material.bottomsheet.BottomSheetBehavior" />
</androidx.coordinatorlayout.widget.CoordinatorLayout>
I've tried changing the idlepreferences so that the timeout is 5 minutes and I still get the error.
EDIT 1
as request I'm adding the code were I add onTransitionToIdle at the bottom of this class bellow
public class WaitForWorkRule implements TestRule {
#Override
public Statement apply(Statement base, Description description) {
return new Statement() {
#Override
public void evaluate() throws Throwable {
WorkTrackerIdlingResource idler = new WorkTrackerIdlingResource();
//Espresso.registerIdlingResources(idler);
IdlingRegistry.getInstance().register(idler);
try {
base.evaluate();
} finally {
IdlingRegistry.getInstance().unregister(idler);
}
}
};
}
/**
* #hide visible for injection
*/
public static class WorkTrackerIdlingResource implements IdlingResource {
private static final String TAG = WorkTrackerIdlingResource.class.getSimpleName();
#Inject
WorkTracker mWorkTracker;
#Nullable
ResourceCallback mResourceCallback;
public WorkTrackerIdlingResource() {
getUiTestInjector().inject(this);
}
#Override
public String getName() {
return TAG;
}
#Override
public boolean isIdleNow() {
boolean idle = !mWorkTracker.isAnyoneWorking();
if (idle && mResourceCallback != null) mResourceCallback.onTransitionToIdle();
return idle;
}
#Override
public void registerIdleTransitionCallback(#Nullable ResourceCallback callback) {
mResourceCallback = callback;
}
}
}
And this is the code were we register it as a rule.
#Rule
public final RuleChain mRuleChain;
public UiTest(#NonNull Class<T> activityClass) {
mActivityTestRule = new ActivityTestRule<>(activityClass);
mRuleChain = createRuleChain();
}
#NonNull
private RuleChain createRuleChain() {
RuleChain chain = RuleChain.emptyRuleChain()
.around(new InjectionRule())
.around(new WaitForWorkRule());
if (loggedIn()) {
chain = chain.around(new LoggedInRule());
}
return chain.around(mActivityTestRule);
}
Here's a brief explanation why your test always time out: you set a test rule that registers an idling resource before test begins, then when your activity starts, the main thread becomes busy. Generally, the test will continue when the main thread becomes idle, but your idling resource now blocks it and does not call onTransitionToIdle. So no matter how much time you set, it will always time out.
I think your usage of idling resource here may be not useful or incorrect. If there's no use case for it, you can take the rule out and it should work fine.
I apologise if this is a bit too vague here, but I'm not allowed to post my whole actual code. All I can say is I have a problem running this test as a part of ./gradlew connectedAndroidTest
#RunWith(AndroidJUnit4.class)
#LargeTest
public class MobileAppSanityTest extends AbstractEspressoTest {
#Rule
public ActivityTestRule<MainActivity> mActivityRule =
new ClearPreferencesActivityTestRule<>(MainActivity.class, getFiles());
#Override
protected Context getContext() {
return mActivityRule.getActivity();
}
#BeforeClass
public static void beforeAll() {
RoboGuice.Util.reset();
}
#Test
public void test_SingleUserFlow() {
navigateSplashScreen();
logIn();
doSomethingElse();
}
}
What happens here is that when I run this test class on its own - it runs fine, but when I run it as a part of 'connectedAndroidTest' the activity is stopped right after 'navigateSplashScreen' and login cannot be performed.
Error I get is:
java.lang.RuntimeException: No activities found. Did you t to launch the activity by calling getActivity() or startActivitySync or similar?
I'm quite new to Espresso and Android in general, so it's a bit hard to wrap my head around this. Please let me know if you need more information. I'll try to provide it out if that's the case.
a jUnit TestCase looks differently; think one can only use Espresso in there.
#RunWith(AndroidJUnit4.class)
public class MainActivityTest extends TestCase {
/** Log Tag */
private static final String LOG_TAG = MainActivityTest.class.getSimpleName();
/** the Activity of the Target application */
private MainActivity mActivity;
#Rule
public ActivityTestRule<MainActivity> mActivityRule = new ActivityTestRule<MainActivity>(MainActivity.class) {
};
#Override
public void setUp() throws Exception {
super.setUp();
}
/* obtaining the context from the ActivityTestRule */
#Before
public void setUpTest() {
this.mActivity = this.mActivityRule.getActivity();
}
/* add Espresso code eg. here */
#Test
#UiThreadTest
public void navigateSplashScreen() {
}
#Override
public void tearDown() throws Exception {
super.tearDown();
}
}
I've started learning android unit tests, but it looks very hard to find some good guides or information. Every example have a stupid example about 2+2 = 4
Say I write a little SDK which has few functions
MySdk.Init(Context context)
MySdk.CallTask()
I create an androidTest file
How should I call my SDK functions to check how they work? Somewhere required parameters like int/string/context. I just really don't understand, please help me.
This is what I've tried
public class AndroidTest {
private Activity context;
//default test
#Test
public void addition_correct() throws Exception {
assertEquals(4, 2 + 2);
}
#Test
public void checkContext() {
context = getActivity();
assertNotNull(context);
}
#Test
public void testInitPhase() {
MySdk.Init(context, new SdkInitializationListener() {
#Override
public void onInitializationSuccessful(String adv_id) {
assert (adv_id != null);
}
#Override
public void onInitializationError() {
}
});
}
}
For context i was tried context = new mockContext();. It's passed as context = null and my SDK failed with initialization.
Unit tests are mainly about testing an individual class in isolation, so that you can check if individual public methods of a class behave as you intend them to, and continue to do so if you change that class' code in the future. Let's say you have a class like this:
public class UtilityFunctions {
public int double(int value) {
return value * 2;
}
public String mirror(String value) {
if (value == null) return "";
return value + new StringBuilder(value).reverse().toString();
}
}
You want to test these two methods with:
valid input values, and check the output is as expected
invalid values, and check that errors are handled accordingly (and the correct exceptions thrown if necessary)
So a test class for the above class may look like this
#RunWith(JUnit4.class)
public class UtilityFunctionsTest {
private UtilityFunctions utility;
#Before
public void setUp() {
// Initialises any conditions before each test
utility = new UtilityFunctions();
}
#Test
public void testDoubleFunction() {
assertEquals(2, utility.double(1));
assertEquals(8, utility.double(4));
assertEquals(-12, utility.double(-6));
assertEquals(0, utility.double(0));
}
#Test
public void testMirror() {
assertEquals("", utility.mirror(null));
assertEquals("", utility.mirror(""));
assertEquals("aa", utility.mirror("a"));
assertEquals("MirrorrorriM", utility.mirror("Mirror"));
}
}
These standard Java unit tests are run from the test directory. However, you'll need to run tests in the androidTest directory whenever you're using Android-specific classes such as Context. If you're creating a MockContext, you're simply creating an empty Context whose methods don't do anything.
Without me knowing anything about what your MySDK does, I think you may need to pass a fully-functioning Context into your class for your tests. The Android JUnit runner does provide this with InstrumentationRegistry.getTargetContext(), so for your example, you may need to add this #Before method:
#Before
public void setUp() {
context = InstrumentationRegistry.getTargetContext();
}
You'll also need to remove the context = getActivity(); line from your first test.
I'm looking to do a simple test. I just want my espresso test script to verify that I'm not on production. Bad things happen if I run a purchase on production, let alone lots them..
I know in Java you need to add a -ae to run assertions. Which doesn't seem to be as simple in an android espresso test. I'll be handing this code off to the testers so I really really need it to fail if it's on the production. (obviously I'll wrap it in an IF, but I want it to be more ugly -- you messed up -- kinda thing.)
public class PurchaseTest extends BaseFooTest<HomeActivity> //ActivityInstrumentationTestCase2<LoginRegisterActivity>
{
final static String TAG = "PurchaseTest";
static final String PROD_URL = "https://api.foobar.com";
public PurchaseTest()
{
super(HomeActivity.class);
}
public void test()
{
System.out.println(fooApplication.hostUrl);
assert fooApplication.hostUrl.equalsIgnoreCase(PROD_URL) == false;
assert fooApplication.hostUrl.equalsIgnoreCase(PROD_URL) == true;
// No assert! Not being read then!
}
////////////////////// boss mans code, that the class is extending, I don't think it matter, but included it incase the extends basefootest confused someone.
public class BaseFooTest<T extends Activity> extends ActivityInstrumentationTestCase2
{
public BaseFooTest( Class<T> activityClass )
{
super( activityClass );
}
#Override
public void setUp() throws Exception
{
super.setUp();
getActivity();
tryClickOk();
}
protected ViewAssertion isDisplayed()
{
return ViewAssertions.matches( ViewMatchers.isDisplayed() );
}
protected void tryClickOk()
{
try
{
onView( withText( "OK" ) ).perform( click() );
}
catch ( NoMatchingViewException e )
{
// Eat it
// System.out.print( e );
}
}
}
It's possible to enable keyword asserts, but it requires a manual step, and it would be unusual to use keyword asserts in test code.
It's best to use the JUnit methods (assertTrue, etc.) as you would in unit tests.
I want to run a test multiple times. For example, lets say I have a test class like this :-
RunWith(RobolectricTestRunner.class)
public class Common {
int n = 0;
#Test
public void shouldRun() {
System.out.println(n++);
...
}
}
Now, how can I keep repeating the test until a condition is met, like suppose if n = 10 then stop.
I tried a way as described in this blog, and created a custom test runner that extends RobolectricTestRunner and wrote the test class like this :-
#RunWith(ExtendedRobolectricTestRunner.class)
public abstract class Common {
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.METHOD})
public #interface Repeat {
int value();
}
int n = 0;
#Test
#Repeat(10)
public void shouldRunCase1Port() {
System.out.println(n++);
}
but didn't succeed.
Can anybody advice on it ?