I am using Robotium Solo to test the app
since I'm fairly new to app test, and Robotium
I have 3 methods in my test case - however, I want to run those tests, under certain conditions
otherwise they fail
I can do that if i write the entire test in one method, but i don't want to run it in one method, I
want to separate it into 3 methods
how do I make sure that the tests run only if I called the test methods, and not one after the other
In Robotium Test Application, every public method starting with keyword 'test' will be considered as one test case.
So, you can try using 2-3 methods with names starting by keyword test.( e.g. testA(), testB() etc.)
So, even if testA() fails, automation testing will continue to run for testB().
And results will be shown to the JUnit viwe.
Note:- the test cases are executed in the alphabetical order as per their names.
Hope this helps , ask if you are stuck somewhere.
Related
I know that, in test automation, we have to avoid sequential test-cases.So, the order of running the test-cases are not important.
I believe in some cases the sequential test-cases is unavoidable:
1.Consider a scenario which a user needs to take some previous steps in order to complete a final goal. For example, a user needs to be logged in, so that he can purchase.
Given User is logged in
When User adds an item into its basket
And User Complete his purchase
Then He receives an Email
So, in above scenario, each part (Given, When, And, or Then) are seperate testcases. But still the order of testcase is crucial.
2.Also, Junit team provides a method called #FixMethodOrder(MethodSorters.NAME_ASCENDING) for such usage, but i am not sure when we are allowed to use this?
So, how do you write independent test-cases in end-2-end testing?
Tests are meant to be atomic, in the sense that they should NOT rely on a status that was generated by a previous test. That's why sequential test-cases should be avoided. In case any part of the sequence fails, then the rest of the tests fail, so it really doesn't make much sense to have them sepparate (they could be just one test).
If you need preconditions on your tests you can include them in: 1) before all tests, 2) at start of the test suite, 3) before your specific test, 4) inside your specific test. No matter where/when this is done, the important thing is that you assume that all that's needed to generate that status works as expected, so you just focus on what you are testing.
Now, if you want to test a complete flow of a system, I would recommend you to put that inside one unique test. In case there are variations, then you would have multiple tests.
Let's say for example that you want to test D, but D needs C, and then C needs B, and finally B needs A; one strategy would be to generate a test in which you would do A->B->C->D. Another strategy is to generate multiple tests in which you test the individual steps: test 1 does A; test 2 does A->B (but doesn't care if A is ok); test 3 does A->B->C (but doesn't care if A and B are ok); and so on. Which strategy to use would depend on your goals and the size of the scenario. I personally prefer the second option for big things, and the first one for simple/short ones.
The espresso UI test framework allows to implement such core-user-journey scenarii.
If you want unit tests, then you should set the expected preconditions at the beginning of each.
The saying of "avoid sequential test cases" only apply to unit tests. And that is where Junit and mockito shine. In these cases, we are only worried about a unit correctness. We don't care about other units fail or pass or whatever, we will mock other unit behaviors, so that we can focus on testing exact behaviors of current test.
The case you described don't fall into this category. In your case, you need an integration test suite since you need to test end to end flow. You can use the following tools for integration test.
Cucumber
Soap UI Test
Serenity
etc
I am heavily testing my application with unit tests and Espresso tests. My next step is to make sure my application hits all required apis. For that reason I am looking for a way to write a test, which will verify the api calls.
I would really appreciate any suggestions.
What you are describing is called a "unit test". Unit tests are meant to test as many lines of code as possible regardless of UI.
Espresso tests are "instrumentation tests" (or "UI tests") intended to check if the app is responding to UI events correctly. They're not meant to verify the correctness of code, but the correctness of the functionality of the app as used by the user.
You can read about both at the official documentation. You'll find that unit tests are very different than instrumentation tests, and often harder to write because they require more engineering of your code to do correctly. You will likely have to "mock" the various parts of your application to make sure their APIs were called exactly as you expected.
There are 2 main goal when I was writing api tests:
First is component based. The goal was to make sure each class / component makes an api call when certain criteria is met (for example calling an api A when onDestroy() is called)
Second, is to make sure the Apis are called is certain order for the analytics purposes.
The first step I achieved by using unit tests with injected mock objects via Mockito and PowerMockito. PowerMockito was used primarily to mock static methods and to make sure the methods were called at least n times.
For the second step, UI test could be used, since it runs the real application. I have implemented the helper class, which was recording the instances when api requests were made. The script in Espresso was validating the order of api calls by referring the helper class.
In my test class, i have multiple test cases written. Now when I run the test project, after each test case the app is getting stopped and started again. In teardown I am calling solo.finishOpenedActivities().
I want to run all test cases without closing the activities.
I tried to remove solo.finishOpenedActivities from the tearDown method. In that case the next testCases are not getting executed and the test hangs after the first one.
So what is the proper way to have multiple test cases and not close the activities and continue running all the tests?
If the functionalities between your test cases are dependent on each other, then instead of writing them in separate testcases, write different functions related to different functionality and call then in same testcase according to their order of execution.
Because, after completion of each testcase, robotium will definitely close the activity and start a new activity for the other testcase.
I'm trying to write a batched instrumentation test (using ActivityInstrumentationTestCase2) for a particular Activity where I change the intent each time the test runs. I can do this with a single test, and just loop through stopping and restarting the Activity with the new intent, but this is not what I want. One reason is they really should be separate test runs. The other reason is, I'm using Spoon to generate a report when the tests finish, and the report will rightly think I only ran one test.
What I would like is it to treat a single test as a possibly infinite number of tests, and pass the data into the test each time the test runs.
Unfortunately you can't use Theories because it results in a RuntimeException where the InstrumentationTestRunner can't find my tests. Anyone have any luck with this?
You could always just create a "testing" intent. In order to simulate the relaunching of the application, make a method or several methods that reset all your static variables between tests. Then you can test the classes from within a testing intent inside the application itself using
assert("value", MyClass.myMethod);
resetStatics();
assert(true, MyClass,myMethod);
resetStatics();
I don't know how much this will help you, if at all, but this is how I started writing my own tests.
I recently discovered that you can add a public static Test suite() method to a test class, and when you run just this single test class, InstrumentationTestRunner will run the Test returned by this method. This is helpful because suite() can explicitly call any constructor of your TestCase, including one with parameters.
So I am trying to write automation test using Robotium for Android
I have a test suite related to my LoginPage, the problem is that setUp and tearDown get called beforfe and after every test, so it closes and opens the app on every single test case.
Is it possible to somehow avoid this, so that setup and tear down get called once for every test suit?
EDIT:
I am using ActivityInstrumentationTestCase2
Use SingleLaunchActivityTestCase instead.
I'm not sure about Robotium, but junit has #BeforeClass and #AfterClass annotations you can apply to a method such that it gets called only once before or after instantiation of the test suite. See more info here:
http://junit.sourceforge.net/doc/faq/faq.htm#organize_3