I'm writing data driven testing of my android app, and have begun writing a CSV to espresso testing framework which will allow me to quickly design and alter tests. Espresso can verify basic things about the UI such as successful clicks and text entry, but can it handle more advanced logic?
For example, I'm attempting to create a receipt based on some clicks that are made during the test. This will have been costed before runtime and entered into the CSV for automatic comparison of actual vs expected outcomes. In order to do this, I will need to access the receipt object to check its values? Or at least the textbox which displays it (easy, but not really what I'm after).
So, is there any way to get objects stored in memory from the main application into the espresso testing framework?
Found the answer to my question.
In order to get to any objects/variables, you need to get to a class in which they exist. This is achieved in espresso with the #Rule tag:
#Rule
public ActivityTestRule<MainActivity> mainActivityTestRule = new ActivityTestRule<MainActivity>(MainActivity.class);
From here you can do:
mainActivityTestRule.getActivity();
and from there all objects can be accessed
EDIT:
Primitives seem to come through fine, but I'm still trying to confirm if objects are actually passed through. It seems that the objects that are in my Espresso class are default, as though the constructor has just been run. Will update with more info.
Edit 2:
This does NOT work for objects. Putting a break-point in the espresso class and one in the main application and comparing them shows different object IDs. Getting the object from the main application doesn't return the object in the same state.
Related
I know that, in test automation, we have to avoid sequential test-cases.So, the order of running the test-cases are not important.
I believe in some cases the sequential test-cases is unavoidable:
1.Consider a scenario which a user needs to take some previous steps in order to complete a final goal. For example, a user needs to be logged in, so that he can purchase.
Given User is logged in
When User adds an item into its basket
And User Complete his purchase
Then He receives an Email
So, in above scenario, each part (Given, When, And, or Then) are seperate testcases. But still the order of testcase is crucial.
2.Also, Junit team provides a method called #FixMethodOrder(MethodSorters.NAME_ASCENDING) for such usage, but i am not sure when we are allowed to use this?
So, how do you write independent test-cases in end-2-end testing?
Tests are meant to be atomic, in the sense that they should NOT rely on a status that was generated by a previous test. That's why sequential test-cases should be avoided. In case any part of the sequence fails, then the rest of the tests fail, so it really doesn't make much sense to have them sepparate (they could be just one test).
If you need preconditions on your tests you can include them in: 1) before all tests, 2) at start of the test suite, 3) before your specific test, 4) inside your specific test. No matter where/when this is done, the important thing is that you assume that all that's needed to generate that status works as expected, so you just focus on what you are testing.
Now, if you want to test a complete flow of a system, I would recommend you to put that inside one unique test. In case there are variations, then you would have multiple tests.
Let's say for example that you want to test D, but D needs C, and then C needs B, and finally B needs A; one strategy would be to generate a test in which you would do A->B->C->D. Another strategy is to generate multiple tests in which you test the individual steps: test 1 does A; test 2 does A->B (but doesn't care if A is ok); test 3 does A->B->C (but doesn't care if A and B are ok); and so on. Which strategy to use would depend on your goals and the size of the scenario. I personally prefer the second option for big things, and the first one for simple/short ones.
The espresso UI test framework allows to implement such core-user-journey scenarii.
If you want unit tests, then you should set the expected preconditions at the beginning of each.
The saying of "avoid sequential test cases" only apply to unit tests. And that is where Junit and mockito shine. In these cases, we are only worried about a unit correctness. We don't care about other units fail or pass or whatever, we will mock other unit behaviors, so that we can focus on testing exact behaviors of current test.
The case you described don't fall into this category. In your case, you need an integration test suite since you need to test end to end flow. You can use the following tools for integration test.
Cucumber
Soap UI Test
Serenity
etc
I'm using the newer way to instantiate Android's UiAutomator that utilizes the InstrumentationRegistry to retrieve the context. The problem is that the new methodology for automation doesn't extend UiAutomatorTestCase or any set of classes that can pass parameter to retrieve via getParams() in other tests. I tried making static variables but they seem to reset after each case.
I was wondering what the newer methodology of adding retrievable parameters was? I also can't seem to find a dupe question which is why I'm posting this...
EDIT:
As another note to the original question, this is a completely automated test suite that has nothing to do with our main application. It still uses the standard UI automator functionality but is a separate code project.
So basically I'm just looking to store results from one #Test scenario to the next #Test scenario that runs in succession.
Thank you
You should use InstrumentationRegistry.getArguments(). It returns the same Bundle of command line arguments as UiAutomatorTestCase.getParams().
I'm trying to add test to a quite big android project. So, I'm able to add the test, both to activity both to fragments, and get they run correctly.
But in my case, most of the fragments, when started, starts a Runnable into a new Thread. This thread connect to APIs, request and parse data, and return them to a handler, in the calling fragment.
The question is: How am I supposed to ask the test to wait until the data are retrieved by handler, so I can check them values?
How about instead of waiting for the values to return from real objects. You create mock objects that can return whatever value want instantly. Mockito is a great mocking framework that is very easy to use on Android.
http://code.google.com/p/mockito/
The way you go about using it is to create mock objects as opposed to real objects and you have to find "seams" to insert them into your tests. With a mock object, you can get it to return whatever value you want for any of its methods. With a mock object you can isolate dependencies. Therefore, in your example, you may want to create mock objects that mock the object that is connecting to the api because then you can return whatever value your tests need for the server to return to test whichever part of your app that you are testing. For example, if your testing whether the signUp activity is opened if the server returns that the user has entered a wrong password, you create a mock object that will return false to your activity and then insert that into where the real object is supposed to be.
Most people have issues testing existing code (that was not built with testability in mind) because there usually aren't any seams to insert test objects. Check out this awesome blog about how to make code more testable. It is contains really great advice.
http://googletesting.blogspot.com/2008/08/by-miko-hevery-so-you-decided-to.html
I have troubles with my Robolectric unit test.
I can assert without problem that a click started a new activity when the listener uses the method startActivity( Intent )
But it seems Robolectric has trouble when a new activity is started with the method startActivityForResult(Intent, int) : putting some breaks in the code made me figure out that the activity wasn't started ( and just changing for method startActivity( Intent ) made the assert pass).
Is that normal ? It's a pity since the first activity of my app uses startActivityForResult(Intent, int).
Did someone manage to make tests with this way of launching activities ?
Thanks for your help ..
The short Answer to your question is that, due to the way Robolectric converts Android classes to code that executes in the JVM, a lot of their functionality doesn't behave as you'd expect. Many system callbacks won't execute, and you'll have to rely on what Robolectric provides in their Shadow implementation of classes. (See the link provided by #Steven_BDawg).
The long answer: It may be possible to implement this whole flow in one big test, but it's not what the framework is designed for.
Robolectric and Unit Testing in general aren't meant to be used in the way that you describe. The Unit Testing page on wikipedia states that one can view a unit as the smallest testable part of an application. A unit testing suite should contain many lightweight tests, where each test isolates a bit of functionality in your app and ensure it's working properly.
Consider a basic Application that contains two Activities, A and B. Activity A displays some information about a topic, and Activity B allows the user to select which topic to show in A. When the user moves from Activity A to Activity B, B gets called with startActivityForResult() and should return to A with the selected topic.
Now say we want to Unit Test this flow of A getting the result from B and displaying the data. We can break this up into two tests:
Activity Under Test - Activity A. In our test, we'll create a new instance of Activity A. In the Robolectric Test, we create the Intent that we expect B to return to A, and call the shadow method receiveResult() for A, filling out the arguments with a result code of OK and this Intent. After receiveResult(), run your assertions. You now know that Activity A handles the result properly!
Activity Under Test - Activity B. In our test, we'll create a new instance of Activity B, setting it up as if it were started for result from Activity A. In the Robolectric Test, we'll perform all actions needed to select the data, create the intent we'll send back, then run assertions on the intent to ensure it was created correctly.
This is a very simple example. These two steps could probably be broken out into many more tests, as, again, each unit test should only be testing the smallest unit of functionality that your app can be broken into. The example is mainly to help you start thinking in a unit testing kind of way. I've found that as my understand of unit testing deepens, the way I write code has changed. I try to avoid writing methods and classes in such a way that they do too much work and cannot be properly unit tested. As a rule of thumb, code that's easy to unit test performs very specific operations which are readily apparent when reading the code for the first time.
Finally, if you want to take this a step further, mocking frameworks can greatly aid your ability to Unit Test. Mockito is a mocking framework I've had success with in the past. The purpose of a mocking framework is to create stub Objects whose behavior you tightly control. Mockito (or any other Mocking Framework), will allow you to define an object that extends from any type you need and only implement the methods you need. You'll be able to directly control the response to any of these method invocations. This aids Unit Testing because the only real object that you'll need is the Object Under Test; by Mocking all other objects, you'll have a better sense of whether or not the Object Under Test is behaving properly because all other behavior is explicitly defined by you, the tester. (And yes, this does lead to lots of extra code, but such is the life of a good unit tester. However, as previously stated, as you get more comfortable with unit testing, you may find yourself writing methods that require less mocking and are more conducive to writing tests. Some coders will even write their unit tests BEFORE they code, in order to keep their code tight and focused on a single purpose)
Hope this helps!
I did some Google'ing for you. I don't know if this will definitely help you or not, but I think it is a good start!
Roboelectric: Testing startActivityForResult() and onActivityResult()
I would like to make some automatic UI tests for my Android application using recomended approach ( http://developer.android.com/tools/testing/index.html ). My activity fetches data from server and after getting response it modifies some UI elements. I want to check whether UI is modified properly but I also want to test this issue without connecting to server. I thought about providing mock component instead of object which connects to server on behalf of activity. But how could I in my test cases tell acivity to replace original object with mock object? Do you know any good approches?
I thought about putting a special flag in Intent using setActivityIntent (indicating that mock object should be used) before calling getActivity in my ActivityInstrumentationTestCase2 instance.
But this approach requires adding some if statements in application code what makes code less clear and more complicated.
Do you have any better suggestions? Thanks in advance.