As of Android Studio 1.1rc there's Unit testing support and
I'm wondering what's the difference between Android Instrumentation Tests and Unit tests.
As I understand it:
Unit tests are useful for testing code which doesn't call the Android
API, and the Android instrumentation tests are rather integration
tests to test Android API specific elements or GUI components.
However if you use a framework like Robolectric or Mockito in your
unit tests, you can test Android code (without the need of a device) if I'm not mistaken.
Is this correct, or is there a bigger difference? If so, what's the use of each?
It seems to me that instrumentation testing is integration testing with the ability to control the life cycle and the events (onStart, onCreate etc) of the app.
Unit testing, as i understand it, is testing a Unit (eg Class) for its data and behavior.
For example, say you have a game: this game runs on an activity (main activity) and you have a character based on a Robot class, that has 2 methods (fire and move). You would test the main activity with an instrumentation test to see if it saves correctly when you leave the app, if it restores correctly when you restore it etc. and you would test Robot with a Unit test, to test its attributes and behavior.
Disclaimer: I'm not a java person, but i took interest in your question and i answered it based on a minor search online. You probably have to dig deeper into this to find a more detailed answer.
Unit tests isolate the component under test, and this is the reason why are often used together with Mocks frameworks as Mockito:because isolate the unit from their dependencies. Please notice what you say regarding the Android API is partially true, because there are also Instrumented Unit tests, namely Instrumentation is part of the Junit package as well, and also the classes that extend TestCase as the class AndroidTestCase is part of the Junit package but allows the use of A)Context, that you can call with getContext(), and B)Resources that are part of the Android API! Also please consider AndroidTestCase is a base class and there are several other classes quite useful that extend this class. They test specifically Loaders, ContentProviders and even Services and also them have access to the Android API. so these classes provide JUnit testing framework as well as Android-specific methods. Now with Junit4 there is the ServiceTestRule that extends directly from Object and allow you easier to test a Service, although you cannot start an Intent directly inside this class.
Instrumentation tests they are also into the Junit package, but the control of the Android API is quite total because Instrumentation Tests are instantiated in the system before any application code is run, and to test you need to open the real application(emulator or a phone connected with USB). They access to android components(ex. click a button) and application life cycle, they are usually slower than Junit tests that extend TestCase( the ones examined above) the typical use is with ActivityInstrumentationTestCase2 that has a functional test approach, more user oriented.
EDIT: Regarding Roboelectric and Mockito, that are togheter with Espresso between the most popular testing frameworks at the moment(13 July 2016),
Roboelectric allows you to run multiple tests in seconds instead of minutes, and this comes really handy in teams that have to run continuous tests, and are subject to continuous integration.
From the site of Robolectric:
An alternate approach to Robolectric is to use mock frameworks such as
Mockito or to mock out the Android SDK. While this is a valid
approach, it often yields tests that are essentially reverse
implementations of the application code.
Roboelectric allows a test style that is closer to black box testing,
making the tests more effective for refactoring and allowing the tests
to focus on the behavior of the application instead of the
implementation of Android. You can still use a mocking framework along
with Robolectric if you like.
Mockito that also can be used with Junit, is really used with the exception of when have to manage final classes, anonymous classes or primitive types.
UNIT TESTING
Unit tests that run on your local machine only. These tests are
compiled to run locally on the JVM to minimize execution time. Use
this approach to run unit tests that have no dependencies on the
Android framework or have dependencies that mock objects can satisfy.
So basically, you run plain java code to test for example a content provider, database connections, input and output of methods. This doesn't run on Android. To run it you DON'T need a device.
INSTRUMENTATION TESTING
Unit tests that run on an Android device or emulator. These tests have
access to Instrumentation information, such as the Context of the app
under test. Use this approach to run unit tests that have Android
dependencies which mock objects cannot easily satisfy.
So it mocks how the user will use the actual application, therefore you NEED a device (physical or emulator) to run it. It has access to views, activities, context, etc.
Reference: http://developer.android.com/tools/testing/testing_android.html
Android tests
[Test types]
The diagram of test types in Android world
Local Unit test: JUnit, Mockito, PowerMock
Instrumentation tests(a kind of functional tests): Espresso, Robolectric, Robotium
//Functional tests(Integration, UI)
src/androidTest/java
//Unit tests
src/test/java
[Run test via Command line]
[Test double types]
Unit Testing:
Often Unit tests are referred to as “local tests” or “local unit tests”. The main reason for this seems to be that you want to be able to run tests without a device or an emulator attached.
Unit tests cannot test the UI for your app without mocking objects such as an Activity.
Instrumentation tests:
Instrumentation tests run on a device or an emulator. In the background, your app will be installed and then a testing app will also be installed which will control your app, launching it and running UI tests as needed.
Instrumentation tests can be used to test none UI logic as well. They are especially useful when you need to test code that has a dependency on a context.
Ref link for example
Unit Testing
It Runs on the local machine only.
Instrumentation Test case
It Runs on the android device or emulator .if you check the test case it runs on emulator or android device
Unit tests focus on small portions of code (i.e. class-level methods) and provide basic validation that code is working as expected.
Espresso tests provide basic validation that the UI is working as expected.
Related
I have go through Test Android Apps in Android document. Google introduced AndroidX testing, but I am still very of confused of its implications.
I am using MVVM Android architecture components with ViewModel, LiveData.
Which parts of the Android X Framework do I need to use?
View(Activity.Fragments) --> Espresso?
ViewModel(with LiveData) --> RoboElectric?
Model --> JUnit or RoboElectric?
Why is Google still confusing us with lot of libraries, why can't they move all Local test, Instrumentation test, and everything under the same Framework(Espresso)?
The Google Testing team has made many improvements to their testing framework AndroidX.
Why Google still confusing with lot of libraries, why cant they move all Local test, Instrumentation test everything under same Framework(Espresso)?
With AndroidX Test, they are actually attempting to solve this!
First off, regular "Junit" unit Tests are unchanged. You should test your java code on the JVM the same way you were doing.
But now, with AndroidX, they have made the Robolectric 4.0 API the same as Espresso's so it is now possible to run your Espresso Tests as Robolectric tests. This means when developing locally, you can iterate more quickly because the Instrumentation Tests can run so much faster instead of having to run on emulators. It also makes it a lot easier to learn Robolectric, because you are able to use the same API as espresso.
When you get to the PR or CI phase, you can then submit those same tests that ran on Robolectric JVM to run on your CI on real devices using Espresso.
I suggested checking out Testing Rebooted (with AndroidX Test) (Android Dev Summit '18) for more context.
In this picture, the failing UI and Unit tests would represent the same test but represented as Espresso/Robolectric Instrumentation Test depending on what you want to run and when. For clarity, you would want to run it as Robolectric when you are doing local development especially with TDD. When ready to submit a PR, you can then run the same test on CI with emulator or real device.
So, while AndroidX.test can be confusing, I suggest watching this video to get a bit more of a grasp on Google's intent with the framework.
Also, I agree they can be providing more examples and guidance on using the new Androidx.Test!
I have gone through the doc of espresso but still do not get the benefits of it so that I can move to espresso over instrumentation. Can somebody help with that?
What I know from espresso is:
A key benefit of using Espresso is that it provides automatic synchronization of test actions with the UI of the app you are testing.
Espresso detects when the main thread is idle, so it is able to run your test commands at the appropriate time, improving the reliability of your tests.
This capability also relieves you from having to add any timing workarounds, such as Thread.sleep() in your test code.
Here at point 1 I am not able to get how it is maintaining synchronization while taking action.
At point 2 how threads become idle and How we can avoid Thread.sleep() is espresso wait for element until it is not shown on the screen.
Please suggest how I should go with it. Thanks
but still not getting the benefits of it so that I can move to espresso over instrumentation
I think you are confusing some things here:
Instrumentation-Test is not neccessary meaning you have to use espresso.
Generally Instrumented Tests are tests that:
...can take advantage of the Android framework APIs and supporting APIs, such as the Android Testing Support Library.
Source
This means that you can either write "classic" unit tests or instrumented unit tests that for example need an android context to run. Unit tests are tests that test single parts of your code, e.g. services or stuff like that.
UI tests on the other hand are tests that really start up your activity and perform some action in your app.
Espresso itself is just a library to help you write automated UI tests. You can use it to interact with UI within your app (click buttons, check content of views, ...).
Furthermore there is the ui-automator library which can do a similar thing but is intended for UI interactions that are not within your app but for example system dialogs (permission dialog, recent-app or home button, ...) or if you have test cases that span across multiple apps.
So to answer your question: You cannot "favor" espresso over instrumentation tests since espresso is a library used for automated instrumentation tests.
This is first time I have to write unit test cases in Android.
So I have searched lots of things.
Robolectric framewordk - Runs on JVM
Mockito Framwork - Mocking objects
So I have some doubts in Robolectric & Mokito.
Should I have to use Robolectric only with JUnit in Android app?
Should I have to use Mockito only with JUnit in Android app?
Should I have to go with both framework?
What is the difference between Mockito & Robolectric?
I have search for difference between Mokito & Robolectric but don't get any proper answer for that.
Please suggest.
They have slightly different usages and I tend to use both in my projects.
Mockito
is used for making mocks of your classes.
When you are testing a particular class you mock all of its dependencies with Mockito.
Where possible most of your tests should use mockito. To make this possible most people split their code up into MVP, etc where the business logic is separated from the View logic. This way your business logic (Presenter) has no knowledge (or dependencies) on the Android library and has no need to have mocks of them.
Robolectric
is a library which contains many mocks of Android classes.
The Robolectric test runner injects these 'shadow objects' in place of the actual Android classes when the tests are run. This is what allows the tests to run on the JVM without booting up an instance of Android.
When using MVP your View layer tends to be implemented by the Activity/Fragment and this is where you can use Robolectric to mock these.
Notes
Use Robolectric only where necessary. It basically re-implements parts of the Android framework but not always in exactly the same way.
You may also need another library such as PowerMock. This allows the mocking of static classes such as Math or can be used to mock static Android classes such as TextUtils.
Both are used with JUnit
Mockito alone can cover most cases.
However, Robolectric can also provide limited operations on Android Component such as Activity or Fragment in Unit Test (not instrumentation test, which has no dependency on Android SDK), which does not require any emulator or devices and is considerably faster than instrumentation tests.
My suggestion: use Mockito for unit test and Espresso for UI test since they are semi-official test frameworks for Android.
Add Robolectric in your Unit Test if there are some dependencies on Android SDK.
First of all we need to understand that Roboelectric and Mockito are two different tools commonly used in android Test Driven Development. So mostly you will find both of the tools in a same project.
Below I am explaining the common use cases for both-
Mockito is used for mocking the dependency which means if you want to access an real object in test environment then you need to fake it or we can say mock it. Now a days it is very easier to do mocking of the objects with Mockito.
Roboelectric is the industry-standard unit testing framework for Android. With Robolectric, your tests run in a simulated Android environment inside a JVM, without the overhead of an emulator.
Simple test written using roboelectric is
`#RunWith(AndroidJUnit4.class)
public class MyActivityTest {
#Test
public void clickingButton_shouldChangeResultsViewText() throws Exception {
Activity activity = Robolectric.setupActivity(MyActivity.class);
Button button = (Button) activity.findViewById(R.id.press_me_button);
TextView results = (TextView) activity.findViewById(R.id.results_text_view);
button.performClick();
assertThat(results.getText().toString(), equalTo("Testing Android Rocks!"));
}
}`
Wondering which one is the better choice to write unit test cases for android apps and libraries: Using Robolectric library or sticking with Android Testing framework. I want to run test suite at commandline and want it be independent of need of configuring emulator or letting a device attached with build machine. Does anyone of you run a comparative analysis on both of these or something better? Your experiences will be great help me to decide on the better solution.
I use a tiered system, where I prefer earlier tiers where possible:
Pure unit tests. I try to make as much code as possible fully independent of Android APIs, and then use "pure" unit tests which can run on any JVM. These tests are the fastest, and it helps keep code that has no need to be Android-specific portable.
Robolectric-supported unit tests. Where my code has only small dependencies on Android APIs, that can be satisfied by Robolectric shadows, I test it with Robolectric. There is a little more setup time for Robolectric compared to pure tests, but it's still faster than starting/running on an emulator.
Android framework tests. Where Robolectric doesn't cut it - either because the shadows don't exist, or because I'm heavily using Android APIs (and therefore want to test against the Real Thing) - I write test that run on the emulator/device with the default framework.
The point of the tiers is to keep things as simple as possible, which keeps the full suite faster and helps promote cleaner code.
I worked on both, what i found is :-
1) Robolectric do not support API 19, it's mention in its document -
http://robolectric.org/eclipse-quick-start/. It's a great
disadvantage of it.
2) Robolectric run on JVM not on DVM. So we can not
detect that on that particular time GPS is enable in device or not
etc. We can only pass our pre-decided value for it.
3) Code writing in Robolectric is complex than junit specially for
fragment there are lot of complexity and issues.
4) Robolectric needs external jar and configuration and for junit test
we do not need any external library.
5) Robolectric is faster because it runs on JVM but this have
disadvantage too, we can not see UI on our device, what screen code
is executing.
For Android, i like jUnit test.
I have considerable experience in making Android applications. For my new project, we have decided to do Test Driven Development (TDD). I have been getting my hands wet on Robotium for User Scenario Testing, and it works fine and looks easy too.
For unit testing, I tried to mock Context using (MockContext Android Class) but I am unable to do so. I went through this blog http://sites.google.com/site/androiddevtesting/ and through this http://sdudzin.blogspot.com/2011/01/easy-unit-testing-for-android.html , which suggests that mocking in Android apps is still very limited and hard, and have suggested to use PowerMock, jMockit, JeasyTest, or Roboelectric (in combination with Mockito and Maven) and even RoboGuice.
I would like to get any suggestions from you guys on which unit testing framework in your opinion is the best for testing Android applications. (particularly testing Android classes, possibly giving mock Contexts and other mocking features so that I can make my test cases as independent as possible). Any suggestions or pointers would be helpful . Thanks
For off-device testing, look at Robolectric
For on-device testing, look at Borachio
Bottom line: it's still very, very difficult to do well. Things are improving (the situation is dramatically better today than it was 6 months ago) but Android is comfortably the most test-hostile environment I've ever written programs for.
To do TDD in Android, I personally use all of the following:
assertj-android: assertions for Android
Mockito: Mocking Framework
Robolectric: Unit testing framework that runs without the need of Android emulator
Robotium: UI tests (Needs emulator or device to run)
Also: Using dependency injection libraries such as Dagger or Roboguice will greatly simplify your unit/integration tests. To run tests on multiple devices, consider using Spoon.
I use ActivityInstrumentationTestCase2 in the case of Activities for TDD (or BDD rather), and write normal unit tests for all logic. This also helps me separate logic from Activities.
Mobile applications by nature are UI centric. Therefore it does not
make sense to mock out the UI, even if it makes the Unit test look
like a Functional Test.
For adding Extras to intents, you can set a custom intent for the test, or do it for all tests by overriding setup.
Mocks give a lot of issues on Android, so I use stubs.
An example is given below. The Activity shows Hello World when you click a button -
public class HelloWorldActivityTest extends
ActivityInstrumentationTestCase2<HelloWorldActivity> {
private HelloWorld activity;
public HelloWorldActivityTest() {
super(HelloWorldActivityTest.class);
}
public void testShouldRenderGreetingOnButtonClick() {
activity = this.getActivity();
Button button = (Button) activity.findViewById(R.id.btn_greet);
TouchUtils.clickView(this, button);
assertEquals("Hello World!",
((TextView) activity.findViewById(android.R.id.greeting_text))
.getText());
}
}
EDIT: Things have changed since I posted the answer. Mockito now has reasonably good support for Android. And for tests, we've moved from ActivityInstrumentationTestCase2 to Robolectric, just in order to tap the sheer speed of JVM in the development phase. The Android Testing Framework is great for Integration and Functional testing, just not for Unit Tests.
To apply TDD for android, Android Testing Codelab will be very helpful to you. Code lab shows a use of the testing tool and how you can apply TDD for android.I tried it and it was very helpful to me.
Bonus: Check Clean Architecture
Android Testing Support Library provides an extensive framework for testing Android apps. This library provides a set of APIs that allow you to quickly build and run test code for your apps, including JUnit 4 and functional user interface (UI) tests. You can run tests created using these APIs from the Android Studio IDE or from the command line.
Read more about:
Espresso
AndroidJUnitRunner
JUnit4 Rules
UI Automator
Thank you :)
we have
https://developer.android.com/training/testing/start/index.html
and can able to test local(Runs on JVM) and instrumental test (Runs on Device or Emulator )
For this we need to add
Android Testing Support Library
The Android SDK includes two tools for functional-level app testing
Monkey and monkeyrunner