I'm new to testing.I've developed a application,now i need to test.I googled about testing for some time,learnt ,what different types of testing are there in general.I wrote few test cases.
Three things,i would like to know,
Is there any different types of testing for android,if yes,can you give me some links which could help me to refer.
How do generally a android user test his apps,Will he uses test frame works or generally write test cases and testing that on real phone to see how they are performing.
Is there any sample test cases written document which will give me some basic idea.
For integration testing I use Robotium. It is a nice convenient layer on top of the build in instrumentation testing. These Tests need to be running in an emulator or on a real device. It is recommended in both cases to have an extra test project (producing an additional APK) that depends on the project under test.
Personally I like to partition my app so I have one or more libraries that do not depend on Android specific classes and can therefore be tested in a regular JVM using JUnit.
There is a third way to test and that is by mocking the android classes and have the tests run in a JVM. I have not yet used it but I hear Roboelectric is a framework that allows for this kind of testing.
Android Monkey tool can be a handy little tool. I find it handy the pseudo random fashion is handy for generating unusual use cases.
http://developer.android.com/guide/developing/tools/monkey.html
Related
I'm taking over an android project and I wish to introduce unit tests to the project, to help avoid possible regressions.
For normal java projects, I have two source folders: src and test. The src source folder contains all of my source files and my test source folder contains all of my unit tests, which I believe is pretty standard for keeping test separate from the source, so you don't have to ship with them.
I've been doing some reading online and the approach with android apps looks to be a little bit different. Several examples talk about setting up a 2nd project for an android test project and then referencing the android project.
I wish to confirm a few things:
Is having a 2nd project for the testing the appropriate thing to do when it comes to testing android projects or am I just finding bad examples?
Should all unit tests be android unit tests? E.g. Yes they should all be, or no I should mix between android unit tests and junit because junits have less overhead.
What additional benefits do android unit tests give over junit tests? E.g. Handles to the emulator, etc.
Is having a 2nd project for the testing the appropriate thing to do
when it comes to testing android projects or am I just finding bad
examples?
Yes, it's typical to have a separate "test project" for testing Android specific code. http://developer.android.com/tools/testing/testing_android.html
Should all unit tests be android unit tests? E.g. Yes they should all
be, or no I should mix between android unit tests and junit because
junits have less overhead.
You'll usually have a mix because you can't test Android specific code on a standard JVM with regular ole' JUnit (not without some helper libraries, more on that in a moment).
In practice I've found that it makes sense to divide your application into plain JVM components and Android portions. For instance, if you need to communicate with a REST API you can have a separate component that does only this and which is plain Java. These types of components can easily be tested with standard JUnit. This type of architecture also leads to a clearer separation of responsibility and often and easier to understand and maintain design as well. (Such components can be included in your Android app as regular JARs.)
What additional benefits do android unit tests give over junit tests?
E.g. Handles to the emulator, etc.
Android testing can be slow and painful because a full Android test runs the Android stack in an emulator (or on a device). Android tests are however necessary for testing the parts of your application that are Android specific, such as Context/Activity/Service and so on.
Because of the cumbersome and slow nature of native Android tests several frameworks have been created that mock or stub parts of the SDK and take different approaches to help. You may want to look into Robolectric and Robotium, for instance. (These each have their own pros and cons.)
Wondering which one is the better choice to write unit test cases for android apps and libraries: Using Robolectric library or sticking with Android Testing framework. I want to run test suite at commandline and want it be independent of need of configuring emulator or letting a device attached with build machine. Does anyone of you run a comparative analysis on both of these or something better? Your experiences will be great help me to decide on the better solution.
I use a tiered system, where I prefer earlier tiers where possible:
Pure unit tests. I try to make as much code as possible fully independent of Android APIs, and then use "pure" unit tests which can run on any JVM. These tests are the fastest, and it helps keep code that has no need to be Android-specific portable.
Robolectric-supported unit tests. Where my code has only small dependencies on Android APIs, that can be satisfied by Robolectric shadows, I test it with Robolectric. There is a little more setup time for Robolectric compared to pure tests, but it's still faster than starting/running on an emulator.
Android framework tests. Where Robolectric doesn't cut it - either because the shadows don't exist, or because I'm heavily using Android APIs (and therefore want to test against the Real Thing) - I write test that run on the emulator/device with the default framework.
The point of the tiers is to keep things as simple as possible, which keeps the full suite faster and helps promote cleaner code.
I worked on both, what i found is :-
1) Robolectric do not support API 19, it's mention in its document -
http://robolectric.org/eclipse-quick-start/. It's a great
disadvantage of it.
2) Robolectric run on JVM not on DVM. So we can not
detect that on that particular time GPS is enable in device or not
etc. We can only pass our pre-decided value for it.
3) Code writing in Robolectric is complex than junit specially for
fragment there are lot of complexity and issues.
4) Robolectric needs external jar and configuration and for junit test
we do not need any external library.
5) Robolectric is faster because it runs on JVM but this have
disadvantage too, we can not see UI on our device, what screen code
is executing.
For Android, i like jUnit test.
I am using AndroidAnnotations(2.5) in a sample project I am currently working on.
Since you can annotate your classes with for example #EActivity,#ViewById,#Click which will all lead to generated compile-time code, I was wondering how one would go about creating unit tests / functional tests for any Android Annotations powered application.
I would love to hear some opinions on this matter.
Cheers,
I responded to a similar post here.
There are a couple of options available to you. You can, of course, test your code pre-generation in, what I claim, is a more unit testing style. This should test the Java code in isolation, preferably without generated code involved.
You can also test the code post-generation. The MyActivity_ classes generated by AA can be instantiated directly after compile time and test them accordingly. I claim this is edging towards an integration testing style.
While, I think it is always better to test than not to test, I think for integration tests, you should test on the hardware in a situation similar to production. This gives you a total picture of how your application will behave in a real world situation. Thus, for integration tests I prefer high level "is everything working well together" tests.
Robolectric and Robotium can help greatly in these two efforts. Robolectric allows you to instantiate you Activities in a unit test, while Robotium allows you to test selenium style directly on a device.
To recap, I prefer to heavily unit test my code without generation then do some light integration testing to make sure everything is working well together.
I have been developing Android application for a small company and during the development process we need to do repetitive testing of some modules, So i searched tools for doing automation testing (unit testing) of the app. Android has a unit test tool however to write those test cases will itself take more time then to actually test it by hand.
I found some apps which do some great stuff and provide good charts for example Robolectric, robotium, fonemonkey4android, but am confused to what to be used, any one with any experience with the same can help.
I checked for previous questions on the similar terms like below
https://stackoverflow.com/questions/522312/best-practices-for-unit-testing-android-apps
But all the threads are very old and not so informative to decide on which to choose..
I think first you need figure out which part of your code you want to test.
For codes which doesn't related to user interface, you can test them with Robolectric. With Robolectric, the unit test code is the same to those written for java application. But it's not suitable for test ui components.
If you want to test ui, then you can choose robotium. But i always doubt whether it's worth writing tests for ui, they change too often..
I am new to Android test frameworks ,Would like to know the differences between existing test frameworks : Monkey , CTS ,Instrumentation Framework & Robotium ?
Instrumentation is a category of testing, opposite to Unit-testing.
The framework provides hooks for instrumentation testing, but you are going to need an additional third-party framework to really get going.
Robotium is such a framework. It allows you to write "scripts" that run through the user interface, saying "click this", "type that", etc. Well-written it can take you through your usecases and thus provide a good feeling that your app isn't broken. It also allows you to test multiple activities and activities interacting.
Unit-testing in my experience is very hard for Android, especially for the "regular" code dealing with UI, databases, activity state, etc., unless you write your code for testability.
The Android Monkey also uses instrumentation to run through your user interface but it does not follow a script. It does this randomly, with the idea that whatever it does it should not crash your app. By generating 100000's of events it tries to get coverage as high as possible, based on statistics. Other than Robotium, the monkey never leaves your app (that would be dangerous). It's a perfect complement though and it comes nearly for free (the setup is really cheap and there is no maintenance).
CTS is only relevant to the operating system and framework itself.
You'll probably also want to know about mocks?
Observe the testing Pyramid below:
Manual testing - self explanatory
Functional testing - testing a feature
Integration testing - checking the units play nicely
Unit tests - make sure an individual unit works as expected (See SRP)
It suggests how many tests you should have of each level. Below the pyramid are the Android frameworks that you can use at each level.
In Android, the following frameworks are commonly used for each section:
Functional:
Monkey runner "kind of" falls under this section, it basically just bashes around the app to see if any combination of interactions crashes it
Integration:
Instrumentation falls under this category.
Espresso (Made by Google, recommended, uses Hamcrest matchers)
Robotium
Unit:
JUnit4
Mockito, Powermock, other mocking libraries
Matching frameworks like Hamcrest, Fest, AssertJ
Robolectric (provides Android specific methods)