I'm trying to set up continuous integration with an Android project and Cucumber.
The idea is to write tests in Cucumber and run the tests on my Android build via Cuke4Duke and NativeDriver for Android.
When I have this running, I plan to use Maven and a Jenkins server to automate the testing, so it is run every time i commit to the Subversion repo.
Has this been done before? Is there a good guide somewhere? Or is it a bad idea to do it this way?
We're doing exactly what you're planning to do with Maven, Jenkins, and Git. The missing ingredient is the android/cucumber integration from lesspainful.com.
I don't think that what you've planned is a bad idea. But I don't know of anyone that's doing Android CI with that particular setup.
You may also want to take a look at Robotium, it's like Selenium for Android and offers a very rich DSL that will help out with your cuke4duke step implementations.
In my company we use a little different setup (but probably you will have to solve similar challenges): Jenkins + Jenkins Android Plugin + Robotium + Ant. We find out that ant is hard to maintain when you try to use it to something more complicated then simple build and we are rewriting our scripts to gradle.
It works quite well, however you should be aware of two potential problems:
1. emulator is slow (even on fast server) - you can consider attaching physical device to your server.
2. you probably have to setup lock (or use only one executor) for emulator since using multiple emulator instance is hard/tricky.
What we have done is write an test instrumentation engine above Robotium. This engine is mainly a state machine reading keywords from a text file and converting them into Robotium API calls. We did initially notice that inputs and outputs were the same: user taps on the screen, a new screen is displayed or new text is displayed.
That allows us to implement keyword test driven but it runs on the device so not remotely.
It is 20% of effort to get 80% of the benefit: easy to write/add new tests that are readable by anybody. Of course there are limitations but our goal was achieved.
Cheers
Ch
Related
I'm attempting to add some test automation to my XF app, but I'm having some difficulty as I've never used Appium before so I'm hoping that someone with experience may be able to help with my issues.
When I'm setting up my tests - if I set the "automationName" to be "UiAutomator2", then I can launch my app without problems and interact with the UI fine.
I need to do some tests with specific methods inside my app, but Appium only allows interaction with the UI. Doing some searching, I found that Appium contains an Espresso driver which is supposed to allow interaction with specific methods (this sounds exactly what I'm after).
The suggestion is to change the "automationName" from "UiAutomator2" to "Espresso" - the example I was going through in the Appium documentation was a simple test that just launched the app. It suggested that just by changing this setting, that the same test would work regardless.
I have created a simple test that just launches my app - this works fine with "UiAutomator2" but as soon as I change to "Espresso", my app doesn't launch.
The Appium server mentions not being able to find the signed .apk.
According to the documentation, this should all be working fine but as it isn't I'm guessing that there's something else that I need to do so I'm hoping that someone with experience with this will be able to help me get the tests to work.
I don't know whether I need to add a reference to Espresso somewhere in my XF app or if I need to build the .apk in a special way that will make it work.
Any advice or help on this matter would be grateful.
IMHO there are better options to achieve what you want. I'd always test the methods that are not platform specific separately, e.g. with with NUnit. You can run Unit- and Integration-Tests created with NUnit with the NUnit GUI or ReSharper if you've got a license. Unless you are using very exotic functions (or relying on timings, etc.) this should work well across all platforms.
If you are not sure that your methods will behave the same across all platforms (better to be sure), I'd create a separate App that runs tests on those methods on an actual device. There is a runner for NUnit-Tests for Xamarin.Forms, you could use for this purpose. The last release is from 2017, but I'd give it a shot. This way you can make sure that your methods work as intended, but avoid having to access them via a framework that is aimed at UI-tests.
I have an Android app which I am starting to run functional tests with. I'm currently looking at using Calabash and Cucumber style tests written in Ruby. What I'm looking for is a good way to mock out the backend when running these tests, so I can control the responses, and set up tests for various scenarios (i.e set up things for the happy path as well as various error paths that might occur). It'd also be nice if this could work whether running in emulator, Genymotion, or through a device that is connected, but if it will only work on one of the first two, that'd be fine.
If I could get the setup and tear down of this solution integrated into my Gradle build scripts, that'd be fantastic.
I have seen others use Factory Girl for mocking backend. You can see the readme at http://rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md
Or you could have a look at http://mock-server.com/ and see if that could help you out.
What I mean is automating test writing itself. Tester can perform some actions, they get recorded, and then can be used as a base for writing tests. I wasn't able to find any existing solutions, is it that hard to implement? Or did I miss something?
I mean it might be as simple as logging all touch/key events, and write tests using for example https://github.com/square/javawriter
UPD: I should've mentioned that I know of standard testing approaches:
UiAutomator
Roboelectric
Espresso (android-test-kit)
But that wasn't what I meant. Writing tests by hand is a pain, and I wander if this process can be simplified.
UPD1: If anyone stumbles apon this question, this is what I meant. It's non-free, so I'd gladly check out free/opensource analog.
You can try to use Appium (iOS/Android) which is free and has a feature that lets you record actions, and run them through. However this only works for UI functionality and any standard Unit tests will still need to be developed the standard way (manually). As a note, Appium is new (~1 year old) so there may still be bugs (such as not running in iOS devices with Xcode 5.0.1 +)
appium.io
I feel it's worthwhile to mention that appium doesn't require any modifications/additions to your android/iOS projects like a few of the other solutions I have found.
Please Calash https://github.com/calabash which is open and compatible with both iOS and Android.
I'm new to testing.I've developed a application,now i need to test.I googled about testing for some time,learnt ,what different types of testing are there in general.I wrote few test cases.
Three things,i would like to know,
Is there any different types of testing for android,if yes,can you give me some links which could help me to refer.
How do generally a android user test his apps,Will he uses test frame works or generally write test cases and testing that on real phone to see how they are performing.
Is there any sample test cases written document which will give me some basic idea.
For integration testing I use Robotium. It is a nice convenient layer on top of the build in instrumentation testing. These Tests need to be running in an emulator or on a real device. It is recommended in both cases to have an extra test project (producing an additional APK) that depends on the project under test.
Personally I like to partition my app so I have one or more libraries that do not depend on Android specific classes and can therefore be tested in a regular JVM using JUnit.
There is a third way to test and that is by mocking the android classes and have the tests run in a JVM. I have not yet used it but I hear Roboelectric is a framework that allows for this kind of testing.
Android Monkey tool can be a handy little tool. I find it handy the pseudo random fashion is handy for generating unusual use cases.
http://developer.android.com/guide/developing/tools/monkey.html
I found couple of difference frameworks which can test Android apps, but all of them are limited on testing of just one app (because mainly they use Instrumentation)
My app contains a service which could be called by other apps and I want to automate testing of this too. So, I would like to be able to write some tests which automate UI in other apps.
Have you seen anything, except MonkeyRunner? I looked at it, but the API is quite poor.
Take a look at Sikuli IDE it's easy enough to use and is based on Python.
You can bascially run integration tests using it, (kind of like Selenium for desktops).
There is also Selenium Android Driver if you want to run automaton tests from the Android WebView!
You are right, a bare monkeyrunner is perhaps not enough, but if you combine it with other tools perhaps you can find your way. For example, AndroidViewClient is a library that adds the ability of:
finding Views by ID, very much like you normally do in your Android Activity (using ViewClient.findViewById())
obtaining the value for almost any of the Views properties (using for example View.isChecked() or View.mText())
sending touch events to the Views by simply invoking View.touch()
More details an a complete example can be found at http://dtmilano.blogspot.com/2012/02/monkeyrunner-interacting-with-views.html