What I mean is automating test writing itself. Tester can perform some actions, they get recorded, and then can be used as a base for writing tests. I wasn't able to find any existing solutions, is it that hard to implement? Or did I miss something?
I mean it might be as simple as logging all touch/key events, and write tests using for example https://github.com/square/javawriter
UPD: I should've mentioned that I know of standard testing approaches:
UiAutomator
Roboelectric
Espresso (android-test-kit)
But that wasn't what I meant. Writing tests by hand is a pain, and I wander if this process can be simplified.
UPD1: If anyone stumbles apon this question, this is what I meant. It's non-free, so I'd gladly check out free/opensource analog.
You can try to use Appium (iOS/Android) which is free and has a feature that lets you record actions, and run them through. However this only works for UI functionality and any standard Unit tests will still need to be developed the standard way (manually). As a note, Appium is new (~1 year old) so there may still be bugs (such as not running in iOS devices with Xcode 5.0.1 +)
appium.io
I feel it's worthwhile to mention that appium doesn't require any modifications/additions to your android/iOS projects like a few of the other solutions I have found.
Please Calash https://github.com/calabash which is open and compatible with both iOS and Android.
Related
I'm attempting to add some test automation to my XF app, but I'm having some difficulty as I've never used Appium before so I'm hoping that someone with experience may be able to help with my issues.
When I'm setting up my tests - if I set the "automationName" to be "UiAutomator2", then I can launch my app without problems and interact with the UI fine.
I need to do some tests with specific methods inside my app, but Appium only allows interaction with the UI. Doing some searching, I found that Appium contains an Espresso driver which is supposed to allow interaction with specific methods (this sounds exactly what I'm after).
The suggestion is to change the "automationName" from "UiAutomator2" to "Espresso" - the example I was going through in the Appium documentation was a simple test that just launched the app. It suggested that just by changing this setting, that the same test would work regardless.
I have created a simple test that just launches my app - this works fine with "UiAutomator2" but as soon as I change to "Espresso", my app doesn't launch.
The Appium server mentions not being able to find the signed .apk.
According to the documentation, this should all be working fine but as it isn't I'm guessing that there's something else that I need to do so I'm hoping that someone with experience with this will be able to help me get the tests to work.
I don't know whether I need to add a reference to Espresso somewhere in my XF app or if I need to build the .apk in a special way that will make it work.
Any advice or help on this matter would be grateful.
IMHO there are better options to achieve what you want. I'd always test the methods that are not platform specific separately, e.g. with with NUnit. You can run Unit- and Integration-Tests created with NUnit with the NUnit GUI or ReSharper if you've got a license. Unless you are using very exotic functions (or relying on timings, etc.) this should work well across all platforms.
If you are not sure that your methods will behave the same across all platforms (better to be sure), I'd create a separate App that runs tests on those methods on an actual device. There is a runner for NUnit-Tests for Xamarin.Forms, you could use for this purpose. The last release is from 2017, but I'd give it a shot. This way you can make sure that your methods work as intended, but avoid having to access them via a framework that is aimed at UI-tests.
I have an Android app which I am starting to run functional tests with. I'm currently looking at using Calabash and Cucumber style tests written in Ruby. What I'm looking for is a good way to mock out the backend when running these tests, so I can control the responses, and set up tests for various scenarios (i.e set up things for the happy path as well as various error paths that might occur). It'd also be nice if this could work whether running in emulator, Genymotion, or through a device that is connected, but if it will only work on one of the first two, that'd be fine.
If I could get the setup and tear down of this solution integrated into my Gradle build scripts, that'd be fantastic.
I have seen others use Factory Girl for mocking backend. You can see the readme at http://rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md
Or you could have a look at http://mock-server.com/ and see if that could help you out.
I'm interested if someone already tried to use thucydides for acceptance tests for android application.
It looks like we could replace WebDriver with Robotium for example.
Let me know if question is too generic.
I have no experience of using thucydides but I believe from having a quick look you are going to have a lot of issues.
Basically the problem is that robotium runs on the phone and requires an android thing called instrumentation to run which you only get from extending certain classes.
This does not fit in to the way that webdriver works on android which is a test running on your machine which sends commands to a server on the phone.
Robotium is working on a new version that will work similar to webdriver and this will help you but currently this is in beta and has been for a while. What might actually be of help to you though is some projects done by some people at eBay: http://calabash-driver.github.com/
This gives a selenium like interface for you to use and will probably work as you want.
I have googled about testing uni and systematic testing. But I am confused what exactly is systematic testing? I’m developing a mobile app and like to know how I can approach this technique testing.
Any ideas? Thanks in advance.
Systematic testing is a much more exhaustive means of debugging software. The goal is to test the software against a variety inputs in order to find as many defects/errors as possible. The difficult part about systematic testing is usually coming up with the inputs/situations to test against (especially in the case of Android development, as a number of things can go wrong). Does your app work correctly after configuration changes? Do your EditText boxes specify the correct inputType? Systematic testing provides a means of uncover such errors, and often results in a more secure/robost application.
The Android team provides several tools that can automate this kind of testing (although obviously there is no guarantee that it will cover ALL of the cases). One such tool is monkeyrunner. Check out the Android developer's guide on testing for more information.
A systematic test is a test, in which
the setup is defined,
the inputs are chosen systematically,
the results are documented and evaluated by
criteria being defined prior to the test
I'm looking at using Frank for UI testing our iOS application(s). Is there a similar library with support for Android? I'm currently using Robotium, but thought it would be nice to be able to specify tests in the same format across both platforms.
Would it be possible to get something like JBehave to run on Android?
As scompt.com said. The project is now officially open sourced and is called Calabash.
I wrote a short blog post about it:
http://blog.lesspainful.com/2012/03/07/Calabash-Android/
Calabash is also available on iOS and with a bit of work and two similar apps you can run the same feature on iOS and Android.
You can read more general stuff about Calabash here:
http://blog.lesspainful.com/2012/03/07/Calabash/
The guys from lesspainful.com have open sourced some of their special sauce on github. It lets you do cucumber testing on android in the emulator and local devices. Because it's the same code that they're running for their service, you should be able to test your features on multiple devices if you sign up.
Could you use Cuke4Duke to drive Robotium? Or use JRuby/Cucumber to drive Robotium?
No, robotium is a BDD on dvm not jvm thus no driver exists with jvm frameworks such as jbehave, etc
I've recently started using www.LessPainful.com - there are some limitations (you can't write your own steps) and it's quite an early stage company, but so far the testing has at least proved useful, especially with different screen sizes, plus they've automated some good things like orientation (I think they use motors on real phones!).
(for reference on iOS we're using Frank - https://github.com/moredip/frank- and on WP7 we've written our own - https://github.com/Expensify/WindowsPhoneTestFramework)
I've been able to open-source my work in this area:
https://bitbucket.org/proxama/windows_android_cucumber_runner
https://bitbucket.org/proxama/android_cucumdroid
It allows you to write your own features and steps to exercise the UI automator. This means it runs all on the device.
It's probably not as easy to use as it could be but the Windows gui app is designed to wrap the ANT tasks that make it actually run.
I have tested my app using MonkeyTalk. It may help you. https://www.cloudmonkeymobile.com/monkeytalk