We've built an app using Cordova. Automated tests are implemented using Protractor and Appium, followed this guide. This works OK, but I'm not sure if this is the best way. We run into some random failed tests.
Then I read this on the WebDriver Github site:
It is possible to use wd to test mobile devices using either Selenium or Appium. However in either case the full JsonWire protocol is not supported (or is buggy).
If I read this correctly, using WebDriver with Appium is buggy.
But even Appium has examples where they are using WebDriver, see here.
I'm not sure what the alternative is, maybe WebdriverIO? But I can't find much documentation about it. Or did I read it wrong, and should we be using wd without issues here?
In answer to your specific question, Protractor, Selendroid, and WebdriverIO (amongst others) are all JavaScript implementations of (Selenium 2.0) WebDriver - which is a layer/protocol/API through which you can query and control Appium. Bindings exist in all sorts of languages: python, Java, etc.
So, yes, it is perfectly possible to use Protractor and Appium together.
(I wouldn't personally, because I still associate Protractor with AngularJS applications, plus it is by no means the only game in town for JavaScript WebDriver. However, that is not down to bugs or lack of support.)
I'd definitely want to query the quote on Mr. Christian's site - which not the official WebDriver Github site by any means. It's not wrong, but the WebDriver spec is big, and hardly anyone implements all of it. So I'd take the statement with a pinch of salt.
Having established that your basic approach is not at all wrong, it would be good to explore your random failed tests, and to work out where the issues lie.
Certainly as far as newbies go, a very large proportion of "random" failures are completely avoidable race conditions within their test code, but even more experienced users will have to deal with the same problem in other forms.
Related
What I mean is automating test writing itself. Tester can perform some actions, they get recorded, and then can be used as a base for writing tests. I wasn't able to find any existing solutions, is it that hard to implement? Or did I miss something?
I mean it might be as simple as logging all touch/key events, and write tests using for example https://github.com/square/javawriter
UPD: I should've mentioned that I know of standard testing approaches:
UiAutomator
Roboelectric
Espresso (android-test-kit)
But that wasn't what I meant. Writing tests by hand is a pain, and I wander if this process can be simplified.
UPD1: If anyone stumbles apon this question, this is what I meant. It's non-free, so I'd gladly check out free/opensource analog.
You can try to use Appium (iOS/Android) which is free and has a feature that lets you record actions, and run them through. However this only works for UI functionality and any standard Unit tests will still need to be developed the standard way (manually). As a note, Appium is new (~1 year old) so there may still be bugs (such as not running in iOS devices with Xcode 5.0.1 +)
appium.io
I feel it's worthwhile to mention that appium doesn't require any modifications/additions to your android/iOS projects like a few of the other solutions I have found.
Please Calash https://github.com/calabash which is open and compatible with both iOS and Android.
I'm interested if someone already tried to use thucydides for acceptance tests for android application.
It looks like we could replace WebDriver with Robotium for example.
Let me know if question is too generic.
I have no experience of using thucydides but I believe from having a quick look you are going to have a lot of issues.
Basically the problem is that robotium runs on the phone and requires an android thing called instrumentation to run which you only get from extending certain classes.
This does not fit in to the way that webdriver works on android which is a test running on your machine which sends commands to a server on the phone.
Robotium is working on a new version that will work similar to webdriver and this will help you but currently this is in beta and has been for a while. What might actually be of help to you though is some projects done by some people at eBay: http://calabash-driver.github.com/
This gives a selenium like interface for you to use and will probably work as you want.
I have been looking at the different packages for python on android and IOS as i am going to be upgrading my phone soon.
However, i dont know which OS to go with. For android they have the SL4A package, but for IOS they have their equivilent. However, im stuck on which to go for. Does anyone have any suggestions for this?
I know its kind of a stupid question, however im just getting into mobile development since they allowed scripting languages on the mobile platform.
The python support for Android is pretty good, but still under development. You can do things that a scripting language can but not the stuff that Python as a OO language can.
Example:- When you go through the api reference here, you can see that one can write scripts to do basic to complex tasks, but, in a restrictive way. You can never build full blown apps, and even if you do, it will always be slow because the SLA4 is just another layer on top of few other layers in the Android hierarchy. There have been a lot of fun stuff that has been done using Python scripting, example: http://www.wired.com/gadgetlab/2010/07/nexus-one-phone-rides-a-rocket-up-28000-feet/#ixzz0v7LFT7ay (A bit over-board and crazy, but awesome nonetheless.)
I don't have much experience on an IPhone to pass on such comments. But, an IPhone is expensive. You can always get a used Nexus phone and do all kinds of Python hacking/scripting. All depends on your budget and your need.
That is all.
I had been on the lookut for Python on iOS for sometime. About 2-months ago, one one released to the iTunes app store -- http://pythonforios.com/ and it is fantastic! It includes the complete set of Python 2.7.2 documentation -- which alone is worth the $3 price.
It's a bit of a challenge to code any serious apps (which I suspect is the result of the developer having to comply w/ Apple's app guidelines), but it is a perfect place to test ideas from a train or airport (when you are without a laptop).
1 caveat -- I've only tried it on an iPad. It could be a bit cramped screen real-estate on an iphone.
Pythonista is an excellent alternative for the iOS platform - here's a nice article on it. And this one details how you can achieve workflow automation on the iPAD.
Knocking my head against the wall trying to figure out a good workflow for sharing code between a few WP7, iOS, and Android applications, all implemented in .NET/Mono. Anyone has figured this out?
I have some experiments in code sharing that you can see here: http://granite.codeplex.com/
What I've learned so far:
Pure business objects are your best friend. Pushg as much logic as possible into these.
Testing frameworks on mobile devices are just about useless, but you can unit test the heck out of your business objects using the desktop platforms (.NET or Mono).
Mono has more bugs that .NET, so make sure you run your unit tests against it.
Make sure you use a source control that works with MonoDevelop. That means no TFS.
Make sure you use a Unit Test framework that Mono supports. (e.g. NUnit)
View-models can be shared with the right abstraction layer, but it may not be worth it.
I use SVN in order to code in Mono on PC/MAC/LINUX - tho i rarely do anything on Linux other than POC's.
As far as workflow for sharing libraries between all of these different systems - good luck. All i can offer is:
- Be sure to unit test as Jonathan pointed out
- Know all of your compiler hints that are available
- Avoid P/Invoke or any O/S specific calls if possible
- Pray
Good luck :-)
I'm looking at using Frank for UI testing our iOS application(s). Is there a similar library with support for Android? I'm currently using Robotium, but thought it would be nice to be able to specify tests in the same format across both platforms.
Would it be possible to get something like JBehave to run on Android?
As scompt.com said. The project is now officially open sourced and is called Calabash.
I wrote a short blog post about it:
http://blog.lesspainful.com/2012/03/07/Calabash-Android/
Calabash is also available on iOS and with a bit of work and two similar apps you can run the same feature on iOS and Android.
You can read more general stuff about Calabash here:
http://blog.lesspainful.com/2012/03/07/Calabash/
The guys from lesspainful.com have open sourced some of their special sauce on github. It lets you do cucumber testing on android in the emulator and local devices. Because it's the same code that they're running for their service, you should be able to test your features on multiple devices if you sign up.
Could you use Cuke4Duke to drive Robotium? Or use JRuby/Cucumber to drive Robotium?
No, robotium is a BDD on dvm not jvm thus no driver exists with jvm frameworks such as jbehave, etc
I've recently started using www.LessPainful.com - there are some limitations (you can't write your own steps) and it's quite an early stage company, but so far the testing has at least proved useful, especially with different screen sizes, plus they've automated some good things like orientation (I think they use motors on real phones!).
(for reference on iOS we're using Frank - https://github.com/moredip/frank- and on WP7 we've written our own - https://github.com/Expensify/WindowsPhoneTestFramework)
I've been able to open-source my work in this area:
https://bitbucket.org/proxama/windows_android_cucumber_runner
https://bitbucket.org/proxama/android_cucumdroid
It allows you to write your own features and steps to exercise the UI automator. This means it runs all on the device.
It's probably not as easy to use as it could be but the Windows gui app is designed to wrap the ANT tasks that make it actually run.
I have tested my app using MonkeyTalk. It may help you. https://www.cloudmonkeymobile.com/monkeytalk