We're gearing up for the next CogniDox software release and are starting to see payback from tools adopted to help us with the process. In particular, a tool called Selenium has proven itself very useful.
Selenium (http://docs.seleniumhq.org/) is a web browser automation tool that's often used to automate the testing of web applications. It's open source software available under an Apache 2.0 license that you can download for free from the project's googlecode repository.
A web application such as CogniDox is well-suited for Selenium testing. We've been developing CogniDox for around ten years and the number of web pages and configuration options is now a staggeringly high number. We have to support the full range of web browsers/versions that are used in our (corporate) user base. We have multiple developers making changes. We use a number of third party software libraries such as jQuery UI which are themselves constantly evolving - any one change could impact us. It all requires a lot of regression testing and doing this manually would be tedious and time-consuming.
Selenium comes in two variations. We started using the Selenium IDE version some time ago as a simple way to record browser actions that you can playback (also handy for making training videos, by the way) when you want to see if a code change still works well with existing functions. That was adequate when it was one developer testing part of the system against a server, but we out-grew that.
In the past six months or so we switched to using Selenium WebDriver instead. The differences between the Selenium projects is well-explained in this blog but a quick way of explaining it is that we now have a suite of tests we can sent as commands to drive individual versions of browsers and retrieve the results. This can be done for a range of programming languages including Python, Ruby, Java, and C#. Our setup uses Python/Selenium WebDriver in a hub/node configuration.
Here's a simple example of a Selenium hub controlling a CentOS server running Firefox and CogniDox:
The Python functions can be used in a modular fashion - if a test requires that we access the Manage CogniDox page from the Homepage as its starting point, we can implement that with one line of code. These tests are also more readable, so we have more collaboration between developers and work is more re-usable.
Selenium sits well with our increasing use of cloud-based deployments and virtualisation technologies. It's a crucial element in the overall toolchain because it combines well with continuous integration (CI) tools for automatic overnight testing of each daily CogniDox build. The high level of activity around the project and the increasing support for mobile apps is also useful.
Selenium began as an open source alternative to proprietary products such as Mercury Interactive QuickTest Professional (the chemical selenium is a cure for mercury poisoning); a product now owned by HP. License cost is our least consideration, but ~$10K might be an important saving for many startup software companies. There are "true cost of open source" papers out there on the Internet that claim staffing costs are higher with Selenium IDE than with e.g. HP QTP. In our experience, that's just not true.