We reform how you...
Test automation has become “easy”, there are lots of tools and lots of people who have used the tools. It has become pervasive, but is it really delivering? Has is made that “game changing” difference to the way testing is performed? In some cases yes, but in many, where conditions are less favourable, standard approaches to automation do not deliver all that could be achieved. In many cases they deliver very little apart from an ability to say “we use automated testing”.
Often automation is a numbers game, the wrong numbers game. A simplistic focus on the percentage of tests automated, declared as ever increasing in regularly circulated slide decks. The measure itself is susceptible to gamesmanship, replicating easy to automate tests to swing the percentage having no real impact on what has to be done manually.
Even when the spirit of the measure has been complied with, a high score will not necessarily equate to a significant benefit. Automating 70% plus of the execution of manual test cases may only save 20% of the effort and little time. Automation needs to be a numbers game but the numbers need to measure impact not test cases.
Strategic test automation aims to deliver a sustained long term benefit that grows over time. Scripting is but one part of such an endeavour. The five main elements of a strategic automation operation are:
You want to use automation to support a step change in the way deliver software solutions. You recognise that automation of simple low level tests, run as part of the code development process, is not the complete answer. You need to automate higher level, more complex, end to end and cradle to grave tests that work across your, non-trivial, mixed IT landscape. You are interested in real, sustainable, results and not a simple test numbers game.
We have a broad perspective. We understand what will make a solution, in the wider meaning of the word, succeed. Its not just running tests, its evidence and management information and maintenance and environment mobility. We are experienced building the team practices required to run a successful service. We have engineering depth, our test solution engineering experience delivers solid tool chains that operate reliably, have the required capabilities and support efficient test creation and maintenance.
By bringing our capabilities to bear you can address your strategic automation challenge. We can deliver the solution and build your people’s ability to operate it for the long term.
Enterprise Test Automation - The global systems integrator responsible for this programme had not succeeded with test automation. Three years on, despite pressure from the client, attempts to exploit test automation had not delivered. We worked collaboratively, directing resources seconded from the integrator, to exploit automation in ways that maximised the benefit to the programme. Rather than playing the script “numbers game” a careful targeting approach, one reviewed on a regular basis, was used to shape a rounded solution. In particular, key success factors were identified and addressed to provide a firm foundation. These factors were:
The attention paid to the “bigger picture” issues around automation led to an innovative and robust approach. Establishing a full ecosystem within which automation could thrive and on delivering specific benefits delivered highly successful outcomes.
The Framework Paradox So the answer is a framework, is it not? Well that depends on what you mean by a framework. If you mean a spreadsheet, or similar, offering a standardised, and thus limited, vocabulary to be used to trigger a tool then… No. However, if you mean a establishing an operating model and supporting sets of processes and then implementing IT to support this work, including where appropriate automatically executing tests on other IT systems that are being developed and maintained, then yes frameworks are needed.
Strategic, sustainable automation needs, at least, three “frameworks”. These are:
The last of these is closest to what is generally referred to as a “framework”. The difference is that the tool chains need to selected for a purpose rather than selecting a tool chain and then finding out what it can do.