We reform how you...
Flawed test automation initiatives can be worse than no test automation. Day to day they waste money. Perhaps worse, they fail to deliver a strategic change, eighteen months on you find you are in the same position you were at the start and facing the need to start yet another automation initiative.
The Challenge
Test automation has become “easy”, there are lots of tools and lots of people who have used the tools. It has become pervasive, but is it really delivering? Has is made that “game changing” difference to the way testing is performed? In some cases yes, but in many, where conditions are less favourable, standard approaches to automation do not deliver all that could be achieved. In many cases they deliver very little apart from an ability to say “we use automated testing”.
A Different Numbers Game
Often automation is a numbers game, the wrong numbers game. A simplistic focus on the percentage of tests automated, declared as ever increasing in regularly circulated slide decks. The measure itself is susceptible to gamesmanship, replicating easy to automate tests to swing the percentage having no real impact on what has to be done manually.
Even when the spirit of the measure has been complied with, a high score will not necessarily equate to a significant benefit. Automating 70% plus of the execution of manual test cases may only save 20% of the effort and little time. Automation needs to be a numbers game but the numbers need to measure impact not test cases.
Strategic Test Automation
Strategic test automation aims to deliver a sustained long term benefit that grows over time. Scripting is but one part of such an endeavour. The five main elements of a strategic automation operation are:
- Targeting and shaping what automation will do
- Implementation of automation capability
- Provision of suitable environments and test data
- Operation execution of automated test
- Maintenance and continuous improvement
Your Requirements
You want to use automation to support a step change in the way deliver software solutions. You recognise that automation of simple low level tests, run as part of the code development process, is not the complete answer. You need to automate higher level, more complex, end to end and cradle to grave tests that work across your, non-trivial, mixed IT landscape. You are interested in real, sustainable, results and not a simple test numbers game.
How we can help
We have a broad perspective. We understand what will make a solution, in the wider meaning of the word, succeed. Its not just running tests, its evidence and management information and maintenance and environment mobility. We are experienced building the team practices required to run a successful service. We have engineering depth, our test solution engineering experience delivers solid tool chains that operate reliably, have the required capabilities and support efficient test creation and maintenance.
By bringing our capabilities to bear you can address your strategic automation challenge. We can deliver the solution and build your people’s ability to operate it for the long term.
Case Study
Enterprise Test Automation - The global systems integrator responsible for this programme had not succeeded with test automation. Three years on, despite pressure from the client, attempts to exploit test automation had not delivered. We worked collaboratively, directing resources seconded from the integrator, to exploit automation in ways that maximised the benefit to the programme. Rather than playing the script “numbers game” a careful targeting approach, one reviewed on a regular basis, was used to shape a rounded solution. In particular, key success factors were identified and addressed to provide a firm foundation. These factors were:
- The creation and automation of a test data supply chain.
- Preparation of infrastructure and tools to support the configuration management, build, deployment and execution of test suites.
- Establishing a disciplined operational culture and supporting processes around test execution, one that included intrinsic monitoring and corrective action activities that maintained the health of the testing.
- Implementing the right building blocks to support rapid and consistent production of tests.
- Active quantified measurement of what was being executed, how well it worked and of benefits delivered.
- Creating defined service offerings available to the wider testing community.
The attention paid to the “bigger picture” issues around automation led to an innovative and robust approach. Establishing a full ecosystem within which automation could thrive and on delivering specific benefits delivered highly successful outcomes.
Can we have a framework?
The Framework Paradox So the answer is a framework, is it not? Well that depends on what you mean by a framework. If you mean a spreadsheet, or similar, offering a standardised, and thus limited, vocabulary to be used to trigger a tool then… No. However, if you mean a establishing an operating model and supporting sets of processes and then implementing IT to support this work, including where appropriate automatically executing tests on other IT systems that are being developed and maintained, then yes frameworks are needed.
Strategic, sustainable automation needs, at least, three “frameworks”. These are:
- An operating model that can deliver work and deliver it in a way that integrates with the wider organisation.
- Well defined processes that facilitate the enactment of automated testing and automation of these where practicable.
- Test implementation toolchains that supports the effective implementation of high value tests or parts of tests.
The last of these is closest to what is generally referred to as a “framework”. The difference is that the tool chains need to selected for a purpose rather than selecting a tool chain and then finding out what it can do.