This article was originally published in Project Manager Today magazine The Benefits of Smarter Testing

Organisations that treat testing as a necessary evil, a thing that can be done mechanically, reap what they sow, according to Neil Hudson, CEO at SQC.

Expenditure on ‘low cost’ delivery of poorly focussed generic activities to get a ‘tick in the box’ can actually do more harm than good. He tells Dan Matthews how testing can accelerate delivery and bake-in quality, rather than falling into the traditional trap.

Why is good quality testing important?

Why is testing important? It is important because it is the eyes and ears of project governance. Sceptical testers can keep everyone honest, but only if the testing is of high quality. It is important because, working under pressure, a collective delusion, that ‘everything will be alright’, can develop and the results of testing can cut through this. On top of this, it is the thing that prevents the gradual drift away from a ‘quality’ product to a ‘shoddy’ product, the drift that occurs one oversight or compromise at a time.

That said, this is not easy. It needs informed, experienced people. It demands the use of a diverse set of approaches tailored to specific objectives. On the surface, to the uninitiated, it may appear that ‘a test is a test is a test’, but if that is really the case then things are not good.

So, let’s illustrate this, consider testing a complex business process on an enterprise system verses testing a self-registration service on a website. The former has a more nuanced notion of ‘quality’, it depends on the training and discipline of the users, it has a much more variation and involves significant judgement. It can only be tested by a someone who ‘gets’ the business context. In comparison, the website registration testing is much more ‘black and white’, less complex and easier to test. The two would warrant very different approaches. One of SQC’s specialisms is helping organisations to get the benefits of a more informed approach to testing. We can help educate and transform how existing teams operate or we can work within a project to provide testing as a managed service. We bring new approaches and ideas into play, but in an industrialised rather than an academic way.

Why is testing often poor value for money?

People generally look at testing and say it is too expensive. It is clear that, often, this is a fair complaint. There is a lot of money spent on mediocre testing, an entire industry sprang up around the Y2K – the so-called Millennium Bug – issue.

When you look at the actual content of the work, something we do a lot of, you see naïve choices, gaping holes and low-quality practices. It can be argued that as the industry has scaled up, any engineering discipline that underpinned testing has dissipated. The dominant factor being an individual’s ability – and that is highly variable.

This situation drives people to look for lower unit costs to attempt to balance the situation, to get their money’s worth. Ironically, this can make things worse, with lower skills and less intellectual rigour leading to poorer outcomes and a worse imbalance between expenditure and benefit.

What are the root causes of low value?

We see many factors that contribute to low value and missed opportunities. Too little thought and clarity around what is to be achieved and how to go about achieving it.

On the surface it looks simple; in reality finding the best way to test is an extremely complex problem that demands intellectual rigour. There is a lack of adequate practices that people can use to deal with this complexity. As a result, the complexity gets ignored and the exercise becomes cosmetic and not useful.

The low status of testing in many places is another major contribution. Who are the ‘superstars’ in your organisation? Who gets listened to? Who dominates the conversation? The technical people, the developers or the testers? This may be a personality thing, but I am not sure if it is nature or nurture. Are testers quiet by nature or have they had it knocked out of them? Either way, ‘low status’ means lack of influence and that means not being able to shape things for success.

Leadership is another factor. Testing has to be led in order to deliver high-value. It must take positions and hold them. Where it simply ‘goes with the flow’ it does not make effective contributions and then, ultimately, gets much of the blame when things go wrong. To do this the people leading it need to know the business, understand the technology and know the art of testing. Then they need a sound model for people to work to. Again, a lack of this level of leadership means low value outcomes.

What does it look like when it is ‘great’?

Firstly, the test operation should be an equal player to other aspects of the business. It has the respect of the other disciplines and an equal voice at the table. It helps to shape the agenda and the road-map of future work.

Then there is what is does. It is based on science; on a well thought out, fit for purpose of analysis, selection and action. It is not just a set of individuals going off and doing what they think is right and that they can fit into the time available.

Test engages from the start, not cosmetically, but with people who have the experience and aptitudes to contribute and influence during the early stages.

Test has an ‘esprit de corps’ and an ethos. The people are passionate about what they do and are constructively assertive. They are often the ones doing the pushing rather than waiting to be pushed by a project manager.

What does it give?

The test operation can become the engine at the heart of the delivery machine. A pump, pulling in work from earlier stages and injecting it into production. A bit like a Kanban model, regular slots of testing to be filled up and sent on their way to production.

It can raise the game when it comes to quality. Don’t believe it when people say, “you can’t test quality in”. Sometimes there is no alternative. A test team with a real understanding of what is important, and of what is not, with a robust attitude, good leadership and management commitment can drive up quality in virtually any situation. It may be hard work, it may take time, but it can make that key difference.

Transparency, testing can provide the transparency desired and avoid the surprises feared by many Programme and Project managers. It has to be done well. Cosmetic testing will give false positives, but effective well-grounded testing will provide a true indication of a projects status. It might not give good news but it will give time to act