Here are two interesting propositions. Number one; test managers should focus on getting as quickly as possible to a state where it is obvious that further testing offers little benefit compared with finding out how the system survives in the wild. Number two; it is easier to make the decision to release a system when delaying the release to permit further testing is not likely to put you in any better position than you are already in. The interplay of these two propositions is discussed below.

For a number of years I was part of the programme leadership team that governed the development and release of a very large and very critical telecoms OSS system. This system was so large, so complex and so important that release decisions were never simple. We would spend a lot of time converging on a good deployment position; one that realised the maximum benefits from the release whilst containing the risks.

As you might expect sometimes making a decision was hard; things were not clear and it could go either way. These decisions often involved long debates based on uncertain information. We found that ways of thinking evolved that made decisions easier. One of the most powerful tools that evolved was a very simple question – “If we delay the release another two weeks and carry on testing then will we be in any better position to make a decision?”.

When the answer to that question was “no” we knew it was time to take a deep breath go for it and deal with any consequences that arose (and we became quite effective at dealing with those occasions when there were consequences you would not want to experience). This question worked well in that environment because the cost of not deploying was high; it was a high intensity delivery environment with a heavy emphasis on deploying and moving onto the next release. That said the question is a tool that can be used in many environments.

Returning to test managers and to their aims. If a key part of the decision to release a system is a question of the form “Can any more testing be of benefit?” then test managers should plan to get to a position where the answer would be “No” as soon as possible and to manage execution to achieve this answer as soon as possible. In doing this they accelerate delivery of the system. The sooner the answer can be “more testing is a waste of time” the sooner the benefits of the system will be seen.

Epilogue

Just to be clear. It is very easy to get the answer “more testing is a waste of time” if testing is simplistic and ineffective testing or worse is simplistic and ineffective testing executed ineffectively. This approach is not recommended. Rather do well thought out highly effective testing and do it quickly. You and your colleagues on the development side should hold similar opinions as to when the optimum point has been reached. If there is a caveat that goes something like “but we would spend more time testing if the testing were better” then there is some need for improvement.