Introduction

The majority of the test packages created will be intended for a fault finding role. Their purpose is to find faults in the system. This is the traditional and obvious role for test packages. In an Assurance Analysis Led Approach ( see Software Assurance Analysis ) these packages deliver tests to address the Assurance Requirements. However in addition to fault finding in new or changed software there are other roles for test packages. Appropriate test packages can be used as project management tools, as project control tools and to assist the development effort. As they have different requirements these roles are not generally addressed by the fault finding packages and additional packages need to be defined and created.

Packages for Fault Finding

The packages created for the primary fault finding role are created because there are clusters of Assurance Requirements suitable for testing in a single package. This is described in Software Assurance Analysis.

Additional Roles

In addition to the fault finding role there are a number of highly diverse uses for tests, some of which are more commonly recognised than others. These roles include:

• Regression tests that check for unforeseen side effects of changes.
• Acting as a quality gateway to allow an item to be promoted within a configuration management process.
• Acting as an initial check for entry into an expensive process.  For example a test that checks the suitability of a build for use in system testing when introducing a new build that is unstable could have major cost implications.
• Providing a framework for developers to work against in test led development.
• Giving project management an assessment of the completeness and stability of a system.

The test manager should address all of these additional roles when selecting the test packages to be produced. The benefit, practicality and cost of producing a package for each candidate use should be addressed. As with any management activity this requires a trade off between competing demands for resources. This trade off is not limited to these additional roles but also involves the main fault finding role. The development of packages for these additional roles reduces the resources available to develop packages for fault finding roles. A judgement has to be made as to the distribution of resources that will best support the project objectives.

The Diverse Nature of Test Packages

It is important to recognise that different roles demand different things from a test package.  Even where the packages test a common target each role can demand a different test package. This is because the roles place different priorities on factors such as:

• The ease with which the package can be set up and run.
• The amount of manual action required during the execution and the levels of skill required.
• The length of time taken to execute the package.
• The amount of analysis / interpretation of the responses required, the skills required to do the analysis and the time taken to do this analysis.
• The extent to which the target is exercised by the package - the coverage of the package.
• The depth and extent of monitoring & checking of the software under test.
• The impact of target changes on the operation of the package and the ease of amending the package to match target changes.
• The timescales for delivery of the package as they must be available when the project requires them.

Thus the different roles lead to different requirements on a package. They may use common elements but the characteristics for each role should be defined separately and the delivery of the package managed as a separate objective.