Assessment of the Implemented System

The mainstay of most software project quality assurance programmes is the testing of the software. This is a major part of the assessment of the implemented system. However SQC incorporate two other elements in the assessment process. Firstly there is Assurance Analysis and secondly the use of targeted implementation reviews. The approach is described in this section.

Planning & Managing the Testing - A Risk Based Analysis Led Approach

Traditionally projects attempt to define the testing to be done in a Test Strategy. Meaningful Test Strategies are hard to write and projects often have problems working to them - the testing diverges from the strategy. Factors in this include a lack of information when the strategy is written and the difficulty of defining useful meaningful rules for generic 'classes' of testing ( see Problems With Test Strategies ). During the project the apparent existence of the Test Strategy as a route map for testing means an absence of active management. This leads to sub-optimal use of test resources and the delivery of lower priority tests at the expense of higher priority tests.

The Analysis Led Approach places less emphasis on fully identifying the testing / types of testing to be performed during the planning phase. Strategic test packages are still identified but identification of more general testing and of lower level internal testing is deferred. These details then emerge from a systematic analysis of what needs to be proved about the system. This analysis proceeds in parallel with the development of the system. A new test package is created because the analysis output has identified something that needs to be proved, a test package is required to prove it and all higher priority items have been addressed by existing test packages. This process focuses effort on the risk areas for the project.

Assurance Analysis

This is a systematic process to identify what needs to be tested or checked within the implemented system. The analysis collates information about the system and identifies & prioritises things to be addressed - these are referred to as Assurance Requirements. Clusters of Assurance Requirements are used as the basis for selecting and defining the scope of test packages to be developed. The similar requirements forming a cluster are allocated to the package and act as a definition of what is to be tested.

Testing

Testing remains a fundamental fault finding activity in any software project. Effective testing must be systematic and target the risks of the system. Our processes provide a clearly defined scope for each test package. This scope and the associated Assurance Requirements provide a basis for the systematic definition of a test set.

Targeted Implementation Reviews

Some Assurance Requirements will not be suitable for dynamic testing. It may be infeasible to establish the required level of confidence or the cost of doing so may be too high. Resources can be wasted if dynamic testing is attempted where it is not a suitable solution.

Where this is the case and where it is not acceptable to ignore the Assurance Requirement we apply targeted implementation reviews. These reviews focus on the specific assurance requirement(s) and check the implementation does not contain faults in these areas. Targeted reviews differ from normal design reviews or code reviews in that they focus on particular assurance requirements rather than a deliverable. Also targeted reviews may involve significant amounts of supporting analysis.

SQC Capabilities

We apply systematic approaches to developing each test set. Our capabilities go beyond basic GUI based testing at the system level. Where appropriate, we develop tests that target components, sub-systems and the integration of elements. We produce tests sets that assess functional and timing behaviour.  Our test design process includes threat led test design - we develop tests to target specific causes of failures such as concurrency clashes and resource exhaustion. Out test automation capabilities enable us to deliver effective re-usable test solutions. Our experience includes the testing of complex reactive real-time systems.

We have extensive experience of performing Targeted Implementation Reviews, of analysing designs and software to locate risks / faults related to particular failures or failure scenarios. This enables us to address areas that can not be addressed effectively using dynamic testing.