Introduction

This section explains two concepts:

• Assurance Target - this identifies the 'bit' of the software that is the subject an assurance activity.
• Assurance Focus - this identifies both the 'bit' ( the Assurance Target ) and the things about it that are of interest.

The Assurance Focus is an extension of the Assurance Target, it adds information on the behaviour that is of interest.

An objective when defining an Assurance Focus, which incorporates an Assurance Target, is that it should provide the basis for a coherent package of work and should be fairly unambiguous. The approach given here is intended to meet these objectives. They have been developed in response to difficulties encountered when attempting to document / communicate the scope and intent of analysis and testing activities.

Assurance Target

The role of Assurance Targets is to identify a 'bit' of the software. So what is a 'bit'? How is a 'bit' defined / documented? An adequate definition can normally be provided by detailing:

• The interfaces by which the target interacts with the world outside the target to provide the its services.
• The services provided through those interfaces - other services using the same interfaces are excluded.
• Whether the assurance is of the external delivery of the service or the operation of internal mechanisms that implement the service.
• The layer within this software whose behaviour is to be analysed / tested.
• Software involved in implementing the layer that is excluded from the target.

Identifying the Interfaces & Services

The first two items, the interfaces and the services, are addressed by identifying a Service Delivering Assembly(SDA). SDAs are a way of providing a fairly unambiguous definition of a sub-set of a software system. They can be applied in a consistent way at all levels from the complete system down to the smallest function within the system.

See Service Delivering Assemblies for an explanation of this concept.

External Service Delivery & Internal Service Operation

Assurance activities are more effective if they are coherent. To ensure this the Assurance Target is either:

• The external operation of the services - the behaviour seen by the outside world at the interfaces, or
• The operations of the mechanisms within the assembly delivering the service.

In the external operation case the target denotes that analysis or testing should be concerned with anomalies in the behaviour of the service and service demands that threaten this. In the internal case the target denotes that the key mechanisms involved in delivering the service should be identified and analysis or testing should be concerned with anomalies in their operation and situations that threaten this.

The internal / external distinction results in activities of a different nature. For example if the SDA is the delivery of an account management service across a 3 tier client service architecture then:

• External will lead to system style testing from the user interface.
• Internal could lead to integration style testing of the client server and server to server interactions.

Identifying the SDA Layer to be Analysed / Tested

The SDA consists of all of the software delivering the services. One Assurance Target associated with the SDA may well be intended to analyse or test this overall behaviour. However years of experience have shown that a piece of software, an SDA, that is delivering correct service to its clients may be going wrong internally or may be on the edge. Eventually it will fail. To detect this it is necessary to test in more detail the upper layers of the SDA and how they use the infrastructure and services provided by other software within the SDA.

An Assurance Target defines the layer to be Analysed / Tested by identifying the internal interfaces between this layer and the rest of the SDA. When responding to a service demand involves one of the internal interfaces the analysis needs to consider what should happen at the interface and testing needs to check this. The definition of tests now define inputs and outputs at both the service interfaces and the internal interfaces rather than just at the service interfaces.

Exclusion of Software from the Target

The target refers to a Layer within the SDA. Software outside this layer is by definition excluded from the target. When a build is tested the software outside the layer can be experimental, prototype or stubbed by a test harness. The software outside the layer is only required to facilitate linking and testing.

The layer under test should be built from the software that will provide the layer when it is released for other uses. The layer is a sub-set of the SDA and in a similar way to the SDA includes all software that could ever be executed as it operates. However in reality some software used by the layer will be so far removed from the core software that variations in the implementation will have little risk of changing its behaviour In these cases it is safe to allow stubbing / replacement of this software if this eases test implementation.

An Assurance Target allocated to a test package should identify the elements of the SDA / SDA Layer that can be modified, replaced by stubs or simulated by a test tool.

The Difference Between Exclusion Behaviour from the SDA Layer and Excluding Software from the Implementation

Defining the SDA Layer for the target, by denoting internal interfaces, excludes the software outside the layer from the target. Why can't all software be excluded in this way? Why have a separate exclusion list for testing purposes? The reason is that analysis and testing are required to focus on the behaviour at the layer interfaces. Detailed analysis of what happens across an internal interface at all times is required. During testing outputs from an internal interface must be checked and any inputs defined by the test.

All this analysis / testing is too big an overhead if, for example, it is intended that a cryptography library that is not available yet can be replaced by a dummy plain text simulation during test execution. It is not intended that the inputs / outputs at the cryptography library interface be analysed in detail or checked in detail during testing and so its operation remains part of the layer.

Assurance Focus

The Assurance Focus incorporates the Assurance Target but provides more detail on what about it is to be analysed / tested. There are a number of perspectives that are involved in this, these are:

• The characteristics of the behaviour of the SDA / SDA Layer. This includes:
  • Behaviour observable at the SDA / SDA Layer service and internal interfaces that are of interest.
  • The level of resource utilisation placed on the SDA / SDA Layer infrastructure and on the processing system executing the software.
• The operating conditions to be addressed. That is:
  • The nature of the service demands arriving at the SDA / SDA Layer.
  • The condition and other demands of the infrastructure within which the SDA / SDA Layer operates.
• The types of failures that are of interest.

An assurance focus can constrain each of these to a greater or lesser degree. Generally some constraints are required to provide a coherent well understood package of work. Simply specifying the Account Management target with no constraint as the focus for an Assurance Analysis activity is too open ended - the work has too large a scope. Examples of better definitions would be:

• Account Management functional behaviour under minimal concurrent load.
• Account Management functional behaviour under high concurrent load.
• Account Management response time under high concurrent load.

Identifying the Observable Behaviour of Interest

No single scheme for identifying different aspects of behaviour is applicable to all systems. A scheme has to be created in each case but there are some generic areas that can be used as a starting point. These are:

• Functional Behaviour - generally if inputs are 1 and 2 is the result 3?
• Functional Consistency - given that general 1 and 2 gives 3 is this consistent or occasionally is the result 4?
• Response Time - how long does it take from the service request arriving to the response being provided?
• Direct Resource Utilisation - use of infrastructure and processing system & network resources that is directly associated with the operation of the SDA / SDA Layer.
• Overall Resource Utilisation - All of the resources used as a result of the SDA operating on the computer system whether directly with the SDA or within middleware or the operating system as a consequence of the SDA operating.

So, for example, a focus intended to address capacity issues could include response time and resource utilisation as the interesting behaviour.

Operating Conditions

As with behaviour there is no single scheme that can address this for all systems but again there are some generic areas that can act as a basis for a scheme. These include whether the following are to be addressed:

• Transaction paths.
• Application data ranges and data topology.
• Interactions between overlapping transactions that are identified in the service definition. For example if one transaction is updating an account and another one starts to update it then the service definition may require that the second one is rejected.
• Potential interactions between transactions arising through their use of the same application domain resources.
• General transaction concurrency.
• Incorrect usage / abuse of the service by clients.
• Resource exhaustion during operation.
• Failure of the infrastructure to provide the expected service to this SDA Layer.

Types of Failure

Again no single scheme applies but there are general classes that provide a basis for formulating a scheme. These include:

• Incorrect functional behaviour at the service or internal interfaces. Problems at the service interface mean that from the client point of view the SDA is not operating correctly. Problems at the internal interfaces mean that the SDA Layer is not working as intended.
• Abnormally high resource utilisation.
• Incorrect management of resource usage - using unsecured resources, failing to release resources.
• Abnormally high transaction times.
• Modification of data or memory that is not associated with SDA / SDA Layer.
• Lockup or failure of threads operating within the SDA / SDA Layer.

Using the Assurance Focus

The depth and rigour of an analysis depends upon the scope. A wide scope will lead to less depth and rigour. A narrow scope to an in-depth analysis done rigorously. Similar principles apply to the depth and rigour of testing. The Assurance Focus provides the manager with a means of identifying the scope for the analysis and testing. It provides control not only of what is considered but also of the depth and rigour of the coverage.