• Within the natural hierarchy of interactions. A higher level interaction say create an order will include conceptually sound lower level interactions such as complete order details form. So a Test-Interaction CompleteOrderDetails can be used as part of a CreateOrder Test-Interaction.
• To provide more checking and/or a wider view of the system. The CreateOrder Test-Interaction may operate on the HCI interface. It may then be used as part of CreateOrderAndCheckDatabase Test-Interaction.

These concepts are introduced in this section.

Example System

An example will be used to illustrate layering. The example is based upon a function to create user accounts within a diary system. When create account is selected by a supervisor a creation form is displayed. The form has a user-name field, a password field, a password confirmation field and an email address field. The form has a submit button. The submit button is enabled when all the fields contain data.

When the submit button is pressed the fields are validated checking that the passwords are identical. If correct the form is sent from the client to the application server. The server checks that the username is unique and the email address acceptable. If everything is ok then an account created message is sent to the client and the database updated. If there are problems a rejection message is sent to the client and the database remains unchanged.

The Test Interaction Architecture

Test Interaction Structure

A possible arrangement of Test-Interaction suitable for testing this function is illustrated below.

This structure incorporates a pattern in which each interaction has associated with it a driver / observer test-interaction and a checker interaction that uses it. The driver / observer aims to act and report. The check applies context independent anomaly detection. This pattern is useful because:

• The checker test-interaction can be used by other interactions & by tests, it provides checking of behaviour 'for free'.
• The driver / observer provides a clear description of what happened prior to the interpretation of any rules.
• The driver / observer can be used by different checking test-interactions where rules vary.
• The driver / observer can be used directly by tests, as opposed to test-interactions, to rapidly develop tests for new rules.

The structure contains three clusters of test-interactions:

• Those dealing with the operation of the creation form (01,02,03,04).
• The ones addressing submission of a creation operation to the system and the response given to the operator (05,06,07).
• Test-interactions focusing on the overall system level behaviour of the creation operation (08,09)

The clustering into domains of interest can be used as a basis for systematically determining the significance of anomalies to a test.

The Test Interactions

01 Field Updates

A field update test-interaction is provided for each of the fields. It accepts a value and enters it into the field. Having entered the value it leaves the field to trigger any event driven processing on the value. The value in the field is then captured and the behaviour determined. The behaviour is classified as:

• Field-Updated value accepted
• Field-Updated value modified

The report on the interaction incorporates the value present in the field after the update attempt.

02 Field Updates With Modification Checks

This interaction first checks whether the behaviour is the expected one given the value supplied. Assuming that the reported behaviour is correct the details of any modifications are checked against the rules for modifying values. Anomalies that are detected are reported. The behaviour is either undefined or:

• Field-Updated-Correctly

Notice that no distinction is made between an update that accepts the supplied value as is and an update that applies any modification rules, such as truncation, correctly.

If the Field Update interaction reports undefined behaviour or the reported behaviour was wrong or invalid modification has occurred then this component reports an undefined behaviour.

03 Enter Account Create Details

This accepts the four field values and updates the fields using the Field Updates With Modification Checks interaction. It checks the state of the submit button after updating the fields. The behaviour classification scheme is:

• Fields-Set-Submit-Disabled
• Fields-Set-Submit-Enabled

The behaviour is undefined if one of the field updates reports undefined behaviour. The report includes the four field values and the observed state for the submit button.

04 Enter Account Create Details And Check Form Behaviour

The state of the submit button is checked against the rules requiring that the button is only enabled when is data present in all fields. The behaviour classifications are:

• Fields-Valid-Submit-Enabled
• Fields-Invalid-Submit-Disabled

If the submit button is in the wrong state then the behaviour is undefined.

05 Press Submit And Wait For Response

This simple interaction activates the submit button and waits for a recognisable response from the system.  A timeout is implemented in case the system fails to respond at all. The behaviour is:

• Submitted-And-Response-Provided

If no response is recognised within the timeout then the behaviour is undefined.

06 Submit Account Create Via HCI

Values are provided for the Username, Password, Password Confirmation and Email-Address fields. The data in entered using the Enter Account Create Details And Check Form Behaviour interaction. Where this reports the behaviour as Fields-Valid-Submit-Enabled the Press Submit And Wait For Response interaction is operated. The behaviour classifications are:

• Submit-Rejected-Passwords-Not-Identical
• Submitted-To-Server-Reported-Creation-Rejected
  • Username-Not-Unique
  • Email-Address-Syntax-Error
• Submitted-To-Server-Reported-Creation-Accepted

This component is designed to be used with values that are valid for an attempt at submission not with values that cause the form to disable the submit button. To guard against this if Fields-Invalid-Submit-Disabled is reported after entering the values then it is classified as a testware anomaly and the test is aborted.

07 Submit Account Create Via HCI And Check Response

The password fields are checked to see if they match. If they match then if the reported behaviour indicates the submission was rejected because the password were not identical then an anomaly is raised and this interaction reports undefined behaviour. Similarly if they do not match then if the reported behaviour is not rejection due to the passwords not matching then an anomaly is raised and undefined behaviour is reported. Otherwise the behaviour reported to this interaction is passed on as is:

• Submit-Rejected-Passwords-Not-Identical
• Submitted-To-Server-Reported-Creation-Rejected
  • Username-Not-Unique
  • Email-Address-Syntax-Error
• Submitted-To-Server-Reported-Creation-Accepted

08 Create Account Operation With Database Audit

The content of the Account Table in the database is captured. A request to create an account is submitted using the Submit Account Create Via HCI And Check Response interaction. After the response the content of the Account Table in the database is captured. The behaviour reported is based on the state of the database and is:

• Table-Unchanged-Account-Not-Present
• Table-Unchanged-Account-Present
• Account-Added-To-Table-All-Others-Unchanged

Anomalies such as spurious deletion or modification of rows or an incorrect new row or additional new rows or duplicate entries cause the behaviour to be undefined.

This interaction is concerned with the operation of the create process rather than the interfaces that allows create requests to be submitted.

09 Create Account Operation With Consistency Checks

Checks are applied that compare the response received from the HCI with the state of the database.

	Table-Unchanged-Account-Not-Present	Table-Unchanged-Account-Present	Account-Added-To-Table-All-Others-Unchanged
Submit-Rejected-Passwords-Not-Identical	Valid	Valid	Anomaly
Username-Not-Unique	Anomaly	Valid	Anomaly
Email-Address-Syntax-Error	Valid	Valid	Anomaly
Submitted-To-Server-Reported-Creation-Accepted	Anomaly	Anomaly	Valid

Operator interaction anomalies are raised to identify a discrepancy between the response to the operator and the action taken by the system. The behaviour reported only addresses the impact on the system state and so echoes the behaviour received from the Create Account With Database Audit interaction:

• Table-Unchanged-Account-Not-Present
• Table-Unchanged-Account-Present
• Account-Added-To-Table-All-Others-Unchanged

Testing Using The Architecture

Examples of how different forms of testing can be built using the test interaction architecture described above are given below.

One target for testing could be the behaviour of the four fields in the form and the control of the submit button. At a simple level this splits into:

• The response of the individual fields as data is entered into them.
• The control of the submit button based on the values in these fields.

That said there are levels of rigour at which the form can be tested. The first level treats the data entry as a trivial atomic operation and then looks at the outcome at the end. The form is tested for faults related to the field values rather than to the way the data is entered. For a field this is a matter of setting the contents in a standard way, navigating out and then reviewing the result. For the form setting all four fields in a standard sequence and checks the submit button.

An implementation of this first level of testing using the architecture is shown below:

Execution of the test involves interactions 01,02,03,04. Anomalies within this domain are classified as test failures.

More rigorous testing may then start to consider the sequence of events that occur on the form. The next level to consider could be the order in which fields on the form are modified. If the status of the submit button is only changed when the email fields is updated then changing another field last can cause invalid functionality. There are various ways this could be addressed with a test-interaction, for example adding an update order parameter to interaction 03, but a dedicated test is also possible.

Here the test drives the Field Update interaction directly. Changes to fields are checked within the interactions but the state of the submit button is not. The test is responsible for checking this. Even here lower level building blocks used to access the button in interaction 03 and to check the rule in 04 can be reused by the test to avoid duplication.

Going one stage further problems could arise due to the way data is entered into a field, navigation and editing within a field and due to the way navigation between fields is performed. If testing at this level is necessary then the test-interactions are bypassed and tests are responsible for accessing the form and checking the outcome at each stage. Again though test interactions are not reused lower level building blocks that manipulate the form, obtain values and check rules are reused to avoid duplication.

Testing The Creation Of Accounts

At the system level the creation of accounts is reflected in the entries in the Account Table in the database. When testing this the indications etc. given on the HCI are secondary.

All of the test-interactions are involved in the execution of the test. Because of the focus on the overall operation of the system test failures are limited to anomalies in the system level behaviour domain - interactions 08 and 09. The exception to this are the discrepancy interactions raised in 09 as they are defined as relating to the operator interaction domain.

Testing The Operator Interaction for Account Creation

The configuration for testing the operator interaction and indication to the operator uses the full set of interactions but focuses on anomalies in this domain. This also include the discrepancy interactions raised in 09 not defined as forming part of this domain. This provides reuse of the testware that confirms consistency between what happened to the database and the indication given to the operator.