Introduction

The concept of multiple variant implementations of a particular test-interaction is introduced elsewhere ( see Alternative Implementations of Test-Interactions ). Simply conceptually organising a group of test-interactions together as described there provides for a more structured set of testware. When the interfaces are standardised more benefits are gained from reuse in the client structures.

Models of reuse are outlined here. Initially reuse simply comes in the form of things being similar so it is quicker to implement variants of the client testware. In a simple approach different variants of client testware are required to use different combinations of test interaction implementations. This is because the choice of implementation is hard coded in the client testware.

More advanced reuse approaches start to reuse client testware itself. A major issue to overcome is a combinatorial explosion when combining components. With no reuse in the client testware putting together a test with 8 test-interactions each with 2 variants could require the generation of 256 bits of testware to hang it all together. The more advanced reuse approaches are designed to address this.

Benefits From Basic Reuse

Combinations of Test-Interactions

If no specific methods are used to enable test-interactions to link together then tests will need a unique entry point for each combination of test-interactions to be used. For example suppose there are test-interactions to CreateAnOrder and DeleteAndOrder each with local client, browser and web-service implementations. A client test-interaction to test the creation & deletion integrity is required to create an order and then delete it. This is shown below.

Now if we want to try all combinations of test-interaction then there are nine versions of the CreateAndDelete to implement. This is without any variation in the way the CreateDelete interaction operates. This is shown below:

As the number of test-interactions increases this problem gets worse. There comes a point where the technique shown above - where each combination of worker test-interaction is called by a dedicated client implementation - becomes untenable. Not surprisingly it does not take long to reach this point.

Where this is a problem, which is most cases, the testware should adopt more sophisticated techniques to manage combining different implementations of a test-interaction. Subsequent sections describe candidate approaches. However prior to that a brief outline is given of the benefits that the test-interaction based approach gives in the simple situation illustrated above.

Inherent Benefits of The Test Interaction Based Approach

The test-interaction based approach results in:

• A standard pattern for interacting with any worker test-interaction. This involves setting the operational specification, operating the test-interaction and processing the report.
• An identical structure for reporting the conceptual parameters used, the behaviour observed and the key values of the interaction for all implementations of a particular test-interaction.
• The number of different control parameter sets being minimised with the use of common sets for a number of implementations.

As with any design and development exercise regular structures and common interfaces provide benefits. These come in the form of:

• Common patterns in any client implementation.
• Sharing testware implementation amongst client implementations.

Common Patterns In Client Testware

The first of these arises from the fact that any interaction with a worker test-interaction follows the same pattern and all interactions with implementations of a particular test-interaction are identical apart from the control parameters and the actual call to the implementations. These characteristics mean that all client testware implementations can be implemented in a similar manner. This allows the development of a common effective 'style' of implementation which can be applied quickly and reliably. The implementors of testware spend less time thinking about how to approach using worker test-interactions and make fewer mistakes.

Sharing Client Testware Implementation

The next level of reuse in this basic scheme arises when the implementation of client testware uses shared testware elements. This is possible because of the common patterns and common interfaces provided by the test interactions. For example suppose some additional checks are to be done after creating the order prior to deleting the order. These checks might involve comparing the relationships of values entered for the order against data reported by the test-interaction.

If the three create order variants are implemented as separate items with different interfaces then the checking has to be different in each case. As they are test-interactions with the same interfaces a standard checker can be developed. This standard checker can be used in each implementation of the create then delete order test-interaction. This reduces the amount of testware to be produced.

Configuring The Choice Of Implementations

When client testware either at the test level or a test-interaction is to use a test-interaction a particular implementation is used. At some point a choice is made. In the simple arrangement described above the choices of implementation is made when the client testware is implemented. The choice is hard coded and fixed with one implementation. If a test requires that the client test-interaction operates with another worker test-interaction implementation then this requires yet another implementation of the client test-interaction.

It would be better if higher level testware could in some way configure the implementation that a given implementation of a lower level client test-interaction will use. This would mean that unless there is an inherent requirement for different implementations of the client test-interaction then only one implementation is needed..

This section outlines some approaches aimed at providing this capability. Three approaches are outlined:

• Hardcoding multiple implementations of a test interaction in a single client test-interaction.
• Passing the choice of test-interaction implementation as a parameter.
• The client requesting the implementation from a catalogue that is configured to choose the appropriate implementation.

The approaches range from a fairly simple approach, one with limitations, to a powerful approach aimed at reducing unnecessary coupling between testware components in complex test systems.

The simplest approach is to encode a selector into the client testware that picks an implementation and provide a control parameter that allows selection of the implementation to use. This is illustrated below:

As shown above additional parameters have been added to the CreateDelOrd test -interaction. There is now only one implementation of this interaction supporting all nine combinations of creation and deletion This simple approach is effective in smaller systems particularly where the nesting of interactions is not too deep. It should be noted that this works best where the implementations have the same control interfaces. Variation in the control interface generally requires a different client.

In larger systems the limitations of the approach may require alternative approaches. The major limitations are:

• A test-interaction may have multiple client test-interactions after all the purpose in reuse. If another implementation of the test-interaction is introduced then all of the client testware will require changes.
• Additional complexity appears within the client testware as the choice must be made and the correct implementation called.
• All selection parameters need to be available to the top level testware. Each test-interaction must provide access to values of all of the selection parameters of all of the other test-interactions that could be used when it operates. For example a client test-interaction of CreateDelOrd would need to include:
  • The selection parameters to be passed to CreateDelOrd - CrtOrdImpl, DelOrdImpl
  • Any parameters they use to choose the implementation of CreateDelOrd or other worker interactions.
• Where there are multiple routes by which a lower level test-interaction can be used the selection parameters it requires may appear multiple times at the top level.

For small systems with few layers of nested test-interactions this approach offers significant benefits over simple hardcoded approach. However for medium sized systems with deep nesting the repeated transfer of selection parameters becomes confusing and could become detrimental.

An improvement on the selection method described above is to pass a link to the implementation rather than a selector. The immediate client testware does not need to encode the choice. The link allows it to locate the implementation and operate on it. Obviously this requires that all implementations passed to the client testware implement the interface that the client testware expects.

The advantages of this approach over the one described above are:

• Addition of another implementation of a test-interaction does not affect its clients as they have no knowledge of the individual implementations.
• The client testware does not have to implement selection logic. This simplifies the client.

The limitation it retains is the large set of implementation parameters that will appear in higher level testware.

This approach is illustrated below:

The most powerful approach to solving these problems is to use a selection service to provide the client test-interaction with a particular implementation on demand. The implementation is selected by applying a set of rules. These rules can be configured by higher level testware. The rules can vary from test to test. The selection provided to a given client test-interaction may be different when it is being used at different stages in a test or by a different client to itself. The architecture of this approach is illustrated below:

When a client is invoking a test-interaction it specifies the test-interaction and the control interface that it requires. The selection service does the rest. Selection can be based upon factors such as the client making the request and the operation sequence leading to the client. The rules can be used to select a specific implementation or a group of implementations from which an implementation can be chosen in either a random or a round robin sequence.

The advantages of this include those gained when passing the implementation as a parameter. That is:

• Addition of another implementation of a test-interaction does not affect its clients. The clients have no knowledge of the individual implementations.
• The client testware does not have to implement selection logic. This simplifies the client.

The major additional benefit of this approach is the elimination of the explicit passing of implementation links. The preceding approach involves passing implementation choices all the way from the top level test specific testware to the point where an interaction is invoked. The implementation to be used for EnterOrderDetails is passed to OrderLifeCycles which passes it to CreateDelete which passes it to CreateOrder. OrderLifecycle and CreateDelete have to provide the communication route.

The need to provide this communication from the top makes the testware difficult to understand and to maintain. If CreateOrder is modified to use another test-interaction, say CheckOrderExists, then all testware along all possible routes to CreateOrder will need to be modified. Each will have to be changed to communicate the implementation of CheckOrderExists that is to be used.

When a selection service type of approach is adopted the need for all of this parameter passing goes away. In fact with this approach only the immediate clients need know of the existence of a given test interaction. In the above illustration OrderLifeCycles does not have any need to know about CreateOrder, EnterOrderDetails and SubmitOrder. Similarly CreateDelete does not need to know about EnterOrderDetails or SubmitOrder.

The selection service based approach is the most sophisticated technique for selecting the implementation of test. It allows not only reuse of the lowest level test-interactions but also reuse of all test-interactions. It does this whilst also eliminating the overhead of explicitly communicating the choice of test-interaction implementations around the system.