What is technical assurance?

Technical, or “non-functional”, assurance is a wide ranging discipline. Activities focus on understanding and improving the Quality of Service of a solution and on targeting very specific technical or operational risks. These are not purely “test” activities, delivering them demands a mix of test and engineering skills.

Assuring quality of service

Quality of Service issues tend to emerge in the real-world if no action is taken to head them off. Issue with performance, reliability and service availability are common. Defenses against these threats can include load testing, soak testing, failure and resilience testing, reliability testing and compatibility testing.

Technical and operational risk

Systems or activities around changing a system can have very specific technical or operational risks. How long will it take to transform the data set? Does the application operate reliably on the new operating system? What happens if a data centre failure occurs? Aspects like this need carefully defined and precisely executed testing.

Too little too late

A common mistake is to attempt an “all encompassing” test of “the technical stuff” just before go-live. As with other types of assurance, a late, big bang, test is a poor plan. Assurance needs to study and influence the requirements and design, starting early, rather than simply test things to destruction.

A layered approach

Analysis, of what the system will experience and of where it is vulnerable, is as important as traditional testing. Layers of activity, using different approaches, each with a different emphasis are needed to detect different types of problems. Work needs to be done during the formative and build stages of the project rather than at the end otherwise it will be found that problems can’t be fixed. Fixes are technically infeasible, too expensive, can’t be done in time or would require too much re-testing.

Our experience and capabilities

At SQC we have many years of experience identifying quality of service and technical threats, of figuring out how to tackle them and of building the tools and the plans needed to test them. Our people are used to leading on these aspects, to leading the execution of tests and then to leading the work needed to find root causes and fix problems. They take accountability for driving technical assurance to a successful conclusion.

Our engineers work with teams, in a structured way, to crystallise meaningful non-functional requirements, to identify risk, to shape designs with sound non-functional behaviours and to help fix things that don’t work well. We convince the sceptics that, really, something down there isn’t working well. We help you to deliver a great Quality of Service and to avoid those occasional, but deadly, technical catastrophes.

The range of services we offer includes assurance of:

• Performance and throughput
• Graceful degradation under stress
• Endurance of operation
• Consistency of operation
• Functional reliability
• Resilience
• Fault tolerance

We do this using our well proven capabilities in the areas such as:

• Teasing out and crystallising non-functional requirements
• Identifying technical risks
• Assessing implementation vulnerabilities
• Load testing of complex systems
• Functional test automation
• Reliability testing
• Failure and resilience testing

• Hardened critical online services for organisations such as Vodafone Italia, Granada Learning, The Caravan Club, The Driving Standards Agency, The British Library, Teletext Holidays and the Charity Commission.
• Determined the root cause of show stopping, “unreproducible” bugs that system integrators’ own teams could not find.
• Recovered the operational health of a tier-1 telecoms provider’s CRM and billing system and created the conditions under which they could make changes without needing to cross their fingers and pray.
• Transformed systems that, at first, fell apart under moderate load into ones that withstand anything thrown at them.
• Performed rapid interventions where the client faced deadlines and was in crisis mode.
• Influenced the mindset and design decisions of development teams.
• Provide tactical testing working alongside developers to deliver in-development, “agile”, non-functional testing.
• Provided an essential, independent, assurance services to a strategic client. A service that has been in place since 2010 and that has assured the development and operational performance of two generations of their flagship online solution.
• Assured specialist service platforms and solutions with complex real-time requirements in the telecoms, industrial control and embedded systems space.
• Established an effective load tests lab for a Microsoft Dynamics AX2012 installation, a feat that defeats most attempts.

Specific capabilities

• An overview of services for load and performance testing
• An explanation of reliability testing for IT solutions

Case study material

• Performance engineering and load testing of a critical online service Case study - Testwise 2010 load testing
• Seven years protecting the customer experience Testwise Service Protection - A managed load test service