The impact of load the importance of response

Today’s systems, large scale, distributed, perhaps running out of the cloud, perhaps incorporating third party APIs, possibly exposed to an unpredictable user base, are very vulnerable to load related threats. When they are “busy” performance can start to falter and strange functional anomalies can occur. Even under “light” load the impact of more than one thing happening at the same time can cause things to malfunction. Ultimately, under “stress” they can grind to a halt. Problems caused by load are seen by most engineering teams.

Doing nothing or going through the motions

It is important to take an active and effective approach to addressing the risk from load. A cosmetic or last minute exercise means slipped dates and problems in production.

The flawed “final test” approach

As with most assurance, relying on a simplistic “all encompassing” test of “everything” just before go-live is a poor plan. Layers of activity, using different approaches, are needed to detect different types of problems. Work needs to be done during the formative and build stages of the project rather than at the end to make certain there is time to fix things. To be effective, assurance needs to run for the full duration of the project alongside the core deliver activities. It can’t be a bolt on.

The building blocks

A sound approach must encompass elements drawn, as appropriate, from the following disciplines:

• Profiling of the load in different operational periods
• Design vulnerability assessment
• Modelling of the working of the system and the volumes involved
• Localised workload-component and sub-system load testing
• Tactical testing of high risk scenarios
• Full breadth simulations of operation under load

The challenge? Failed attempts and wasted money

In a nutshell, delivering this type of assurance is hard. A lot of time and money has been wasted by organisations on ineffective work. Some teams can’t get tests to work at all. Some lack the technical depth to monitor, analyse and draw conclusions. Some get great results, on the wrong data. Some don’t analyse and understand work loads and user behaviour.

Why do these things happen? It is because the teams don’t have the right resources, the right concept of what needs to be done, the required technical depth and the understanding and credibility to get stakeholders to recognise what is involved in doing things correctly.

It is a job for the specialists

Time and time again it can be seen that this is a job for specialists. We have seen that attempts to “do it in-house”, using the development team, or to hand it off to a systems integrator or one of the big consultancies generally fail to deliver. On more than one occasion we have been called in to recover such situations. It is not a job for generalist test consultancies or suppliers, these organisation lack the necessary technical capabilities. It is a job that demands a multidisciplinary team with a history of delivering the full spectrum of tasks that will be required.

Why us?

We have the people, skills, experience and approaches needed to work with you to address your exposure to load related risk. We have delivered across a wide variety of solutions and organisation types. We have worked with “standard” website, high volume transactional portals, APIs, telecoms network elements, financial software products, ERP systems, CRM systems, provisioning systems and billing platforms.

We don’t just address end user activity. In our time we have tackled interface and integration throughput, period end processes, billing cycles, batch processing services, workflow engines, data transformation scripts, reporting services and Citrix systems. We have tested sub-systems and backend services, we don’t need a nice user interface to be able to operate. We don’t just do one hour performance tests, we emphasise the need for extended runs, to check for consistency and sustainability, and for running under stress, to look for weakness.

We have the experience needed to make this work.

We can offer

We don’t just test. We size, analyse, assess risk, target, test and remediate. Often we have to form the non-functional requirements. We help solve the problems we raise. We are not bound to standard tools, where needed we build custom drivers and harnesses to test the “untestable”.

We offer activities tailored to your specific needs. Whether you want a health check, advice on how to proceed, help fixing in-flight work, assistance with a specific performance issue, sanity testing, a full test campaign or oversight of a supplier’s activities, we can work with you to meet your needs in an effective and efficient way.