Helping Orbifold Create a Stable Code Base

Rewriting the code for a new blockchain software solution

Will it work?

Orbifold, a UK start-up, had developed their own blockchain software. They had built everything from the ground up, but like most start-ups, they lacked a solid QA capability. They had prioritised development over test but needed a partner when it was time to ‘get real’ on the testing front.

The setup

Testing a blockchain application, particularly at scale, is more complex than typical testing work. It requires deploying a network of multiple communicating ‘miner’ nodes, simulating the distributed nature of real-world blockchain use.

Orbifold could not manage this themselves, their testing had been limited to running the blockchain on a developer’s desktop. In effect, they had only done unit testing.

SQC’s DevOps Engineering team containerised the Orbifold build to allow Cloud Engineering to deploy a representative network of nodes into AWS Elastic Container Service (ECS), spread across different data centres. The test automation team developed a harness that could drive the blockchain’s client, allowing the injection of transactions into the chain.

This is an example of where SQC’s unique mix of deep technical skills and capabilities amplified SQC’s assurance capabilities. Allowing the test environment to be created and operated without any dependency on others. Allowing SQC to work autonomously.

Peering inside the machine

Testing a blockchain is a different prospect to ‘normal’ system testing. Whilst driving the chain is relatively easy, checking that it is working correctly requires a deep understanding of how it is meant to work and the ability to perform in depth analysis of the events and data structures emerging from its operation.

This type of complex testing challenge is the sort that SQC love to take on. SQC has a history of creating niche tools for specific testing purposes. Examples include tooling for testing non-deterministic embedded systems, protocol testing, validating large scale data transformations, utilities for time-travel insurance quotes, custom XML comparison tooling and, of course, testing this blockchain.

To support the testing, SQC defined clear analysis techniques and wrote utilities to allow them to apply those techniques consistently.

Establishing stability

To be able to help Orbifold evaluate the health of their product, SQC needed to start with a basic network continuously processing transactions. One that could remain in a stable operating state for extended periods, whilst maintaining data integrity across the nodes.

This was the “starter for ten”. Having set up the AWS ECS deployment option and built the harnesses to apply and respond to transactions on the chain, SQC built a small, twenty node, network and brought it to transactional life. SQC then waited to see how well it worked.

Unfortunately, it did not do too well. No surprise really as this was a testing exercise and, as Orbifold clearly acknowledged, the software had not been subject to much testing up to that point apart from unit testing, which is why they had engaged the services of SQC.

Initial runs exhibited instability across the network, communication issues, node crashes and block discrepancies between nodes. Clearly, there was a lot to be done.

Full diagnosis

Encountering this situation, most test organisations would have raised defects, reporting symptoms, and waited for the development team to respond. That is not the SQC way.

SQC initially set out to diagnose the root cause of each issue. Taking that diagnosis as far as identifying which feature of the chain protocol, or the code making up the reference implementation, was causing the problem.

The next logical step

The next logical step was clear. SQC wanted things fixed quickly and with certainty so that the work could move forward. Orbifold were short on development capability, so to allow things to move at pace, SQC took on fixing the bugs that had been found.

Significant changes

To use the term ‘bug’ to describe some of the things that SQC tackled is a bit misleading. Some of the issues uncovered were significant design limitations.

SQC’s rework spanned:

• Relatively simple fixes of obvious localised coding errors
• Adjustments to the ordering of operations to ensure data integrity was maintained under ‘noisy’ operational conditions
• Re-factoring to address issues that emerged as the length of the chain increased
• Architectural revisions to head off next-level issues

This was possible because the team had developed a deep understanding of how the chain worked and of its implementation.

Development experience

Orbifold benefited significantly from SQC’s software development capabilities. SQC’s founder’s development background means that there has always been an emphasis on the ability to work with code.

This was a polyglot capability, with large scale work having been done in multiple mainstream languages including C, C++, C#, Delphi Java, Python, SQL, Javascript and, in the case of Orbifold’s application, Typescript.

SQC regularly uses these capabilities to develop tools and utilities to support specific customer projects and to develop more generally applicable tools.

They have also been used when SQC has undertaken the development of production software for clients. Examples of this include real-time embedded systems software, home automation software and data migration suites.

In the end

SQC’s extensive work on stabilising the Orbifold blockchain took approximately five months. It had started with what was really a prototype, one that had only ever run on a desktop, but by the end it had been matured into a significant pilot network that displayed stable consistent chain operation.

Pull request #61 merged SQC’s scalability changes into the Orbifold master repository as the basis for future developments.

Orbifold.io is a small UK based company focused on creating an effective blockchain for the capital markets.

Orbifold’s vision is that capital markets liquidity can be enhanced and complemented through an easy to use specification language for financial instruments, be they loans, securities, foreign exchange transactions or even OTC derivatives.

By using an Orbifold blockchain implementation customers get the benefits of a shared settlement ledger and readily available peer-to-peer trading.