Is creativity overlooked in software testing?

I was recently asked by a fellow Test Engineer why I’d chosen a career that didn’t allow me to be creative; I studied graphic design and at one time produced music. I was surprised by what he said, I always thought the discovery and exploratory side of testing software to be creative.

I questioned him to why he thought testing wasn’t creative, and this ended up in a discussion on the way he was currently testing. He followed a very orthodox and procedural way of testing, which involves the predefined creation of procedural test cases that are directly a one-to-one mapping of the requirement’s document. The process then dictated manual execution of the test cases followed by recording/reporting of evidence and defects. A familiar and stringent SDLC process which is common in most legacy software based projects. Although he described it as being analytical, I had thought of it as being robotic in nature and something I would have put down as a good candidate for automation.

I explained creativity is more than just artistic abilities and how it takes creativity to successfully use different techniques and approaches, not limiting yourself to a single way of working. I myself consider exploring, investigating and experimentation as creative activities. An example would be the adoption of different personas to look outside the requirements and challenging what others may accept as truth.

As someone who manages a team in an agile environment, I like to encourage creativity when solving difficult problems and also assisting others to do the same. I highly encourage autonomy, giving freedom to enable a dynamic and progressive environment that encourages creative thinking within the testing effort. I’ve always felt strategies to be guidelines to encourage the thought process, but something that should be evolved and moulded into the squad in which it is to be adopted.

Here are some sample case studies/projects in which I feel creativity played a big role in finding solutions to difficult problems at Sky Bet.

In-house trading

A recent in-house project which I had worked on involved the development of a pricing engine which uses mathematical models and a series of inputs to create betting opportunities for the Sky Bet sportsbook. When work initially started on the pricing engine a risk was the validation of odds output from the model; incorrect odds can have an immediate impact on reputation and revenue. The initial proposal was to ask the in-house trading team to manually validate the odds by observing individual selections over the course of a fixture (e.g. a game of football) and compare against a trusted source. With multiple fixture scenarios requiring testing, more than 500 selections per fixture, ideally all would be independently validated. This approach would prove to be expensive, time consuming and prone to error.

The other approaches we looked into was visualisation. This approach proves best when working with complex data sets with interrelated data. With the visual approach in mind we formed a new solution that involved recording the prices for all selections from a trusted source (existing opinion feed). Then playing back this data through the in-house pricing engine with identical match scenarios and pricing variables. Then Plotting the two recorded outputs on a standard axis (e.g. x-axis time and y-axis price). With this approach we could immediately illustrate the deviations from the new pricing engine and the third-party pricing feed. This approach would speed up the process to identify irregularities in the pricing engine’s output compared to the trusted source (existing price feed). You can see from the image below that both the trusted source and the in-house pricing engine track parallel throughout the fixture and when incidents occur.

Smoke testing

Many people consider smoke testing to be a scripted or well-defined series of small checks which many deem to be the least creative activity in the whole test process. A current challenge we face at Sky Bet is smoke testing multiple releases per day across multiple products and services, therefore many would consider a streamlined set of predefined tests to be the only approach to tackle release/deployment activity.

Working with a colleague on an internal learning and development project we developed a rather creative solution for testing one of the core internal services. The service itself handles a large, continuous throughput of messages – any smoke test ran on a new deployment of the service would prove to be inadequate as the service itself would have processed a message before the smoke test would run/complete. To combat this, we instead developed a dashboard that actively monitors messages through service and change to a traffic light scheme on the dashboard.

Creative thinking

With the cases provided above to some common themed problems across testing, there are always opportunities to add an element of creativity into the solutions we provide. With the continuous advances being made in software development projects – the discipline of testing should also encompass new approaches to keep up with the demands of agile and rapid software development practices. Whether this comes from automated testing and a new approach which seems to have taken off in the testing world is monitoring – there are always ways to improve and streamline existing processes.

Creativity also has other benefits across teams when encouraged – it exercises peoples thought processes and gets them to move away from repeatable processes that become second nature. An identification of this can be seen in Adobe’s State of Create report. Creators at work feel 34% happier, 31% more fulfilled, and 29% more energised but only 4 in 10 describe themselves as creative.

There are also methodologies and experiments which can be used to assess creativity and an example of that would be Torrance Tests of Creative Thinking created by Ellis Paul Torrance, an American psychologist. It was first described to me as a creative alternative to the IQ test. Part of the test asks candidates to complete a picture from a starting shape. The exercise was run in an internal community of practice, here are the results created by some of our engineers. The starting shape can be seen to the left.

Creativity is a core skill which aids with essential problem solving, critical thinking and adaptability. Test Engineers need to be able to apply the skills they have in a creative manner to keep up with the ever-changing industry and being more resilient to increasing automation.